EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Delete Folder (files inside) fast ?

Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.
#21526
Posted: 09/14/2012 04:32:43
by ttran Tran (Priority Standard support level)
Joined: 08/20/2012
Posts: 35

Hello,
In our remote Server, given an user account, if a folder contains files then there is a command to delete this folder, and all files belonging to this folder will go way very very fast.

Back to CBFS, in case we choose a folder to delete in the virtual disk, the CbFsEnumerateDirectory will scan all files belonging to this folder and to put these file information in the context. After that, the delete command is raised to each file. As a consequece, to delete a folder requires to send many delete commands to the remote server, instead of only one folder delete command.

To improve these problem, we have an idea as following:
as soon as a folder-delete-command arrives via one of CBFS callback event, we clean in our own logic-data-structure all files belonging to this folder. The delete folder command is sent to the remote server to delete the folder and all files belonging. The CbFsEnumerateDirectory event also can not find any files belonging to this folder when scanning through our own logic-data-structure.

Is that an acceptable solution ? or smt better ?

Thanks,
Thang TRAN
#21527
Posted: 09/14/2012 04:51:21
by ttran Tran (Priority Standard support level)
Joined: 08/20/2012
Posts: 35

this way, may be we need NotifyDirectoryChange
#21531
Posted: 09/14/2012 05:21:39
by Volodymyr Zinin (EldoS Corp.)

The OnDelete callback for the folder only occurs if the folder is empty (the OnDelete callback has been already called for all files/subdirectories inside this folder) and the OnCanFileBeDeleted callback for the folder has returned TRUE. So there isn't necessity to call NotifyDirectoryChange (because CallbackFS already reckons that there is no any files inside the directory).
NotifyDirectoryChange is only necessary to notify CallbackFS about changes that CallbackFS can't know. For example some file or directory which you represent/mirror on a CallbackFS virtual disk has been changed or deleted unexpectedly on the remote server.

I think it's ok if you cache all file/subdirectory deletions locally and pass these deletion commands to the server asynchronously with some delay. But if the the OnDelete callback for the parent directory occurs then just send only this one delete folder command.
#21532
Posted: 09/14/2012 05:29:27
by ttran Tran (Priority Standard support level)
Joined: 08/20/2012
Posts: 35

Hello, thanks!!

Ok, "the OnDelete callback for the folder only occurs if the folder is empty" is the MUST to delete a folder.
So when the user press Delete on a folder, how do we from the CBFS program discover this, in order to clean our own files (belonging to the folder) information in our own logic-data-structure ?

Thanks
Tran
#21543
Posted: 09/14/2012 10:55:59
by Volodymyr Zinin (EldoS Corp.)

Folder isn't allowed to be deleted while it isn't empty. All programs which allow to delete non-empty folder first enumerate and delete its files and subfolders. And only if it's successful then the folder will be deleted.

Quote
ttran Tran wrote:
So when the user press Delete on a folder, how do we from the CBFS program discover this, in order to clean our own files (belonging to the folder) information in our own logic-data-structure ?

Cache all file/subdirectory deletions locally and pass these deletion commands to the server asynchronously with some time delay. But if the the OnDelete callback for the parent directory occurs then just send only this one delete folder command (of course this command should be able to delete a non-empty folder on the server).
#37646
Posted: 09/07/2016 08:42:26
by Redivo Ricardo (Basic support level)
Joined: 09/07/2016
Posts: 2

I have the same question with folder deletion because we want reduce the network load because of many server requests.

When it is not possible to get a callback info for the ongoing deletion sequence inside a directory, my question is the following:

How do we know, when a deletion sequence of subfolders and files inside a deleted folder is finished?

If we can't get this information, it is not possible to cache the OnDelete callbacks. We don't know when to send the request to the server and which folder is relevant. When we try to delete every folder, the deletion of only one file would not renecognized and cached forever.
#37648
Posted: 09/07/2016 09:31:18
by Eugene Mayevski (EldoS Corp.)

Quote
Redivo Ricardo wrote:
How do we know, when a deletion sequence of subfolders and files inside a deleted folder is finished?


There's no way to know this - the user could have initiated deletion of the folder itself, or of only half of the files in it, and you never know.


Sincerely yours
Eugene Mayevski
#37649
Posted: 09/07/2016 09:52:51
by Redivo Ricardo (Basic support level)
Joined: 09/07/2016
Posts: 2

Idea:

The first call on onIsDirectoryEmpty is checking the top folder and traversing downwards. Is onIsDirectoryEmpty only called for pre-deletion-check of folders?

When so, it would be possible to create tasks for server folder deletion requests from here (first deletes root). The following callbacks for the child folders must ignote server deletion when parent was already deleted.
#37663
Posted: 09/08/2016 09:06:19
by Volodymyr Zinin (EldoS Corp.)

The OnIsDirectoryEmpty callback is also called when Windows checks whether folder (and its contents) can be placed to the recycle bin.
#37664
Posted: 09/08/2016 10:13:12
by Eugene Mayevski (EldoS Corp.)

It's ok to use OnIsDirectoryEmpty callback as a guide given that you don't rely on it as an absolute indicator. All in all, the user can just interrupt file deletion in the middle of the operation.

So the smart strategy would be to commit any "pending" deletions if there are no more deletions in the directory hierarchy for some time (say 5 seconds). While time is not 100% definite indicator as well (the system could have paused deletion, e.g. asking the user about deletion of some read-only file), this approach would save you a fair number of network transactions.


Sincerely yours
Eugene Mayevski
Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.

Reply

Statistics

Topic viewed 1946 times

Number of guests: 2, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!