EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Windows Explorer 4K ReadFile events

Also by EldoS: Rethync
The cross-platform framework that simplifies synchronizing data between mobile and desktop applications and servers and cloud storages
Posted: 03/13/2015 09:05:12
by Paolo Manili (Basic support level)
Joined: 02/23/2015
Posts: 16


We are developing a client for our cloud sync solution, and show the user his data feeding CBFS files and folder structures from our database. We only download files when they are required. However, we are encountering a problem.
We are using CBFS mounting the volume as a CBFS_SYMLINK_MOUNT_MANAGER on a folder on the user's disk.

How can we download the files seamlessy (Without failing operations, or stalling explorer) on demand when the user requires them.

When a user of our system opens a file, we have to fetch it on the fly, hopefully not stalling explorer or other applications while we fetch.
Quite similar in concept to how OneDrive handles these scenarios.

We had initially resorted to giving all the "remote" files a special file extension, that would be associated with an executable made by us that would just relay the download call and open the right program (Word for example) once the file was actually available.
This was not an ideal solution as it is not very transparent for the user, and operations like copy and move outside of the volume would fail as the file isn't actually available yet.

We then discovered that we could postpone ReadFileEvents on unavailable files by returning 0 BytesRead in the callback (and beginning to download the file in the meantime, returning the expected data when the file became available). This solution works comparatively well, it looks native, survives copy and move operations, and doesn't appear to freeze explorer either.

This solution has a problem,
we have noticed that when browsing a directory (say for example, highlighting files) Explorer preventively performs a ReadFileEvent requiring the first 4096 bytes of each highlighted file (Or the entire file, for files smaller than that). In our scenario, this entails starting and downloading each file that get's highlighted even by mistake.

Summing it up:
How can we distinguish this kind of read operation, or how can we prevent it?
Especially, are there any other "smart" solutions that have escaped us for similar scenarios?

Thanks in advance for any help you can provide,
Paolo Manili
Posted: 03/13/2015 11:31:40
by Eugene Mayevski (EldoS Corp.)

Unfortunately your question has no positive answer. There's no way to prevent Explorer from reading all files.

The only possible "smart" approach known so far is to inspect the originating process name, and IF it's Explorer AND IF the read operation is for 4096-byte-block, THEN block the operation somehow and not start file download.

Sincerely yours
Eugene Mayevski
Posted: 03/13/2015 14:11:29
by Paolo Manili (Basic support level)
Joined: 02/23/2015
Posts: 16

OK, I see, we have tried the Process name matching and BytesToRead matching (also specifying Position 0) to avoid starting downloads, and true enough, given Microsoft has great sense of humor... sure enough Explorer then tries to read 32K, and if one catches those it tries 2 bytes... so it obviously can't be stopped cold.

We also caught DLLHost doing the same operations (I guess related to thumbnail creation).

Ok, thank you very much, I think we will try and make those 4K and possibly the thumbnail data available before hand through caching.

Sincerely yours,
Paolo Manili

PS- I will post back if I find a better/more effective solution to these issues.
Posted: 03/13/2015 15:13:01
by Eugene Mayevski (EldoS Corp.)

Thank you very much for sharing your experience - even negative result helps other users avoid wasting time on re-checking the things you've checked already.

Sincerely yours
Eugene Mayevski



Topic viewed 1478 times

Number of guests: 1, registered members: 0, in total hidden: 0


Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!