EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Minimizing read in Windows Explorer

Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.
#16639
Posted: 06/11/2011 20:14:46
by Ivan P (Priority Standard support level)
Joined: 04/11/2011
Posts: 66

Hi there,

I have a similar question.

Is there any way to reduce number of requests for Windows Explorer?
I understand that application can do what it wants, but I can see strange behavior and I'm not sure whether it's caused by the way Win Explorer works or by some other filesystem-related features.

We are working with a remote HTTP storage, and non-sequenced reads aren't welcome (since it might require reconnect). And working with it in Windows Explorer causes troubles. I have read that mounting drive as a network share may help, but it doesn't in this case.

Attached is a sequence of calls while doing in Windows Explorer on Win7 next actions:

1. Open file properties (rightclick and select Properties)
2. Open file itself with standard windows Image Preview.


The strange behavior occurs when I open file properties:

Code
OnReadFile \24012009425.jpg:1318224. Offset: 4, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 2, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 6, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 12, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 20, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 118, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 20, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 22, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 34, bytesToRead: 4096
OnReadFile \24012009425.jpg:1318224. Offset: 46, bytesToRead: 4096


Requests are made for 4K bytes while incrementing offset only by several bytes.

When image is opened with Preview, Explorer then does the similar sequnce of reads, but buffer sizes are much more reliable

Code
OnReadFile \24012009425.jpg:1315228. Offset: 4, bytesToRead: 2
OnReadFile \24012009425.jpg:1315228. Offset: 2, bytesToRead: 33
OnReadFile \24012009425.jpg:1315228. Offset: 6, bytesToRead: 6
OnReadFile \24012009425.jpg:1315228. Offset: 12, bytesToRead: 8
OnReadFile \24012009425.jpg:1315228. Offset: 20, bytesToRead: 2
OnReadFile \24012009425.jpg:1315228. Offset: 118, bytesToRead: 4
OnReadFile \24012009425.jpg:1315228. Offset: 20, bytesToRead: 2
OnReadFile \24012009425.jpg:1315228. Offset: 22, bytesToRead: 12
OnReadFile \24012009425.jpg:1315228. Offset: 34, bytesToRead: 12


If I mount the storage as Network share:
Code
AddMountingPoint("X:;MYCOMP;X:", CallbackFileSystem.CBFS_SYMLINK_NETWORK | CallbackFileSystem.CBFS_SYMLINK_NETWORK_ALLOW_MAP_AS_DRIVE, null);


the situation is not changed.


Can any filesystem parameters affect such behavior?
Are there any recommendations when mounting disk for better work especially in Windows Explorer (other than file cache) to minimize random reads?


Thanks,
Ivan


[ Download ]
#16641
Posted: 06/11/2011 23:42:59
by Eugene Mayevski (EldoS Corp.)

I'd suggest asynchronous caching of the file if it's opened by Explorer (you can always check originator process name in OnOpenFile). This is the only way to improve performance with Explorer.


Sincerely yours
Eugene Mayevski
#16650
Posted: 06/13/2011 11:32:34
by Oleg Savelos (Standard support level)
Joined: 08/25/2008
Posts: 21

Here is my cached stream implementation.
Its still have some bugs and needs some improvement but perhaps it can get you a genneral idea on hwo to improve performance with read ahead and caching.


[ Download ]
#16661
Posted: 06/14/2011 10:38:17
by Eugene Mayevski (EldoS Corp.)

Ivan, we have observed the same behavior in our SFTP Net Drive product, and this is clearly Explorer's bug. I suggest that you report the bug to Microsoft. We will do the same, but the more different reports are made, the better.


Sincerely yours
Eugene Mayevski
#16811
Posted: 06/24/2011 09:11:54
by Jeff Mancuso (Standard support level)
Joined: 06/09/2011
Posts: 6

Ivan -

You'll need some sort of data cache - Explorer is certainly not the only app which behaves badly. And it's not just reads and writes of data. Some apps enumerate the directory hundreds of times for simple operations.

-Jeff
Also by EldoS: CallbackDisk
Create virtual disks backed by memory or custom location, expose disk images as disks and more.

Reply

Statistics

Topic viewed 1523 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!