EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Metadata cache

Also by EldoS: SecureBlackbox
200+ components and classes for digital security, signing, encryption and secure networking.
#17823
Posted: 10/11/2011 16:16:49
by Ben Schwarz (Standard support level)
Joined: 08/27/2011
Posts: 19

The metadata cache works pretty well for us and reduces the # of OnGetFileInfo callbacks by a factor of 6-8x.

But I have noticed there are still some sequences where we get the same GetFileInfo request on a filename many times in a row (10+) with no other callbacks occurring between them. Is this expected in some cases?

Also, are any features of the cache (e.g., size) configurable by our usermode filesystem?
#17833
Posted: 10/12/2011 08:15:49
by Volodymyr Zinin (EldoS Corp.)

The cache can contain up to 256 elements. So if a file has been opened and then closed and more than 256 other files have been opened-closed after that then the metadata (file attributes, last access time, last write time, etc) for the first file is removed from the cache. Maybe this is the reason of the behavior you observe. But if you gave me a Process Monitor logs for this situation I would try to find the real reason.

Quote
Ben Schwarz wrote:
are any features of the cache (e.g., size) configurable by our usermode filesystem?

Only a possibility to disable the cache.
#17848
Posted: 10/12/2011 14:01:24
by Ben Schwarz (Standard support level)
Joined: 08/27/2011
Posts: 19

Attached is an image showing the process monitor log on the top, and the bottom is a corresponding snippet from the log of the callbacks that our usermode FS received. In this example we received 5 GetFileInfo callbacks for "tools\x86\ui" in a row.


#17878
Posted: 10/14/2011 09:08:49
by Volodymyr Zinin (EldoS Corp.)

It seems the problem is the following -
during the call CreateFile("\Device\000000b3\tools\x86\ui\SwDRM.dll") the CallbackFS driver tries to obtain information for the path "TOOLS\X86\UI", but because it doesn't exist the GetFileInfo callback returns an error and the information for this folder isn't added to the metadata cache. It causes that the GetFileInfo callback is called again for the subsequent requests somewhere connected with this folder. Why are there a lot of such requests in the logs you specified? Perhaps the reason is because there are some file system filter drivers (antiviruses, etc) above the CallbackFS driver and they also perform some operations during the original CreateFile request. Or maybe the previous CreateFile("D:\cb\tools\x86\ui\SwDRM.dll") call which returned STATUS_REPARSE caused it.
In any case we will take it into account and will try to minimize the quantity of the GetFileInfo callback if it's possible.
#17880
Posted: 10/14/2011 13:53:17
by Ben Schwarz (Standard support level)
Joined: 08/27/2011
Posts: 19

So if a GetFileInfo call does not find a file -- e.g., "FileExists" is set to false -- then it is not cached? For reference, 99% of the GetFileInfo calls I see do not find a file. I think this would help us greatly to be able to cache these.
#17897
Posted: 10/17/2011 02:57:47
by Eugene Mayevski (EldoS Corp.)

Yes, at the moment it doesn't get cached. We will implement this in the next build.


Sincerely yours
Eugene Mayevski
#17901
Posted: 10/17/2011 04:20:24
by Ben Schwarz (Standard support level)
Joined: 08/27/2011
Posts: 19

Great! -- Thanks much. Looking forward to it.
Also by EldoS: Rethync
The cross-platform framework that simplifies synchronizing data between mobile and desktop applications and servers and cloud storages

Reply

Statistics

Topic viewed 1358 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!