EldoS | Feel safer!

Software components for data protection, secure storage and transfer

File attribute caching issue

Also by EldoS: CallbackRegistry
A component to monitor and control Windows registry access and create virtual registry keys.
Posted: 02/24/2009 05:32:39
by Vijay Mani (Standard support level)
Joined: 03/09/2008
Posts: 29

We're using cbfs 2.5 beta release, and ran into a really interesting caching issue.
Basically in our virtual file system, any direct FS ops work fine.
However when the change comes from a server thread, and we update the file attributes, the CBFS calls still request wrong sizes. Let me explain with an example.

Say we have a doc file of size 3250 bytes in our virtual FS.
We can open this in word just fine. The read callbacks show that word is trying to read all 3250 bytes.
Now there's an edit from the server, changing the size to 4000 bytes.
We then touch the file to indicate that it has changed.
The FS layer makes additional calls to getAttributes which return the new size of 4000 bytes.
However when we try to open the file from word again, word only issues a read of upto 3250 bytes.

We tried touching the file, releasing unused file handles.
We had not seen this issue in the earlier versions.
Please let me know if you need further explanation.
Posted: 02/24/2009 05:55:16
by Volodymyr Zinin (Team)

Use for your purpose the NotifyDirectoryChange method. But use it only from the latest 2.5 build. It has been significantly modified here. The name of the function is slightly incorrect. It should be something like NotifyChangeFileOutside or ReportExternalFileChange. We are going to correct it in future.
Below is an additional information about the function (we are going to add it in the documentation):
The function notifies the system about the specified file changing and the system then notifies about it interested components/programs (for example a program that uses win32 API ReadDirectoryChangesW). If the file is opened at the time of the notification or if its children are opened (additional streams or files in the case if the notified file is directory) then the functions at first tries to flush and release them. If it is not possible then handles for the opened files (i.e. those ones that the CreateFile function returns) are marked as "bad" and any future operations on them (except the CloseHandle request) return an error.
Also it is possible to call the function from the user mode callbacks but in this case it returns immediately (GetLastError() returns ERROR_IO_PENDING) and will be called later internally. It isn't possible to obtain the status of such asynchronous call.
Posted: 02/24/2009 07:36:47
by Vijay Mani (Standard support level)
Joined: 03/09/2008
Posts: 29

As you suggested I upgraded to the latest version, and made the necessary code changes.
That still didn't seem to work.
Bascially when the file is modified from the async thread we update the file attributes then call NotifyDirectoryChange on the file's virtual path with action Modified.
And in our case word still tries to read as if the file hasn't been updated. (i.e using the stale size) (We made sure getAttributes, returns the new size, the updated last modified time stamp)
Any ideas?
Posted: 02/24/2009 07:54:47
by Volodymyr Zinin (Team)

Hi Vijay,

Do you call NotifyDirectoryChange from the async thread after the file was modified? If so does this call return any error?
Also the path to the notified file must start from the backslash (i.e. the root directory) and must include file name. Is it so?
Posted: 02/24/2009 08:14:09
by Vijay Mani (Standard support level)
Joined: 03/09/2008
Posts: 29

Argh, it was 87 invalid param.
Slash Slosh error.
Thanks for the quick responses. You rock Vlad :-)
Posted: 03/22/2009 18:56:17
by Vijay Mani (Standard support level)
Joined: 03/09/2008
Posts: 29

Upon calling NotifyDirectoryChange on a file, clearly a blocking call is made to close the file handles. But is that the only FS call that could be made? Can we make any assumptions that perhaps read(or any other FS call) will not be called as part of the blocking call?

Posted: 03/23/2009 01:32:18
by Volodymyr Zinin (Team)

Such situation occurred because NotifyDirectoryChange was called for an opened file. In this case the file just forcibly closed. So only the OnClose callbacks are called. Programs that had opened the file (and have handles to it) still will have handles but these handles will be invalid and only CloseHandle call for them will be successful.
Posted: 03/23/2009 09:52:14
by Sid Schipper (Standard support level)
Joined: 03/14/2008
Posts: 285

I have this same issue. I'm not quite sure how to solve it in my caser, I think I just have to think about it some more, let me tell why my case may be a little more difficult.

My application is a client server application, where the server is a Windows service that I start at login and it communicates with the clients using named pipes. Unfortunately, any application can see my virtual disks, they may not be applications that even know about my system, such as Windows Notepad, Word, etc.

It is also possible to communicate with my database system using an external interface that we provide that can modify the virtual files on my disks. I'm thinking that the external interface to my database system needs to communicate with the Windows service when it modifies a virtual file, and the Windows service would then look up the correct callback file system pointer that it is keeping and use it to call the NotifyDirectoryChange function. Can that function be called from outside the callback routines? If so, then I think I'm OK, but if not I'm in trouble.
Posted: 03/23/2009 12:52:09
by Volodymyr Zinin (Team)

Yes it can. It is absolutely safe to call it outside the callbacks.
Posted: 03/23/2009 16:32:54
by Sid Schipper (Standard support level)
Joined: 03/14/2008
Posts: 285

OK, I tried something and it didn't seem to work. After thinking about it, it turns out that the easiest place for me to call this function is from within one of the callbacks, namely the GetFileInfo callback.

So, what is happening now is this, I see the file in explorer with a file size of 32K. I then use my external interface to my database system to truncate the file to 1K. I then go to explorer again, where the file size shows as 1K, and that is because I pass back that 1K value as the *EndOfFile parameter in the call to GetFileInfo, where I also call NotifyDirectoryChange.

But, when I right click and choose Open With notepad, the call to CbFsReadFile still tries to read 32K bytes. I'm thinking that I'm not executing the NotifyDirectoryChange in the right place. What do you think?

Where do you think would be the best place for me to call that function. It is difficult or maybe even impossible for me to call it from my application when I do the truncate and that is because that application interfaces with a database system that may not even be on the same system as the CbFs system. My application has no way of knowing who is looking at the database system using CbFs, so it is up to the CbFs application that is actually looking at the file to notify the Windows system that it is on that the file size has changed. Do you have any suggestions as to where I should do that notification? It seems like doing it in the GetFileInfo callback is not working, although the return code from NotifyDirectoryChange is TRUE.
Also by EldoS: CallbackRegistry
A component to monitor and control Windows registry access and create virtual registry keys.



Topic viewed 25082 times

Number of guests: 1, registered members: 0, in total hidden: 0


Back to top

As of July 15, 2016 EldoS business operates as a division of /n software, inc. For more information, please read the announcement.

Got it!