EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Cache Management Policy

Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.
#10266
Posted: 06/03/2009 19:10:58
by Sangmin Lee (Standard support level)
Joined: 06/03/2009
Posts: 57

Dear developers,

I'm studying Callback FS as an alternative in out project.
We're developed a clustered FS like google FS or Hadoop.
We'd like to deploy our client FS on Windows.

What I want to know is a cache management policy of Callback FS.
The kernel driver of FUSE in Linux manages metadata caching in time basis.
Once clients open a file, file information takes from our metadata server to client and remains caching by timeout after closing.
If clients re-open same file, then file info takes from cache.
If file's modified before re-open, FUSE kernel driver takes it from metadata server again.

How's the kernel driver of Callback FS?
It's the most important factor whether we choose or not.

Peace be with you

Sangmin
#10270
Posted: 06/04/2009 03:10:29
by Volodymyr Zinin (EldoS Corp.)

Hi,

CallbackFS also caches the metadata information (file size, last-write time, last-open time, etc) for recently opened files. But it's possible to either completely disable this cache (by means of the DisableMetaDataCache API) or tell CallbackFS only about changing some files (using the NotifyDirectoryChange API). The last case is more powerful. If a notified file is currently not opened and its metadata is in the cache then the metadata is removed from it. But if the file is opened then all handles to it are marked as bad (and only the CloseHandle calls will be successful after) and the metadata are also removed from the cache.
If you need more information about NotifyDirectoryChange, look for it in this forum. There were several discussions here. For example: http://www.eldos.com/forum/read.php?FID=13&TID=1634
#10533
Posted: 07/06/2009 21:12:52
by Sangmin Lee (Standard support level)
Joined: 06/03/2009
Posts: 57

Thanks for your reply.

Sangmin
#10578
Posted: 07/14/2009 03:04:40
by Sangmin Lee (Standard support level)
Joined: 06/03/2009
Posts: 57

One more question.

You wrote;
CallbackFS also caches the metadata information (file size, last-write time, last-open time, etc) for recently opened files.



I'm designing event handlers to be connected with remote metadata server.
I'm not sure what to put to FileHandleContext on CreateFile event handler.
In sample Mapper code, HANDLE from CreateFile puts to it.

But, in our system, file creation is performed at remote server not local system.
Therefore, there's no a HANDLE.
It may mean file metadata is not in cache. Right?

FileHandleContext should always contain HANDLE?

Hmm....

Sangmin
#10580
Posted: 07/14/2009 05:32:03
by Volodymyr Zinin (EldoS Corp.)

You can save in the FileHandleContext any information. Define some structure that will contain some necessary information about a remote file and then allocate this structure in the OnCreate/OnOpen callbacks and free it in the OnClose one.
For example:
struct MyFileHandleContext {

string FileName;
string ServerName;
...
}

void OnOpen(..., FileName, FileHandleContext, ...)
{
if (FileHandleContext == NULL) {

FileHandleContext = malloc(sizeof(struct MyFileHandleContext));
FileHandleContext->FileName = FileName;
FileHandleContext->ServerName = ...;
....
}
}

Then in the other callbacks (OnRead, OnWrite, etc) you can use the information saved in FileHandleContext.
#10581
Posted: 07/14/2009 05:45:14
by Volodymyr Zinin (EldoS Corp.)

Quote
Sangmin Lee wrote:
But, in our system, file creation is performed at remote server not local system. Therefore, there's no a HANDLE. It may mean file metadata is not in cache. Right?

No, it isn't. Because the cached meta-information for a file (file size, file attributes, etc) is that that you return fr om the OnGetFileInfo callback. And it's up to you where you obtain it. It depends on the callbacks implementation.
#10586
Posted: 07/15/2009 00:50:28
by Sangmin Lee (Standard support level)
Joined: 06/03/2009
Posts: 57

I got it. Thank you~

One more;

Of input arguments of OnEnumerateDirectory event handler,
what 'Mask' and 'Index' mean?
I looked into sample codes, but didn't get any idea.

Sangmin
#10588
Posted: 07/15/2009 02:03:16
by Volodymyr Zinin (EldoS Corp.)

'Mask' means the file name with the wild characters (i.e. an asterisk (*) or a question mark (?)) specified during enumeration. For details see the description of FindFirstFile in MSDN.

'Index' means the first file by index in the enumerated directory which the enumeration must be started from. But it isn't necessary to support this feature. There are a lot of file system (one is NTFS) that don't support it because files in them don't have strict directory position. You can return ERROR_NOT_SUPPORTED if the Index >= 0.
#10609
Posted: 07/16/2009 21:54:22
by Sangmin Lee (Standard support level)
Joined: 06/03/2009
Posts: 57

One more;

'FileName' and a file name returned from CbFsFileInfo->get_FileName() are just a name of the file or a full path name like Z:\test\a\b\c ?

Sangmin
#10610
Posted: 07/17/2009 00:28:07
by Volodymyr Zinin (EldoS Corp.)

File name is a full path name that starts from the root directory (i.e. it doesn't include a mount point). For instance: "\test\a\b\c".
But access to the virtual disk is only possible via mount points (for instance "Z:"), which are created by means of the AddMountingPoint call.
Also by EldoS: SecureBlackbox
200+ components and classes for digital security, signing, encryption and secure networking.

Reply

Statistics

Topic viewed 7455 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!