EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Database connection

Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.
#12437
Posted: 02/15/2010 12:47:48
by web dev (Basic support level)
Joined: 07/14/2009
Posts: 3

I have to connect our database with the Callback Filesystem and have done some first steps with the CbFs.

But I also found out that it is not so easy to interface the "slower" database queries to a file system. On the one side I want the quick response for enumerating the directories and on the other side I want the current elements and states for the directories from the database.

One possibility is to cache the requests for CbFsEnumerateDirectory and CbFsGetFileInfo, but I have no clue for the right time to synchronize with the database.

Has anybody made similiar experiences? Please share some ideas and tips.
#12490
Posted: 02/19/2010 00:02:25
by Chris Schirlinger (Standard support level)
Joined: 02/18/2010
Posts: 38

I have similar question.

From what I can see, there is extensive use of EnumerationContext's (IntPtrs to objects holding folder/file information) which speeds up subsequent calls to the folder structure.

What I can't figure out is how to populate these initially (with, say, the first two levels of the folder structure) and then populate on the fly further directories as the user open them up.
I don't really want to generate several 1000 folders and 10's of thousands of fake files when the user may look at a few 100 only.

I assume this is done in CbFsEnumerateDirectory, but the sample makes no reference to FOLDERS (only FindVirtualFile) nor can I work out how to add "missing" folders here. The samples add folders in the Storage Init but apparently creating them via normal File system access processes

Basically we have a tree of data, with "files" in it, each with a unique ID which I'd also want to store (I see there is a commented out index in the ENUM_INFO which I imagine would be useful.)

Unfortunately, I cannot step through anything (32bit VS2008 on 64bit Win7, running the program through the debugger does not add any folders to explorer) which is making it vastly more difficult than it should be to figure our what the sample code is doing
#12495
Posted: 02/19/2010 00:53:45
by Eugene Mayevski (EldoS Corp.)

Quote
Chris Schirlinger wrote:
What I can't figure out is how to populate these initially (with, say, the first two levels of the folder structure) and then populate on the fly further directories as the user open them up.


The idea is simple:

when the OS asks for directory enumeration (via OnEnumerateDirectory), you give it what you have in the directory you've been asked about. This includes files and subdirectories. If contents of that directory has changed externally (eg. new records have been added to the database), you call NotifyDirectoryChange method to let the OS know that it needs to re-read the directory (use search for NotifyDirectoryChange in the forum for detailed explanations).


Sincerely yours
Eugene Mayevski
#12498
Posted: 02/19/2010 02:47:33
by Volodymyr Zinin (EldoS Corp.)

Quote
Chris Schirlinger wrote:
Basically we have a tree of data, with "files" in it, each with a unique ID which I'd also want to store (I see there is a commented out index in the ENUM_INFO which I imagine would be useful.)

We have added the "unique ID" support in the CallbackFS version 3. See the OnGetFileNameByFileId callback in the documentation. I.e. in order to support this feature you should return unique Id for files during a directory enumeration and if some program decides to open a file via its unique Id (for example by the use of ZwCreateFile with the flag FILE_OPEN_BY_FILE_ID) the OnGetFileNameByFileId callback will be called, you return the full path name for the file, and then the file is opened as usual.

Quote
Chris Schirlinger wrote:
Unfortunately, I cannot step through anything (32bit VS2008 on 64bit Win7, running the program through the debugger does not add any folders to explorer) which is making it vastly more difficult than it should be to figure our what the sample code is doing

Are you using 32-bit CBFSNet.dll for 32-bit program?
#12515
Posted: 02/21/2010 18:05:27
by Chris Schirlinger (Standard support level)
Joined: 02/18/2010
Posts: 38

Beta version 3, Compiled as 32bit (Using 32bit CBFSNet.dll) and building the application as x86 (Changed from ANyCPU which did not work) now builds and is debuggable in Win7 64bit OS

The Create Storage (6 seconds) Mount Drive (6 seconds) and Add Mounting Points (6 seconds) is still a little long, but only occurs on the build machine, it seems to work fine on other machines

I'll investigate that later once I have a test app loading from our DB!
#12546
Posted: 02/22/2010 21:35:10
by Chris Schirlinger (Standard support level)
Joined: 02/18/2010
Posts: 38

I am still having issues creating virtual folders and files on the fly.

I am creating the virtual file structure on the fly when called by OnEnumerateDirectory.
WhenI get a vdir object that reports vdir.Context.IsEmpty() (I also tried adding a Enumerate variable to VirtualFile object which also worked) I check to see if I was expecting files and/or folders in there (checking the DB) then populate them as follows:

Code
VirtualFile newVF = new VirtualFile("foldername");
newVF.Attributes = System.IO.FileAttributes.Directory;
vdir.AddFile(newVF);

In cases of files I just do:
Code
VirtualFile newVF = new VirtualFile("filename.txt");
vdir.AddFile(newVF);

I get almost perfect results with the exception:

1) Excel cannot load virtual files from folders that contain virtual sub-directoriesm but can load files from the lowest sub-directory. I get the errors:
"Excel cannot access "filename.txt". The document may be read-only or encrypted."
followed by...
"'filename.txt' cannot be accessed. The file may be corrupted, located on a server that is not responding, or read-only"

2) In many cases, using explorer, I cannot add folders to virtual sub-directories

3) In cases where folders have sub-directories, Right Clicking the Select PROPERTIES for a virtual file displays incorrect information about the file (0 bytes, wrong creation date etc)

4) Trying to load files via Open With | Choose default program also fails

Otherwise it seems to work fine:
1) The folder tree structure and files show up in the right place in explorer
2) Double Clicking any file loads that virtual file correctly into Notepad (The default app for .txt files)
3) Opening up Notepad and selecting File | Open loads the files correctly
4) Doing the same with Microsoft Word works correctly
5) In explorer selecting Open With | <Program> works correctly for all programs I tried

I have a feeling that I am not supposed to generating the sub-folders in OnEnumerateDirectory or that I am generating the folders incorrectly?
#12553
Posted: 02/23/2010 00:53:50
by Eugene Mayevski (EldoS Corp.)

Quote
Chris Schirlinger wrote:
I have a feeling that I am not supposed to generating the sub-folders in OnEnumerateDirectory or that I am generating the folders incorrectly?


There's something wrong with your code. But this is your application, you design it, it's not easy for us to help you.

In regards to (3) it looks like you don't handle OnGetFileInfo callback correctly. Try starting from there - your first goal is to display the directory structure right and provide the file-related information right. Then you can go to file creation and opening.


Sincerely yours
Eugene Mayevski
#12555
Posted: 02/23/2010 01:12:14
by Volodymyr Zinin (EldoS Corp.)

There is a good utility to check what is wrong with your FS implementation - Process Monitor from sysinternals.com
Run it, set its filter to show only I/O operations on your virtual drive, and reproduce the problem. That helps to significantly narrow it - commonly the problem is somewhere around a kernel operation which returns an error.
#12579
Posted: 02/23/2010 20:20:06
by Chris Schirlinger (Standard support level)
Joined: 02/18/2010
Posts: 38

I am running tests with the raw sample apps (VDisk and VMounter) to try and work out what is going on.

Can you confirm the sample apps are supposed to be capable of the following:

Once mounted, via explorer you can add a new folder to the virtual drive using the example apps

I ask because neither VDisk nor VMouter seem to work on any machine I have tested them on.

With VBDisk, When you Right Click -> New | Folder on the right pane, nothing appears, on the left pane, a folder is added but the FILE in that folder becomes editable

With VMounter it's worse, you get folders eventually but they are corrupt/in the wrong place and generally broken

This is *similar* to the issues I am seeing in my code. If a folder has sub-folders, files in that folder break (They work, add a sub folder, and they don't any more)

I am sure somehow (in my code at least) I am not creating the folders correctly, but I can't find any examples. All I do is:
Code
VirtualFile newVF = new VirtualFile("foldername");
newVF.Attributes = System.IO.FileAttributes.Directory;
vdir.AddFile(newVF);

And then add that newVF to whatever VirtualFile should contain the folder.
#12582
Posted: 02/24/2010 00:10:24
by Chris Schirlinger (Standard support level)
Joined: 02/18/2010
Posts: 38

Still at a loss on creating folders on the fly, I'd really appreciate some help on how to generate them... I wrote a smaller, simpler test app.. This is the only code somewhere around line 548 in the VDisk CbFsEnumerateDirectory() function :
Code
//; ********************* TEST CODE START *****************************
if (vdir.Context.IsEmpty())
{
   //; This folder is empty. Add 2 folders and 1 file to it (Note, if you keep drilling down this is an endless tree)
   VirtualFile newFolder1 = new VirtualFile(Level + "_Folder_A");
   newFolder1.Attributes = System.IO.FileAttributes.Directory;
   vdir.AddFile(newFolder1);

   VirtualFile newFolder2 = new VirtualFile(Level + "_Folder_B");
   newFolder2.Attributes = System.IO.FileAttributes.Directory;
   vdir.AddFile(newFolder2);

   VirtualFile newFile = new VirtualFile(Level + "TheOnlyFile.txt");
   vdir.AddFile(newFile);
   Level++;
}
//; *********************  TEST CODE END  *****************************

...with the exception of a global int Level and a Console.WriteLine in the OnFileInfo section

DblClicking on any of the generated files creates a completly berserk path request in OnFileInfo, consisting of folders that aren't loaded, folders that aren't parents of other folders and folders not actually in the tree the file exists in. The path OnFileInfo is asking about for the filename does not, nor could it, exist.
Also by EldoS: CallbackRegistry
A component to monitor and control Windows registry access and create virtual registry keys.

Reply

Statistics

Topic viewed 7379 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!