EldoS | Feel safer!

Software components for data protection, secure storage and transfer

CBFS long file access time

Also by EldoS: CallbackRegistry
A component to monitor and control Windows registry access and create virtual registry keys.
#5428
Posted: 03/14/2008 08:04:00
by adam  (Basic support level)
Joined: 03/13/2008
Posts: 18

I am using CBFS to evaluate a file system over a network.

I plan to download the file to the local hard drive on onOpenFile and then onReadFile access the file on the local machine.

The network connection speed can be very slow and it is possible that the files are large. What is the appropriate implementation? should i create another thread to download the file and display a downloading dialog box?

Thanks
#5430
Posted: 03/14/2008 08:53:50
by Volodymyr Zinin (EldoS Corp.)

Hello,

Try to increase the CallbackFileSystem.ThreadPoolSize value (so requests to other files in a lot of cases won't wait for a free working thread and will be processed in parallel). Also after the onOpenFile callback is called (with read/write access) you could start caching the file immediately.

Quote
adam wrote:
should i create another thread to download the file and display a downloading dialog box?

You can do it. But also you can download a file from a thread that calls onReadFile. Because from the onReadFile callback you must return either requested data or an error code.
#5459
Posted: 03/16/2008 16:05:36
by adam  (Basic support level)
Joined: 03/13/2008
Posts: 18

Quote
You can do it. But also you can download a file from a thread that calls onReadFile. Because from the onReadFile callback you must return either requested data or an error code.


If my application has is still downloading the file from the thread that was started with onFileOpen or created from a call to onReadFile, how do i postpone onReadFile untill i have the file ?
#5469
Posted: 03/17/2008 15:23:09
by Volodymyr Zinin (EldoS Corp.)

I apologize for the delay.

Asynchronous requests (for example the ReadFile call with lpOverlapped!=NULL) are processed asynchronously by the CBFS driver internally. But it calls the user mode callbacks always synchronously. So you can't postpone them.
#5470
Posted: 03/17/2008 16:04:29
by adam  (Basic support level)
Joined: 03/13/2008
Posts: 18

Using CBFS i am uncertain of a method to process a onReadFile requested from a file located remotely.
My plan was to download the file if not located in the cache (on the local hard drive)

What would you suggest?
#5479
Posted: 03/18/2008 02:45:47
by Volodymyr Zinin (EldoS Corp.)

After the onOpenFile callback is called (with read/write access) you could start caching a remote file immediately. In hope that the file will be requested to read.

Also there is another problem with the system File Explorer. It requests a lot of information (tool tips, thumbnails) about files that it shows in own pane. These requests aren't desired for files obtained via a slow network connection.
There are two workarounds of it (also we are trying to find other ones):

1. To show a CallbackFS virtual storage as remote drive. This will considerably reduce amount of information that Explorer requests for quick tips.
For do this you should create a "private" (i.e. UNC) mounting point, then open a share by using it and map a share as network drive.
You can do it programmatically. Here is my test code:
Code
BOOL
OnCreateSharePointTest(
    IN DWORD StorageId,
    IN PCWSTR DriveLetter OPTIONAL
    )
{
    SHARE_INFO_2 ShareInfo2;
    NET_API_STATUS Status;
    DWORD ErrIndex;
    DWORD LastError;
    BOOL Success = FALSE;
    BOOL SymLinkCreated = FALSE;
    BOOL ShareCreated = FALSE;

#define CBFS_SHARING_LINK_NAME  L"CbFsShareTest"
#define CBFS_SHARE_NAME  L"CbFsShare"

    if (!CbFsCreateSymLink(StorageId, CBFS_SHARING_LINK_NAME, FALSE/*it means global mount point*/))
        goto end;
    SymLinkCreated = TRUE;

    ZeroMemory(&ShareInfo2, sizeof(ShareInfo2));

    ShareInfo2.shi2_netname = CBFS_SHARE_NAME;
    ShareInfo2.shi2_type = 0;//0x40000000;//STYPE_TEMPORARY
    ShareInfo2.shi2_remark = L"Some comment";
    ShareInfo2.shi2_permissions = ACCESS_ALL;
    ShareInfo2.shi2_max_uses = -1;
    //ShareInfo2.shi2_current_uses = 0;
    ShareInfo2.shi2_path = L"\\\\.\\" CBFS_SHARING_LINK_NAME L"\\";
    ShareInfo2.shi2_passwd = NULL;

    Status = NetShareAdd( NULL,
                          2,
                          (LPBYTE)&ShareInfo2,
                          &ErrIndex );
    if (Status != NERR_Success) {

        SetLastError(Status);
        goto end;
    }
    ShareCreated = TRUE;

    LastError = WNetAddConnection( L"\\\\127.0.0.1\\" CBFS_SHARE_NAME,
                                   NULL,
                                   L"T:" );

    if (LastError != NO_ERROR) {

        SetLastError(LastError);
        goto end;
    }  

    Success = TRUE;

end:

    LastError = GetLastError();

    if (!Success) {
    
        if (ShareCreated) {

            ASSERT(0);
        }

        if (SymLinkCreated) {

            CbFsDeleteSymLink(CBFS_SHARING_LINK_NAME);
        }
    }

    SetLastError(LastError);
    return Success;
}

BTW: It's possible to create non persistent network drives mapping (such drives exist until the computer isn't rebooted). I didn't examine this by I think it's possible to do by the use of WNetAddConnection/WNetAddConnection2. Also such mapping can be created by means of the system's command line utility "net" (for example "net use /persistent:no").

But before you can check whether it is suitable for you. Simply create a CallbackFS storage with some mounting point (for example "d:"), share it through the Explorer menu and then open in Explorer the url "\\127.0.0.1" and map the CallbackFS storage share as network drive (for example "z:"). After that try to access the CallbackFS storage via drives "d:" and "z:" and see the differences in behavior (the amount of OnOpen/OnRead callbacks will be considerably less in the second case).

For more information about what Explorer shows in quick tips see here:
http://msdn2.microsoft.com/en-us/library/bb776862.aspx


2. Perhaps there is a possibility to filter quick tips information by writing some shell extension dll. I mean to replace the quick tip Explorer's behavior by writing own shell extension dll. By I don't know whether it's possible to do.
#5481
Posted: 03/18/2008 05:13:26
by adam  (Basic support level)
Joined: 03/13/2008
Posts: 18

thank you for the information. I am still a little confused on what to do if a onReafFile is requested if i do not have the file downloaded ?
#5482
Posted: 03/18/2008 05:32:34
by Volodymyr Zinin (EldoS Corp.)

You should block the current onReafFile request until the necessary part of the file is downloaded.
#5484
Posted: 03/18/2008 06:58:59
by adam  (Basic support level)
Joined: 03/13/2008
Posts: 18

Block by returning an error code for onReafFile
#5485
Posted: 03/18/2008 07:14:30
by Volodymyr Zinin (EldoS Corp.)

No. Either synchronously read the destination file in the context of the thread that calls onReadFile or block it (for example by means of WaitForSingleObject(SomeEvent)) until another thread completes reading the destination file.
But you can also return an error code from onReadFile. In this case the same error will be returned to the originator of the request.
Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.

Reply

Statistics

Topic viewed 21775 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!