EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Rename file / EnumerateDirectory

Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.
#34103
Posted: 07/24/2015 17:08:53
by Tim Sullivan (Standard support level)
Joined: 05/01/2008
Posts: 15

I am working on building a project that exposes a SharePoint repository using Windows Explorer. My current issue is performing a Rename of a file using Explorer. The code works to properly update the filename as well as updating the file in SharePoint. My problem is that the file (with the original name) remains displayed in Explorer ever after the rename operation. The re-named file also displays so, in effect, there are two copies of the file shown. When I perform a Refresh of the Explorer window the original file disappears.

I suspect the problem is related to EnumerateDirectory but possibly with RenameOrMove.

Thanks for your suggestions.

Code
public override void CbFsRenameOrMoveFile(object sender, CbFsFileInfo FileInfo, string NewFileName)
{
    ClientContext clientContext = null;

    string OldFileName = FileInfo.FileName;
    ClientOM.ListItem SPFile = null;

    ListItemCollection listItems = GetSPFile(sender, OldFileName, ref clientContext);
    if (listItems.Count == 1)
    {
        SPFile = listItems[0];

        SPFile.File.MoveTo("/Shared Documents" + NewFileName, MoveOperations.Overwrite);
        clientContext.ExecuteQuery();
    }
}


public override void CbFsCloseDirectoryEnumeration(object sender, CbFsFileInfo DirectoryInfo, CbFsDirectoryEnumerationInfo EnumerationInfo)
{
    if (!EnumerationInfo.UserContext.Equals(IntPtr.Zero))
    {
        if (GCHandle.FromIntPtr(EnumerationInfo.UserContext).IsAllocated)
        {
            SPDocumentsEnumerationContext pInfo = (SPDocumentsEnumerationContext)GCHandle.FromIntPtr(EnumerationInfo.UserContext).Target;
            GCHandle.FromIntPtr(EnumerationInfo.UserContext).Free();
        }
    }
}

public override void CbFsEnumerateDirectory(object sender, CbFsFileInfo DirectoryInfo, CbFsHandleInfo HandleInfo,
    CbFsDirectoryEnumerationInfo EnumerationInfo, string Mask, int Index, bool Restart, ref bool FileFound,
    ref string FileName, ref string ShortFileName, ref DateTime CreationTime, ref DateTime LastAccessTime,
    ref DateTime LastWriteTime, ref long EndOfFile, ref long AllocationSize, ref CBFS_LARGE_INTEGER FileId, ref ACCESS_MASK FileAttributes)
{
    SPDocumentsEnumerationContext context = null;
    ListItemCollection listItems = null;
            
    if (Restart && (EnumerationInfo.UserContext != IntPtr.Zero))
    {
        if (GCHandle.FromIntPtr(EnumerationInfo.UserContext).IsAllocated)
            GCHandle.FromIntPtr(EnumerationInfo.UserContext).Free();
        EnumerationInfo.UserContext = IntPtr.Zero;
    }
    if (EnumerationInfo.UserContext.Equals(IntPtr.Zero))
    {
        string SharePointURL = BaseURL(sender);
        ClientContext clientContext = new ClientContext(SharePointURL);
        List docList = clientContext.Web.Lists.GetByTitle("Documents");
        CamlQuery camlQuery = new CamlQuery();

        string userName = UserName(sender);
        string password = Password(sender);
        bool online = Online(sender);

        if (!String.IsNullOrEmpty(userName) && !String.IsNullOrEmpty(password))
            clientContext.Credentials = GetCredentials(userName, password, online);

        //returns all docs
        camlQuery.ViewXml = @"<View Scope='RecursiveAll'></View>";
        listItems = docList.GetItems(camlQuery);
        clientContext.Load(listItems);
        clientContext.ExecuteQuery();

        context = new SPDocumentsEnumerationContext(listItems);
        GCHandle gch = GCHandle.Alloc(context);
        EnumerationInfo.UserContext = GCHandle.ToIntPtr(gch);
    }
    else
    {
        context = (SPDocumentsEnumerationContext)GCHandle.FromIntPtr(EnumerationInfo.UserContext).Target;
    }

    ListItem item = context.GetNextDocument();
    if ( item != null)
    {
        FileFound = true;
        FileName = (string)item["FileLeafRef"];
        Console.WriteLine(FileName);
        CreationTime = Convert.ToDateTime(item["Created"]);
        LastAccessTime = DateTime.Now;
        LastWriteTime = Convert.ToDateTime(item["Modified"]);

        try
        {
            EndOfFile = Convert.ToInt32(item["File_x0020_Size"]);
        }
        catch
        {
            EndOfFile = 0;
        }

        AllocationSize = EndOfFile;  
        FileAttributes = 0;
    }
    else
    {
        FileFound = false;
        if (GCHandle.FromIntPtr(EnumerationInfo.UserContext).IsAllocated)
            GCHandle.FromIntPtr(EnumerationInfo.UserContext).Free();

        EnumerationInfo.UserContext = IntPtr.Zero;
    }
}
#34104
Posted: 07/25/2015 01:59:21
by Eugene Mayevski (EldoS Corp.)

Thank you for the detailed description.

First of all please disable MetaDataCache (set MetaDataCacheEnabled property to false) and see if the problem is gone. If it remains, we'll investigate the situation deeper.


Sincerely yours
Eugene Mayevski
#34116
Posted: 07/27/2015 18:42:08
by Tim Sullivan (Standard support level)
Joined: 05/01/2008
Posts: 15

That sure seemed like the right idea but after reviewing the other parts of the code I found that the property MetaDataCacheEnabled was already set to 'false'. I swapped it to 'true' to see what might happen and I don't notice any difference when renaming the file.

Any other ideas?
#34117
Posted: 07/28/2015 01:40:54
by Eugene Mayevski (EldoS Corp.)

I don't know sharepoint, but I'd like to check if the call to clientContext.ExecuteQuery(); on line 14 returns only when renaming is complete. It can be that in some circumstances it is delayed by the server, and it can happen that OnEnumerateDirectory in this situation returns the previous filename correctly.

Also I'd add some logging to CbFsRenameOrMoveFile and CbFsEnumerateDirectory to see the sequence of commands. Of course you can use ProcMon (Process Monitor, Volodymyr Zinin posted instructions here many times) to capture and analyze the log of filesystem requests sent to the filesystem, and see what is enumerated.

But I think that the problem is specific to sharepoint somehow.


Sincerely yours
Eugene Mayevski
#34133
Posted: 07/28/2015 16:43:08
by Tim Sullivan (Standard support level)
Joined: 05/01/2008
Posts: 15

I ran the procmon/filter on the Rename action and got the results below. I'm not sure what I'm looking for. I do see "ReplaceIfExists: False" which seems incorrect but I don't know how to control that.

Quote

"1:16:18.9105530 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: None, AllocationSize: n/a, OpenResult: Opened"
"1:16:18.9108183 PM","Explorer.EXE","5872","QueryNameInformationFile","\\ap\sp\","SUCCESS","Name: \ap\sp\"
"1:16:18.9108704 PM","Explorer.EXE","5872","QueryAttributeInformationVolume","\\ap\sp\","SUCCESS","FileSystemAttributes: Case Preserved, Unicode, ACLs, Named Streams, Reparse Points, 0x1000000, MaximumComponentNameLength: 260, FileSystemName: FAT32"
"1:16:18.9110887 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:18.9124743 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:18.9127614 PM","Explorer.EXE","5872","QueryBasicInformationFile","\\ap\sp\","SUCCESS","CreationTime: 0, LastAccessTime: 0, LastWriteTime: 0, ChangeTime: 0, FileAttributes: D"
"1:16:18.9128042 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:18.9171663 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:19.1518677 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\\oldname.txt","SUCCESS","Desired Access: Read Attributes, Delete, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:19.3901317 PM","Explorer.EXE","5872","QueryAttributeTagFile","\\ap\sp\oldname.txt","SUCCESS","Attributes: A, ReparseTag: 0x0"
"1:16:19.3902122 PM","Explorer.EXE","5872","QueryBasicInformationFile","\\ap\sp\oldname.txt","SUCCESS","CreationTime: 7/28/2015 1:15:47 PM, LastAccessTime: 7/28/2015 1:15:47 PM, LastWriteTime: 7/28/2015 1:15:47 PM, ChangeTime: 0, FileAttributes: A"
"1:16:19.3924064 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Write Data/Add File, Synchronize, Disposition: Open, Options: , Attributes: n/a, ShareMode: Read, Write, AllocationSize: n/a, OpenResult: DoesNotExist"
"1:16:19.6281862 PM","Explorer.EXE","5872","SetRenameInformationFile","\\ap\sp\oldname.txt","SUCCESS","ReplaceIfExists: False, FileName: \\ap\sp\NEWname.txt"
"1:16:20.4828574 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:20.6797510 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\NEWname.txt","SUCCESS",""
"1:16:20.6841666 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:20.6873776 PM","Explorer.EXE","5872","QueryBasicInformationFile","\\ap\sp\","SUCCESS","CreationTime: 0, LastAccessTime: 0, LastWriteTime: 0, ChangeTime: 0, FileAttributes: D"
"1:16:20.6874220 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:20.7042846 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:20.7223518 PM","Explorer.EXE","5872","NotifyChangeDirectory","\\ap\sp\","","Filter: FILE_NOTIFY_CHANGE_FILE_NAME, FILE_NOTIFY_CHANGE_DIR_NAME, FILE_NOTIFY_CHANGE_ATTRIBUTES, FILE_NOTIFY_CHANGE_LAST_WRITE"
"1:16:20.7225283 PM","Explorer.EXE","5872","QueryDeviceRelations","\\ap\sp\","INVALID DEVICE REQUEST",""
"1:16:20.7277612 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Read Data/List Directory, Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:20.7283082 PM","Explorer.EXE","5872","QueryRemoteProtocolInformation","\\ap\sp\","INVALID PARAMETER",""
"1:16:20.7283567 PM","Explorer.EXE","5872","QueryDirectory","\\ap\sp\NEWname.txt","SUCCESS","Filter: NEWname.txt, 1: NEWname.txt"
"1:16:20.9638025 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:21.0351030 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Read Data/List Directory, Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:21.0371859 PM","Explorer.EXE","5872","QueryRemoteProtocolInformation","\\ap\sp\","INVALID PARAMETER",""
"1:16:21.0372427 PM","Explorer.EXE","5872","QueryDirectory","\\ap\sp\oldname.txt","SUCCESS","Filter: oldname.txt, 1: NEWname.txt"
"1:16:21.3276183 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
"1:16:21.3308125 PM","Explorer.EXE","5872","CreateFile","\\ap\sp\","SUCCESS","Desired Access: Read Data/List Directory, Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened"
"1:16:21.3313089 PM","Explorer.EXE","5872","QueryRemoteProtocolInformation","\\ap\sp\","INVALID PARAMETER",""
"1:16:21.3314280 PM","Explorer.EXE","5872","QueryDirectory","\\ap\sp\NEWname.txt","SUCCESS","Filter: NEWname.txt, 1: NEWname.txt"
"1:16:21.5263954 PM","Explorer.EXE","5872","CloseFile","\\ap\sp\","SUCCESS",""
#34134
Posted: 07/28/2015 16:59:23
by Eugene Mayevski (EldoS Corp.)

I am sorry but the log is both incomplete and probably not helpful if you can't analyze it.

Please add logging to your callbacks and see what is returned by SharePoint in the callbacks I mentioned. If Sharepoint delays or caches some data, this would be the source of the problem. What I think about is that the file is renamed asynchronously and sometimes you get the old data in OnEnumerateDirectory.

On a side note, the quoted data like logs are easier to handle with QUOTE tag pair or the corresponding button. I've added the tag to your message.


Sincerely yours
Eugene Mayevski
#34152
Posted: 07/29/2015 13:47:56
by Tim Sullivan (Standard support level)
Joined: 05/01/2008
Posts: 15

The logging from the run shows no meaningful delay with the SharePoint request and nothing obviously cached from EnumerateDirectory. I receive three calls all with the renamed file listed (never the former name) after the rename is performed:
Quote

SP_CbFsEnumerateDirectory
NEW.txt
SP_CbFsCloseDirectoryEnumeration


Is there a way to force some kind of refresh or force a delete of the original file name -- ideally in the RenameOrMove override?
#34154
Posted: 07/29/2015 14:17:16
by Eugene Mayevski (EldoS Corp.)

There's no way to refresh anything from the callback, and it's not clear where the old data could be kept if you have disabled the metadata cache. Use of NotifyDirectoryChange in this situation is not appropriate as the change was originated by the OS itself.

Please check whether you disable this cache while the component is inactive (before the storage is created and mounted).


Sincerely yours
Eugene Mayevski
#34158
Posted: 07/30/2015 13:41:24
by Tim Sullivan (Standard support level)
Joined: 05/01/2008
Posts: 15

Problem solved. The issue was due to returning "true" incorrectly for the FileFound argument when the Mask argument contained the original file name.

I had to review all the debug messages for a working scenario in order to understand what was wrong.

I added code like this inside EnumerateDirectory:
Code
                if (Mask != "*")
                {
                    //looking for specific file
                    if (FileName != Mask)
                        FileFound = false;
                }


Appreciate the help.
#34159
Posted: 07/30/2015 13:58:42
by Eugene Mayevski (EldoS Corp.)

Thank you for letting us know about the reason.

HOWEVER I need to say that OnEnumerateDirectory can receive *any* mask -- masks like "ab?d*.f???" are perfectly valid there. You are assuming two border cases - the mask of "*" (which is also not right because *.* can be passed as well) and an exact file name, while there's an almost infinite number of possible masks in behind.

The correct approach would be to use some mask matching function to check if the filename matches the mask. CBFS includes a static method named IsMatchMask (not very grammatically correct name, I know, but the alternatives would be very long) that will help you.


Sincerely yours
Eugene Mayevski
Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.

Reply

Statistics

Topic viewed 5373 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!