EldoS | Feel safer!

Software components for data protection, secure storage and transfer

TElSimpleSFTPClient batch file download

Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.
#28305
Posted: 02/07/2014 04:20:42
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I have build the application on your SFTP client sample. Manual downlod is performed with TElSftpClient and batch download with TElSimpleSFTPClient
#28306
Posted: 02/07/2014 04:23:40
by Vsevolod Ievgiienko (EldoS Corp.)

Please check if you are able to download all needed files with our sample.
#28307
Posted: 02/07/2014 04:40:50
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I can download alle files one by one without problems.
I would like to batch download synchronously. In other words I would like to ensure that a file download is finished before the next download starts. In the current implementation three file downloads runs cuncurrently, but the first download never finishes.
I have looked af DownloadFiles as an alternative, but I have not been able to figure the wildcard character for RemoteMask. I cant se it in the documentation.
#28309
Posted: 02/07/2014 04:46:39
by Vsevolod Ievgiienko (EldoS Corp.)

RemoteMask format is described here: https://www.eldos.com/documentation/sb...masks.html
#28310
Posted: 02/07/2014 04:50:19
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

Thanks! I will try to implement DownloadFiles instead of the current
#28311
Posted: 02/07/2014 05:13:07
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I think i have an error because of impatibility with the server. I have tried this:
Code
                    try
                    {
                        _simpleClient.DownloadFiles(".", "*", DownloadFolder, TSBFileTransferMode.ftmOverwrite, false,
                                                TSBCaseConversion.sccNone, false);
                    }
                    catch (Exception e)
                    {
                        Log("Error batch downloading files " + e.Message);
                    }


And I get exactly the same behaviour. The log reports:
Quote
2/07/2014 12:06:51 - ******* Starting batch mode *******
02/07/2014 12:06:52 - Server key received and validated OK.
02/07/2014 12:06:53 - Transfer All
02/07/2014 12:06:53 - Preparing batch download
02/07/2014 12:06:53 - Downloaded 0 of 11228
02/07/2014 12:06:54 - Downloaded 8192 of 11228
02/07/2014 12:06:59 - Downloaded 11228 of 11228
02/07/2014 12:07:04 - Downloaded 0 of -1
02/07/2014 12:07:09 - Downloaded 8192 of -1
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected (error code is 6)
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected
02/07/2014 12:07:12 - Transfer All
02/07/2014 12:07:12 - Preparing batch download
02/07/2014 12:07:12 - Error batch downloading files SFTP component not connected


The local folder lists alle the files on the SFTP server, but the download never finishes on any of the files. Please see the scrennedump in attached Capture.jpg


#28314
Posted: 02/07/2014 05:52:23
by Ken Ivanov (EldoS Corp.)

Hello Peder,

It seems that the server might have problems with pipelined transfers. Could you please try to configure TElSimpleSFTPClient in the following way and check if it helps at all:

1) AutoAdjustTransferBlock to false,
2) PipelineLength to 1,
3) DownloadBlockSize to 8192.

If it doesn't, could you comment out your current batch download code and replace it with 2 or 3 consecutive DownloadFile() calls in a row (you can hard code the names of the files for testing purposes) and check if the things work this way?
#28315
Posted: 02/07/2014 06:17:08
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

No, that did not do it.

I still think, that if I do the single file download it will work. How do I determine when the file is completely downloaded.

The SFTP Server I am contacting, automatically deletes the source file when the read of the file is completed. But when I batch download this does not happen before the next file is being processed.

If I could halt the next file download untill the first is finished, it would be great. But I don't know which parameter determins Download complete on TElSimpleSFTPClient.
#28316
Posted: 02/07/2014 06:24:53
by Ken Ivanov (EldoS Corp.)

Once DownloadFile() exits, you can be sure that the download has been completed. That is, the following sequence of calls should work just fine:
Code
Client.DownloadFile("file1", "file1", TSBFileTransferMode.ftmOverwrite);
Client.DownloadFile("file2", "file2", TSBFileTransferMode.ftmOverwrite);
Client.DownloadFile("file3", "file3", TSBFileTransferMode.ftmOverwrite);
Client.DownloadFile("file4", "file4", TSBFileTransferMode.ftmOverwrite);


If the above code doesn't work for you, probably it's the server who doesn't manage to complete the transfer operation before the next operation starts. Could you please try to insert Thread.Sleep(2000) right after each DownloadFile() call and check if it helps?
#28317
Posted: 02/07/2014 06:40:45
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

No!

I tried this code:
Code
                Log("Preparing download 5 files");

                if (currentFileList.Count > 0 && currentFileList.Count > downloadCount)
                {
                    try
                    {
                        _simpleClient.DownloadFile("2014-134_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-134_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        _simpleClient.DownloadFile("2014-135_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-135_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        _simpleClient.DownloadFile("2014-136_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-136_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        _simpleClient.DownloadFile("2014-138_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-138_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        _simpleClient.DownloadFile("2014-140_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-140_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                    }
                    catch (Exception e)
                    {
                        Log("Error batch downloading files " + e.Message);
                    }
                }


The download of the first and second file started. Log says:

Quote
02/07/2014 13:37:08 - ******* Starting batch mode *******
02/07/2014 13:37:09 - Server key received and validated OK.
02/07/2014 13:37:10 - Transfer All
02/07/2014 13:37:10 - Preparing download 5 files
02/07/2014 13:37:10 - Downloaded 0 of 11228
02/07/2014 13:37:10 - Downloaded 8192 of 11228
02/07/2014 13:37:16 - Downloaded 11228 of 11228
02/07/2014 13:37:21 - Downloaded 0 of -1
02/07/2014 13:37:26 - Downloaded 8192 of -1
02/07/2014 13:37:26 - Error batch downloading files No such file
02/07/2014 13:37:26 - Transfer All
02/07/2014 13:37:26 - Preparing download 5 files
02/07/2014 13:37:26 - Error batch downloading files SFTP component not connected
02/07/2014 13:37:26 - Transfer All
02/07/2014 13:37:26 - Preparing download 5 files
02/07/2014 13:37:26 - Error batch downloading files SFTP component not connected
02/07/2014 13:37:26 - Transfer All
02/07/2014 13:37:26 - Preparing download 5 files
02/07/2014 13:37:26 - Error batch downloading files SFTP component not connected
02/07/2014 13:37:26 - Transfer All
02/07/2014 13:37:26 - Preparing download 5 files
02/07/2014 13:37:26 - Error batch downloading files SFTP component not connected
02/07/2014 13:37:26 - Transfer All
02/07/2014 13:37:26 - Preparing download 5 files
02/07/2014 13:37:26 - Error batch downloading files SFTP component not connected
02/07/2014 13:37:26 - Transfer All
02/07/2014 13:37:26 - Preparing download 5 files


I have to find a way to stop the second download before the first is finished
Also by EldoS: Rethync
The cross-platform framework that simplifies synchronizing data between mobile and desktop applications and servers and cloud storages

Reply

Statistics

Topic viewed 4561 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!