EldoS | Feel safer!

Software components for data protection, secure storage and transfer

TElSimpleSFTPClient batch file download

Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.
#28319
Posted: 02/07/2014 07:00:56
by Ken Ivanov (EldoS Corp.)

Peder,

There is no need to stop anything - upon exiting the DownloadFile() call the corresponding download process is cleared and closed. Probably it's the server that has the download session hanging around for some time after the download is closed. Could you please try to insert the Sleep() calls as I suggested above and see what happens?
#28320
Posted: 02/07/2014 07:18:12
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

This is really wierd! I tried this code:

Code
                Log("Preparing download 5 files");

                if (currentFileList.Count > 0 && currentFileList.Count > downloadCount)
                {
                    try
                    {
                        _simpleClient.DownloadFile("2014-134_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-134_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 1");
                        Thread.Sleep(2000);
                        _simpleClient.DownloadFile("2014-135_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-135_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 2");
                        Thread.Sleep(2000);
                        _simpleClient.DownloadFile("2014-136_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-136_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 3");
                        Thread.Sleep(2000);
                        _simpleClient.DownloadFile("2014-138_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-138_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 4");
                        Thread.Sleep(2000);
                        _simpleClient.DownloadFile("2014-140_E-ARKIV.AFP", "C:\\Multidata\\RunWinScp\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-140_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                    }
                    catch (Exception e)
                    {
                        Log("Error batch downloading files " + e.Message);
                    }


The log is:
Quote
02/07/2014 14:11:09 - ******* Starting batch mode *******
02/07/2014 14:11:10 - Server key received and validated OK.
02/07/2014 14:11:11 - Transfer All
02/07/2014 14:11:11 - Preparing download 5 files
02/07/2014 14:11:12 - Downloaded 0 of 11228
02/07/2014 14:11:12 - Downloaded 8192 of 11228
02/07/2014 14:11:17 - Downloaded 11228 of 11228
02/07/2014 14:11:17 - Sleep 1
02/07/2014 14:11:19 - Downloaded 0 of 555089
02/07/2014 14:11:19 - Downloaded 8192 of 555089
02/07/2014 14:11:19 - Downloaded 16384 of 555089
02/07/2014 14:11:19 - Downloaded 24576 of 555089
02/07/2014 14:11:19 - Downloaded 32768 of 555089
02/07/2014 14:11:19 - Downloaded 40960 of 555089
02/07/2014 14:11:19 - Downloaded 49152 of 555089
02/07/2014 14:11:20 - Downloaded 57344 of 555089
02/07/2014 14:11:20 - Downloaded 65536 of 555089
02/07/2014 14:11:20 - Downloaded 73728 of 555089
02/07/2014 14:11:20 - Downloaded 81920 of 555089
02/07/2014 14:11:20 - Downloaded 90112 of 555089
02/07/2014 14:11:20 - Downloaded 98304 of 555089
02/07/2014 14:11:20 - Downloaded 106496 of 555089
02/07/2014 14:11:20 - Downloaded 114688 of 555089
02/07/2014 14:11:20 - Downloaded 122880 of 555089
02/07/2014 14:11:20 - Downloaded 131072 of 555089
02/07/2014 14:11:20 - Downloaded 139264 of 555089
02/07/2014 14:11:20 - Downloaded 147456 of 555089
02/07/2014 14:11:20 - Downloaded 155648 of 555089
02/07/2014 14:11:20 - Downloaded 163840 of 555089
02/07/2014 14:11:20 - Downloaded 172032 of 555089
02/07/2014 14:11:20 - Downloaded 180224 of 555089
02/07/2014 14:11:20 - Downloaded 188416 of 555089
02/07/2014 14:11:21 - Downloaded 196608 of 555089
02/07/2014 14:11:21 - Downloaded 204800 of 555089
02/07/2014 14:11:21 - Downloaded 212992 of 555089
02/07/2014 14:11:21 - Downloaded 221184 of 555089
02/07/2014 14:11:21 - Downloaded 229376 of 555089
02/07/2014 14:11:21 - Downloaded 237568 of 555089
02/07/2014 14:11:21 - Downloaded 245760 of 555089
02/07/2014 14:11:21 - Downloaded 253952 of 555089
02/07/2014 14:11:21 - Downloaded 262144 of 555089
02/07/2014 14:11:21 - Downloaded 270336 of 555089
02/07/2014 14:11:21 - Downloaded 278528 of 555089
02/07/2014 14:11:21 - Downloaded 286720 of 555089
02/07/2014 14:11:21 - Downloaded 294912 of 555089
02/07/2014 14:11:21 - Downloaded 303104 of 555089
02/07/2014 14:11:21 - Downloaded 311296 of 555089
02/07/2014 14:11:21 - Downloaded 319488 of 555089
02/07/2014 14:11:21 - Downloaded 327680 of 555089
02/07/2014 14:11:21 - Downloaded 335872 of 555089
02/07/2014 14:11:22 - Downloaded 344064 of 555089
02/07/2014 14:11:22 - Downloaded 352256 of 555089
02/07/2014 14:11:22 - Downloaded 360448 of 555089
02/07/2014 14:11:22 - Downloaded 368640 of 555089
02/07/2014 14:11:22 - Downloaded 376832 of 555089
02/07/2014 14:11:22 - Downloaded 385024 of 555089
02/07/2014 14:11:22 - Downloaded 393216 of 555089
02/07/2014 14:11:22 - Downloaded 401408 of 555089
02/07/2014 14:11:22 - Downloaded 409600 of 555089
02/07/2014 14:11:22 - Downloaded 417792 of 555089
02/07/2014 14:11:22 - Downloaded 425984 of 555089
02/07/2014 14:11:22 - Downloaded 434176 of 555089
02/07/2014 14:11:22 - Downloaded 442368 of 555089
02/07/2014 14:11:22 - Downloaded 450560 of 555089
02/07/2014 14:11:22 - Downloaded 458752 of 555089
02/07/2014 14:11:22 - Downloaded 466944 of 555089
02/07/2014 14:11:22 - Downloaded 475136 of 555089
02/07/2014 14:11:22 - Downloaded 483328 of 555089
02/07/2014 14:11:23 - Downloaded 491520 of 555089
02/07/2014 14:11:23 - Downloaded 499712 of 555089
02/07/2014 14:11:23 - Downloaded 507904 of 555089
02/07/2014 14:11:23 - Downloaded 516096 of 555089
02/07/2014 14:11:23 - Downloaded 524288 of 555089
02/07/2014 14:11:23 - Downloaded 532480 of 555089
02/07/2014 14:11:23 - Downloaded 540672 of 555089
02/07/2014 14:11:23 - Downloaded 548864 of 555089
02/07/2014 14:11:28 - Downloaded 555089 of 555089
02/07/2014 14:11:29 - Sleep 2
02/07/2014 14:11:36 - Downloaded 0 of -1
02/07/2014 14:11:41 - Downloaded 8192 of -1
02/07/2014 14:11:41 - Error batch downloading files No such file
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected
02/07/2014 14:11:41 - Transfer All
02/07/2014 14:11:41 - Preparing download 5 files
02/07/2014 14:11:41 - Error batch downloading files SFTP component not connected


The first file 134 is downloaded, but with 0Kb size. The second 135 is intact. The third 136 is 0Kb. No further files is in Download folder :-(
#28321
Posted: 02/07/2014 07:34:29
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

Tried to extend the Sleep to 20000

The first file downloaded OK with correct file size
Second file download started. Now first file size was reset to 0!!!

Then Visual Studio reports error:
Quote
The CLR has been unable to transition from COM context 0x398cd60 to COM context 0x398cfb0 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.


Log:
Quote
02/07/2014 14:30:16 - ******* Starting batch mode *******
02/07/2014 14:30:17 - Server key received and validated OK.
02/07/2014 14:30:18 - Transfer All
02/07/2014 14:30:18 - Preparing download 5 files
02/07/2014 14:30:18 - Downloaded 0 of 1064545
02/07/2014 14:30:18 - Downloaded 8192 of 1064545
02/07/2014 14:30:18 - Downloaded 16384 of 1064545
02/07/2014 14:30:19 - Downloaded 24576 of 1064545
02/07/2014 14:30:19 - Downloaded 32768 of 1064545
02/07/2014 14:30:19 - Downloaded 40960 of 1064545
02/07/2014 14:30:19 - Downloaded 49152 of 1064545
02/07/2014 14:30:19 - Downloaded 57344 of 1064545
02/07/2014 14:30:19 - Downloaded 65536 of 1064545
02/07/2014 14:30:19 - Downloaded 73728 of 1064545
02/07/2014 14:30:19 - Downloaded 81920 of 1064545
02/07/2014 14:30:19 - Downloaded 90112 of 1064545
02/07/2014 14:30:19 - Downloaded 98304 of 1064545
02/07/2014 14:30:19 - Downloaded 106496 of 1064545
02/07/2014 14:30:19 - Downloaded 114688 of 1064545
02/07/2014 14:30:19 - Downloaded 122880 of 1064545
02/07/2014 14:30:19 - Downloaded 131072 of 1064545
02/07/2014 14:30:19 - Downloaded 139264 of 1064545
02/07/2014 14:30:19 - Downloaded 147456 of 1064545
02/07/2014 14:30:19 - Downloaded 155648 of 1064545
02/07/2014 14:30:19 - Downloaded 163840 of 1064545
02/07/2014 14:30:20 - Downloaded 172032 of 1064545
02/07/2014 14:30:20 - Downloaded 180224 of 1064545
02/07/2014 14:30:20 - Downloaded 188416 of 1064545
02/07/2014 14:30:20 - Downloaded 196608 of 1064545
02/07/2014 14:30:20 - Downloaded 204800 of 1064545
02/07/2014 14:30:20 - Downloaded 212992 of 1064545
02/07/2014 14:30:20 - Downloaded 221184 of 1064545
02/07/2014 14:30:20 - Downloaded 229376 of 1064545
02/07/2014 14:30:20 - Downloaded 237568 of 1064545
02/07/2014 14:30:20 - Downloaded 245760 of 1064545
02/07/2014 14:30:20 - Downloaded 253952 of 1064545
02/07/2014 14:30:20 - Downloaded 262144 of 1064545
02/07/2014 14:30:20 - Downloaded 270336 of 1064545
02/07/2014 14:30:20 - Downloaded 278528 of 1064545
02/07/2014 14:30:20 - Downloaded 286720 of 1064545
02/07/2014 14:30:20 - Downloaded 294912 of 1064545
02/07/2014 14:30:20 - Downloaded 303104 of 1064545
02/07/2014 14:30:20 - Downloaded 311296 of 1064545
02/07/2014 14:30:21 - Downloaded 319488 of 1064545
02/07/2014 14:30:21 - Downloaded 327680 of 1064545
02/07/2014 14:30:21 - Downloaded 335872 of 1064545
02/07/2014 14:30:21 - Downloaded 344064 of 1064545
02/07/2014 14:30:21 - Downloaded 352256 of 1064545
02/07/2014 14:30:21 - Downloaded 360448 of 1064545
02/07/2014 14:30:21 - Downloaded 368640 of 1064545
02/07/2014 14:30:21 - Downloaded 376832 of 1064545
02/07/2014 14:30:21 - Downloaded 385024 of 1064545
02/07/2014 14:30:21 - Downloaded 393216 of 1064545
02/07/2014 14:30:21 - Downloaded 401408 of 1064545
02/07/2014 14:30:21 - Downloaded 409600 of 1064545
02/07/2014 14:30:21 - Downloaded 417792 of 1064545
02/07/2014 14:30:21 - Downloaded 425984 of 1064545
02/07/2014 14:30:21 - Downloaded 434176 of 1064545
02/07/2014 14:30:21 - Downloaded 442368 of 1064545
02/07/2014 14:30:22 - Downloaded 450560 of 1064545
02/07/2014 14:30:22 - Downloaded 458752 of 1064545
02/07/2014 14:30:22 - Downloaded 466944 of 1064545
02/07/2014 14:30:22 - Downloaded 475136 of 1064545
02/07/2014 14:30:22 - Downloaded 483328 of 1064545
02/07/2014 14:30:22 - Downloaded 491520 of 1064545
02/07/2014 14:30:22 - Downloaded 499712 of 1064545
02/07/2014 14:30:22 - Downloaded 507904 of 1064545
02/07/2014 14:30:22 - Downloaded 516096 of 1064545
02/07/2014 14:30:22 - Downloaded 524288 of 1064545
02/07/2014 14:30:22 - Downloaded 532480 of 1064545
02/07/2014 14:30:22 - Downloaded 540672 of 1064545
02/07/2014 14:30:22 - Downloaded 548864 of 1064545
02/07/2014 14:30:22 - Downloaded 557056 of 1064545
02/07/2014 14:30:22 - Downloaded 565248 of 1064545
02/07/2014 14:30:22 - Downloaded 573440 of 1064545
02/07/2014 14:30:22 - Downloaded 581632 of 1064545
02/07/2014 14:30:22 - Downloaded 589824 of 1064545
02/07/2014 14:30:23 - Downloaded 598016 of 1064545
02/07/2014 14:30:23 - Downloaded 606208 of 1064545
02/07/2014 14:30:23 - Downloaded 614400 of 1064545
02/07/2014 14:30:23 - Downloaded 622592 of 1064545
02/07/2014 14:30:23 - Downloaded 630784 of 1064545
02/07/2014 14:30:23 - Downloaded 638976 of 1064545
02/07/2014 14:30:23 - Downloaded 647168 of 1064545
02/07/2014 14:30:23 - Downloaded 655360 of 1064545
02/07/2014 14:30:23 - Downloaded 663552 of 1064545
02/07/2014 14:30:23 - Downloaded 671744 of 1064545
02/07/2014 14:30:23 - Downloaded 679936 of 1064545
02/07/2014 14:30:23 - Downloaded 688128 of 1064545
02/07/2014 14:30:23 - Downloaded 696320 of 1064545
02/07/2014 14:30:23 - Downloaded 704512 of 1064545
02/07/2014 14:30:23 - Downloaded 712704 of 1064545
02/07/2014 14:30:23 - Downloaded 720896 of 1064545
02/07/2014 14:30:23 - Downloaded 729088 of 1064545
02/07/2014 14:30:24 - Downloaded 737280 of 1064545
02/07/2014 14:30:24 - Downloaded 745472 of 1064545
02/07/2014 14:30:24 - Downloaded 753664 of 1064545
02/07/2014 14:30:24 - Downloaded 761856 of 1064545
02/07/2014 14:30:24 - Downloaded 770048 of 1064545
02/07/2014 14:30:24 - Downloaded 778240 of 1064545
02/07/2014 14:30:24 - Downloaded 786432 of 1064545
02/07/2014 14:30:24 - Downloaded 794624 of 1064545
02/07/2014 14:30:24 - Downloaded 802816 of 1064545
02/07/2014 14:30:24 - Downloaded 811008 of 1064545
02/07/2014 14:30:24 - Downloaded 819200 of 1064545
02/07/2014 14:30:24 - Downloaded 827392 of 1064545
02/07/2014 14:30:24 - Downloaded 835584 of 1064545
02/07/2014 14:30:24 - Downloaded 843776 of 1064545
02/07/2014 14:30:24 - Downloaded 851968 of 1064545
02/07/2014 14:30:24 - Downloaded 860160 of 1064545
02/07/2014 14:30:24 - Downloaded 868352 of 1064545
02/07/2014 14:30:24 - Downloaded 876544 of 1064545
02/07/2014 14:30:25 - Downloaded 884736 of 1064545
02/07/2014 14:30:25 - Downloaded 892928 of 1064545
02/07/2014 14:30:25 - Downloaded 901120 of 1064545
02/07/2014 14:30:25 - Downloaded 909312 of 1064545
02/07/2014 14:30:25 - Downloaded 917504 of 1064545
02/07/2014 14:30:25 - Downloaded 925696 of 1064545
02/07/2014 14:30:25 - Downloaded 933888 of 1064545
02/07/2014 14:30:25 - Downloaded 942080 of 1064545
02/07/2014 14:30:25 - Downloaded 950272 of 1064545
02/07/2014 14:30:25 - Downloaded 958464 of 1064545
02/07/2014 14:30:25 - Downloaded 966656 of 1064545
02/07/2014 14:30:25 - Downloaded 974848 of 1064545
02/07/2014 14:30:25 - Downloaded 983040 of 1064545
02/07/2014 14:30:25 - Downloaded 991232 of 1064545
02/07/2014 14:30:25 - Downloaded 999424 of 1064545
02/07/2014 14:30:25 - Downloaded 1007616 of 1064545
02/07/2014 14:30:25 - Downloaded 1015808 of 1064545
02/07/2014 14:30:26 - Downloaded 1024000 of 1064545
02/07/2014 14:30:26 - Downloaded 1032192 of 1064545
02/07/2014 14:30:26 - Downloaded 1040384 of 1064545
02/07/2014 14:30:26 - Downloaded 1048576 of 1064545
02/07/2014 14:30:26 - Downloaded 1056768 of 1064545
02/07/2014 14:30:31 - Downloaded 1064545 of 1064545
02/07/2014 14:30:31 - Sleep 1
02/07/2014 14:30:56 - Downloaded 0 of -1
02/07/2014 14:31:01 - Downloaded 8192 of -1
02/07/2014 14:31:01 - Error batch downloading files No such file
02/07/2014 14:31:01 - Transfer All
02/07/2014 14:31:01 - Preparing download 5 files
02/07/2014 14:31:01 - Error batch downloading files SFTP component not connected
02/07/2014 14:31:01 - Transfer All
02/07/2014 14:31:01 - Preparing download 5 files
02/07/2014 14:31:01 - Error batch downloading files SFTP component not connected
#28322
Posted: 02/07/2014 08:20:46
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

More wierd behaviour. I tried this code which works to some extend:
Code
                Log("Preparing download 5 files");

                if (currentFileList.Count > 0 && currentFileList.Count > downloadCount)
                {
                    try
                    {
                        _simpleClient.DownloadFile("2014-138_E-ARKIV.AFP", "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-138_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 1");
                        Thread.Sleep(20000);
                        _simpleClient.Close();
                        _simpleClient.Open();
                        _simpleClient.DownloadFile("2014-140_E-ARKIV.AFP", "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-140_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 2");
                        Thread.Sleep(20000);
                        _simpleClient.Close();
                        _simpleClient.Open();
                        _simpleClient.DownloadFile("2014-141_E-ARKIV.AFP", "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-141_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 3");
                        Thread.Sleep(20000);
                        _simpleClient.Close();
                        _simpleClient.Open();
                        _simpleClient.DownloadFile("2014-144_E-ARKIV.AFP", "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-144_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                        Log("Sleep 4");
                        Thread.Sleep(20000);
                        _simpleClient.Close();
                        _simpleClient.Open();
                        _simpleClient.DownloadFile("2014-147_E-ARKIV.AFP", "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\2014-147_E-ARKIV.AFP", TSBFileTransferMode.ftmOverwrite);
                    }
                    catch (Exception e)
                    {
                        Log("Error batch downloading files " + e.Message);
                    }
                }

Now all files are downloaded. But the first file is reset to filesize 0Kb on the last file download. No files are deleted on the server as they are, when downloaded on by one on the manual approch.
#28326
Posted: 02/07/2014 08:46:18
by Ken Ivanov (EldoS Corp.)

Peder,

This is really weird indeed. What is the nature of your application - i.e. is it a console application, a GUI one or a service, is it multi-threaded? The symptoms you are getting suggest that there are some thread-related issues in your application. In particular, upon exiting the DownloadFile() method, the component completely forgets about the file that was just downloaded, and thus can't reset it to zero length later.

Could you please try to create a new console application, put the above code into its Main() method and check if it exposes the same issue?

Ken
#28396
Posted: 02/13/2014 07:28:51
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I started the development from a copy of your SFTP client sample, because i like to have some GUI for setup and test purposes. I ran into problems with SFTPClient, and was recommended to switch to SFTPSimpleClient. I got everything working, also batch download.
Then i discovered, that ServerKey validation had to be handled. In an earlier project with the SecureBridge components from Devart it was handled differently. So I had to rework the solution.
Since I implemented ServerKey validation things are playing up strangely.
I have now moved the code to a Console application, and I have got rid of the reset of the first file to 0Kb. But it seems like the read of the last byte is not recognized by the SFTP server. Because the files are still in the list on the server. Normal behaviour on this server is, that the file gets deleted on read of the last byte.
I am on the brink of giving up, i have spend far to much time on this.
#28397
Posted: 02/13/2014 07:40:50
by Eugene Mayevski (EldoS Corp.)

Please check that the component is used in *only one thread*. It can not be used concurrently from different (or the same) thread.

Let me note, that we never ever had reports similar to yours which suggests that you are doing something unexpected and probably not exactly correct. Please review your code. Try to modify one of our samples for your tasks and see if this works.


Sincerely yours
Eugene Mayevski
#28398
Posted: 02/13/2014 08:16:26
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

Please have a look at this. I have uploaded a screenshot, which documents the list of files pending for download on the SFTP server. Please note the File sizes.
I use this code for downloading the first five files:
Code
                        const string firstFile = "2014-157_E-ARKIV.AFP";
                        Log("Downloading " + firstFile);
                        _simpleClient.DownloadFile(firstFile, "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\" + firstFile, TSBFileTransferMode.ftmOverwrite);
                        Log("Finished downloading " + firstFile + " total size " + _total);
                        while (_downloading)
                            Thread.Sleep(20);
                        _simpleClient.Close();
                        Thread.Sleep(5000);
                        _simpleClient.Open();
                        const string secondFile = "2014-158_E-ARKIV.AFP";
                        Log("Downloading " + secondFile);
                        _simpleClient.DownloadFile(secondFile, "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\" + secondFile, TSBFileTransferMode.ftmOverwrite);
                        Log("Finished downloading " + secondFile + " total size " + _total);
                        while (_downloading)
                            Thread.Sleep(20);
                        _simpleClient.Close();
                        Thread.Sleep(5000);
                        _simpleClient.Open();
                        const string thirdFile = "2014-159_E-ARKIV.AFP";
                        Log("Downloading " + thirdFile);
                        _simpleClient.DownloadFile(thirdFile, "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\" + thirdFile, TSBFileTransferMode.ftmOverwrite);
                        Log("Finished downloading " + thirdFile + " total size " + _total);
                        while (_downloading)
                            Thread.Sleep(20);
                        _simpleClient.Close();
                        Thread.Sleep(5000);
                        _simpleClient.Open();
                        const string fourthFile = "2014-160_E-ARKIV.AFP";
                        Log("Downloading " + fourthFile);
                        _simpleClient.DownloadFile(fourthFile, "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\" + fourthFile, TSBFileTransferMode.ftmOverwrite);
                        Log("Finished downloading " + fourthFile + " total size " + _total);
                        while (_downloading)
                            Thread.Sleep(20);
                        _simpleClient.Close();
                        Thread.Sleep(5000);
                        _simpleClient.Open();
                        const string fifthFile = "2014-161_E-ARKIV.AFP";
                        Log("Downloading " + fifthFile);
                        _simpleClient.DownloadFile(fifthFile, "C:\\Multidata\\SftpUtil\\bin\\Debug\\E-arkiv\\" + fifthFile, TSBFileTransferMode.ftmOverwrite);
                        Log("Finished downloading " + fifthFile + " total size " + _total);

I am handling the OnProgress event like this:
Code
        private static void SftpClient_OnProgress(object sender, long total, long current, ref bool cancel)
        {
            _total = total;
            if (current < total)
            {
                _downloading = true;
                Log("Downloaded " + current + " of " + total + " bytes");
            }
            else
            {
                _downloading = false;
                Log("Download completed. Total " + total + " bytes");
            }
        }

The log documents, that the filesize of the first file is correct. This file was deleted from the SFTP Server in this run. The rest of the files have wrong file sizes, and they are still in the file list, and their loca file size does not correspont with the original file size.

Quote
02/13/2014 15:04:31 - TSIconsole started
02/13/2014 15:04:32 - Server key received and validated OK.
02/13/2014 15:04:33 - Transfer All
02/13/2014 15:04:33 - Preparing download 5 files
02/13/2014 15:04:33 - Downloading 2014-157_E-ARKIV.AFP
02/13/2014 15:04:33 - Downloaded 0 of 196226 bytes
02/13/2014 15:04:33 - Downloaded 8192 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 16384 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 24576 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 32768 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 40960 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 49152 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 57344 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 65536 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 73728 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 81920 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 90112 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 98304 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 106496 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 114688 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 122880 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 131072 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 139264 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 147456 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 155648 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 163840 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 172032 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 180224 of 196226 bytes
02/13/2014 15:04:34 - Downloaded 188416 of 196226 bytes
02/13/2014 15:04:34 - Download completed. Total 196226 bytes
02/13/2014 15:04:34 - Finished downloading 2014-157_E-ARKIV.AFP total size 196226
02/13/2014 15:04:40 - Server key received and validated OK.
02/13/2014 15:04:40 - Downloading 2014-158_E-ARKIV.AFP
02/13/2014 15:04:40 - Downloaded 0 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 8192 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 16384 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 24576 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 32768 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 40960 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 49152 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 57344 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 65536 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 73728 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 81920 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 90112 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 98304 of 112537 bytes
02/13/2014 15:04:40 - Downloaded 106496 of 112537 bytes
02/13/2014 15:04:41 - Download completed. Total 112537 bytes
02/13/2014 15:04:41 - Finished downloading 2014-158_E-ARKIV.AFP total size 112537
02/13/2014 15:04:46 - Server key received and validated OK.
02/13/2014 15:04:46 - Downloading 2014-159_E-ARKIV.AFP
02/13/2014 15:04:46 - Downloaded 0 of 7740 bytes
02/13/2014 15:04:47 - Download completed. Total 7740 bytes
02/13/2014 15:04:47 - Finished downloading 2014-159_E-ARKIV.AFP total size 7740
02/13/2014 15:04:52 - Server key received and validated OK.
02/13/2014 15:04:52 - Downloading 2014-160_E-ARKIV.AFP
02/13/2014 15:04:52 - Downloaded 0 of 151429 bytes
02/13/2014 15:04:52 - Downloaded 8192 of 151429 bytes
02/13/2014 15:04:52 - Downloaded 16384 of 151429 bytes
02/13/2014 15:04:52 - Downloaded 24576 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 32768 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 40960 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 49152 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 57344 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 65536 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 73728 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 81920 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 90112 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 98304 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 106496 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 114688 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 122880 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 131072 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 139264 of 151429 bytes
02/13/2014 15:04:53 - Downloaded 147456 of 151429 bytes
02/13/2014 15:04:53 - Download completed. Total 151429 bytes
02/13/2014 15:04:53 - Finished downloading 2014-160_E-ARKIV.AFP total size 151429
02/13/2014 15:04:59 - Server key received and validated OK.
02/13/2014 15:04:59 - Downloading 2014-161_E-ARKIV.AFP
02/13/2014 15:04:59 - Downloaded 0 of 270 bytes
02/13/2014 15:04:59 - Download completed. Total 270 bytes
02/13/2014 15:04:59 - Finished downloading 2014-161_E-ARKIV.AFP total size 270
02/13/2014 15:04:59 - TSIconsole terminating

So it seems, that the filesize reported from the SFTP server is incorrect.


#28399
Posted: 02/13/2014 08:30:16
by Eugene Mayevski (EldoS Corp.)

First, you are using relative path names for remote paths, which is a very bad idea, because SFTP doesn't have a concept of current directory and god knows what you are requesting when using relative paths.

Next, where do the files come from on the server? It is possible that you come across the files being not completely uploaded/written to the server, i.e. you start downloading before the files are complete.

Finally, from the logs it seems that the server itself is inadequate. It's hard to get sane behavior if the server is buggy.


Sincerely yours
Eugene Mayevski
#28400
Posted: 02/13/2014 08:39:32
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I am comming to the same conclusion - that the server is buggy.
What i find triggy is, that when I download a single file and closes the application, the first file seems to get downloaded correctly. It is usually on the second batch download the error occours, which is why I have alle the Sleep implementation.
May be I should free the client from memory and recreate and reinitialize it completely?
This SFTP Server is an old setup in another company, and I have no say over it :-(
Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.

Reply

Statistics

Topic viewed 4589 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!