EldoS | Feel safer!

Software components for data protection, secure storage and transfer

TElSimpleSFTPClient batch file download

Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.
#28278
Posted: 02/06/2014 09:49:59
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I have run into problems downloading multiple files.

I initiate the client like this:

Code
        private void SimpleClientInit()
        {
            _simpleClient = new TElSimpleSFTPClient();
            _simpleClient.Address = editHost.Text;
            _simpleClient.Port = int.Parse(editPort.Text);
            _simpleClient.AutoAdjustTransferBlock = false;
            _simpleClient.PipelineLength = 1;
            _simpleClient.Username = editUserName.Text;
            _simpleClient.Password = editPassword.Text;
            _simpleClient.OnKeyValidate += new TSSHKeyValidateEvent(SimpleClient_OnKeyValidate);
            _simpleClient.KeyStorage = keyStorage;
            _simpleClient.Versions = 8;
            _simpleClient.OnProgress += SftpClient_OnProgress;
            keyStorage.Clear();
            String pwd = editPrivateKeyPassword.Text;
            TElSSHKey key = new TElSSHKey();
            int err = key.LoadPrivateKey(editPrivateKey.Text, pwd);
            _simpleClient.AuthenticationTypes |= SBSSHConstants.Unit.SSH_AUTH_TYPE_PUBLICKEY;
            if (err == 0)
            {
                keyStorage.Add(key);
                _simpleClient.AuthenticationTypes |= SBSSHConstants.Unit.SSH_AUTH_TYPE_PUBLICKEY;
            }
            try
            {
                _simpleClient.Open();
            }
            catch (Exception t)
            {
                Log("SimpleClient Connection failed. " + t.Message);
                
                return;
            }


Initializing and opening works fine.

I use the DownloadFile Method, and when the Download of the second file starts I have problems.

Is initialization OK?

With kind regards
Peder Skånning
#28279
Posted: 02/06/2014 09:51:15
by Eugene Mayevski (EldoS Corp.)

Initialization is fine. What problem do you have with file download? It would be more useful if you show the code that performs downloading.


Sincerely yours
Eugene Mayevski
#28294
Posted: 02/07/2014 02:40:21
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

I use this syntax
Code
_simpleClient.DownloadFile(info.Name, fileName, TSBFileTransferMode.ftmOverwrite);

My log shows:
02/07/2014 09:38:58 - ******* Starting batch mode *******
02/07/2014 09:38:59 - Server key received and validated OK.
02/07/2014 09:39:00 - Transfer All
02/07/2014 09:39:00 - Preparing batch download
02/07/2014 09:39:03 - Ready for download of file 2014-126_E-ARKIV.AFP
02/07/2014 09:39:03 - Downloading 2014-126_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-126_E-ARKIV.AFP
02/07/2014 09:39:03 - Downloaded 0 of 78628
02/07/2014 09:39:03 - Downloaded 8192 of 78628
02/07/2014 09:39:03 - Downloaded 16384 of 78628
02/07/2014 09:39:03 - Downloaded 24576 of 78628
02/07/2014 09:39:04 - Downloaded 32768 of 78628
02/07/2014 09:39:04 - Downloaded 40960 of 78628
02/07/2014 09:39:04 - Downloaded 49152 of 78628
02/07/2014 09:39:04 - Downloaded 57344 of 78628
02/07/2014 09:39:04 - Downloaded 65536 of 78628
02/07/2014 09:39:04 - Downloaded 73728 of 78628
02/07/2014 09:39:09 - Downloaded 78628 of 78628
02/07/2014 09:39:09 - Transfer All
02/07/2014 09:39:09 - Preparing batch download
02/07/2014 09:39:10 - Ready for download of file 2014-134_E-ARKIV.AFP
02/07/2014 09:39:10 - Downloading 2014-134_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-134_E-ARKIV.AFP
02/07/2014 09:39:15 - Downloaded 0 of -1
02/07/2014 09:39:20 - Downloaded 8192 of -1
02/07/2014 09:39:20 - Download error for 2014-134_E-ARKIV.AFP with exception No such file
02/07/2014 09:39:20 - Transfer All
02/07/2014 09:39:20 - Preparing batch download
02/07/2014 09:39:22 - Ready for download of file 2014-134_E-ARKIV.AFP
02/07/2014 09:39:22 - Downloading 2014-134_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-134_E-ARKIV.AFP
02/07/2014 09:39:22 - Download error for 2014-134_E-ARKIV.AFP with exception SFTP component not connected

Why do you think connection is dropped on the second file?
#28295
Posted: 02/07/2014 02:46:50
by Vsevolod Ievgiienko (EldoS Corp.)

Could you post the code responsible for multiple files download.
#28296
Posted: 02/07/2014 03:03:37
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

That is big. I do not expect you to correct all that code. I am distributing files to up to three destinations depending on the content of the first three characters in the filename. The code goes like this:
Code
        private bool DownloadAll()
        {
            TElSftpFileInfo info = new TElSftpFileInfo();
            if (!batchPrepared)
            {
                if (transfer1)
                    foreach (TElSftpFileInfo fileInfo in currentFileList)
                    {
                        if (fileInfo.Name.StartsWith(startingWith1, StringComparison.CurrentCultureIgnoreCase) &&
                            !fileInfo.Attributes.Directory)
                            getFileList1.Add(fileInfo);
                    }
                if (transfer2)
                    foreach (TElSftpFileInfo fileInfo in currentFileList)
                    {
                        if (fileInfo.Name.StartsWith(startingWith2, StringComparison.CurrentCultureIgnoreCase) &&
                            !fileInfo.Attributes.Directory)
                            getFileList2.Add(fileInfo);
                    }
                if (transfer3)
                    foreach (TElSftpFileInfo fileInfo in currentFileList)
                    {
                        if (fileList3.ToUpper().Contains(fileInfo.Name.ToUpper()) &&
                            !fileInfo.Attributes.Directory)
                            getFileList3.Add(fileInfo);
                    }
                batchPrepared = true;
            }
            Log("Transfer All");
            if (transferAll)
            {
                if (!Directory.Exists(DownloadFolder))
                {
                    Log("Error: Download folder does not exist.");
                    return false;
                }
                Log("Preparing batch download");

                if (currentFileList.Count > 0 && currentFileList.Count > downloadCount)
                {
                    info = (TElSftpFileInfo)currentFileList[downloadCount];
                    {
                        if (info.Attributes.Directory)
                        {

                            Log("Can not download directory. Please choose ordinary file.");
                            return false;
                        }
                        string fileName = info.Name;
                        Log("Ready for download of file " + fileName);
                        {
                            if (DownloadFolder.EndsWith("\\"))
                            {
                                fileName = DownloadFolder + fileName;
                            }
                            else
                            {
                                fileName = DownloadFolder + "\\" + fileName;
                            }
                            Log("Downloading " + info.Name + " to " + fileName);
                            try
                            {
                                if (!_downloading)
                                {
                                    _downloading = true;
                                    _simpleClient.DownloadFile(info.Name, fileName, TSBFileTransferMode.ftmOverwrite);
                                    downloadCount += 1;
                                    fileIndex += 1;
                                }
                            }
                            catch (Exception e)
                            {
                                Log("Download error for " + info.Name + " with exception " + e.Message);
                            }                            
                        }
                    }
                }
            }
            if (transfer1)
            {
                if (!Directory.Exists(DownloadFolder1))
                {
                    
                    Log("Error: Download folder " + DownloadFolder1 + " does not exist.");
                    return false;
                }

                if (currentFileList.Count > 0 && getFileList1.Count > fileIndex1)
                {
                    string fileName = getFileList1[fileIndex1].Name;

                    if (DownloadFolder1.EndsWith("\\"))
                    {
                        fileName = DownloadFolder1 + getFileList1[fileIndex1].Name;
                    }
                    else
                    {
                        fileName = DownloadFolder1 + "\\" + getFileList1[fileIndex1].Name;
                    }
                    Log("Downloading " + getFileList1[fileIndex1].Name + " to " + fileName);
                    _simpleClient.DownloadFile(getFileList1[fileIndex1].Name, fileName, TSBFileTransferMode.ftmOverwrite);
                    fileIndex1 += 1;
                    Thread.Sleep(2000);
                }
            }
            if (transfer2)
            {
                if (!Directory.Exists(DownloadFolder2))
                {
                    
                    Log("Error: Download folder " + DownloadFolder2 + " does not exist.");
                    return false;
                }

                if (currentFileList.Count > 0 && getFileList2.Count > fileIndex2)
                {
                    string fileName = getFileList2[fileIndex2].Name;

                    if (DownloadFolder1.EndsWith("\\"))
                    {
                        fileName = DownloadFolder2 + getFileList2[fileIndex2].Name;
                    }
                    else
                    {
                        fileName = DownloadFolder2 + "\\" + getFileList2[fileIndex2].Name;
                    }
                    Log("Downloading " + getFileList2[fileIndex2].Name + " to " + fileName);
                    _simpleClient.DownloadFile(getFileList2[fileIndex2].Name, fileName, TSBFileTransferMode.ftmOverwrite);
                    fileIndex2 += 1;
                    Thread.Sleep(2000);
                }
            }
            if (transfer3)
            {
                if (!Directory.Exists(DownloadFolder3))
                {
                    
                    Log("Error: Download folder " + DownloadFolder3 + " does not exist.");
                    return false;
                }

                if (currentFileList.Count > 0 && getFileList3.Count > fileIndex3)
                {
                    string fileName = getFileList3[fileIndex3].Name;
                    if (fileList3.Contains(fileName.ToUpper()) && transfer3)
                    {
                        if (DownloadFolder1.EndsWith("\\"))
                        {
                            fileName = DownloadFolder3 + getFileList3[fileIndex3].Name;
                        }
                        else
                        {
                            fileName = DownloadFolder3 + "\\" + getFileList3[fileIndex3].Name;
                        }
                        Log("Downloading " + getFileList3[fileIndex3].Name + " to " + fileName);
                        _simpleClient.DownloadFile(getFileList3[fileIndex3].Name, fileName, TSBFileTransferMode.ftmOverwrite);
                    }
                    fileIndex3 += 1;
                    Thread.Sleep(2000);
                }
            }
            if (currentFileList.Count > 0 && ((transferAll && currentFileList.Count > fileIndex) ||
                (transfer1 && getFileList1.Count > fileIndex1) ||
                (transfer2 && getFileList2.Count > fileIndex2) ||
                (transfer3 && getFileList1.Count > fileIndex3)))
            {
                
                return true;
            }

            if (downloadCount > 0)
            {
                Log("Downloaded " + downloadCount + " files.");
                if (_loggingUrl != null)
                {
                    bgLogService.Log(_logCategory, "Exit", 0, "Batch job files", SystemInformation.ComputerName,
                                     "Downloaded " + downloadCount + " files.",
                                     memoLog.Text);
                }
                StartExternalApp();
            }
            else
            {
                Log("No files for download this time");
            }
            batchPrepared = false;
            getFileList1.Clear();
            getFileList2.Clear();
            getFileList3.Clear();
            fileIndex = 0;
            fileIndex1 = 0;
            fileIndex2 = 0;
            fileIndex3 = 0;
            if (BatchMode)
            {
                _simpleClient.Close(true);
                Log("Exiting Application");
                BatchActive = false;
                Application.Exit();
            }
            
            return false;
        }


Right now I am only focusing on the first call to "DownloadFile"
#28297
Posted: 02/07/2014 03:08:22
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

The Log entries "Downloaded" comes from the OnProgress event. I set a swithc _downloading to try to prevent the second download to start before the first is finished.

Code
        private void SftpClient_OnProgress(object sender, long total, long current, ref bool cancel)
        {
            if (current < total)
            {
                _downloading = true;
            }
            else
            {
                Thread.Sleep(5000);
                _downloading = false;
            }
            Log("Downloaded " + current + " " + " of " + total);
        }
#28298
Posted: 02/07/2014 03:36:16
by Vsevolod Ievgiienko (EldoS Corp.)

Could you also post here a call stack of "No such file" exception.
#28300
Posted: 02/07/2014 03:53:21
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

Yes!
Quote

at SBSftp.TElSftpClient.ReadSync(Byte[] Handle, Int64 Offset, Byte[] Buffer, Int32 StartIndex, Int32 Size)
at SBSimpleSftp.TElSimpleSFTPClient.DownloadStream(String RemoteFileName, Stream LocalStream, TSBFileTransferMode Mode, Int64 RestartFrom)
at SBSimpleSftp.TElSimpleSFTPClient.DownloadFile(String RemoteFileName, String LocalFileName, TSBFileTransferMode Mode, Int64 RestartFrom)
at SBSimpleSftp.TElSimpleSFTPClient.DownloadFile(String RemoteFileName, String LocalFileName, TSBFileTransferMode Mode)
at Sftp.frmMain.DownloadAll() in c:\Multidata\SftpUtil\FormSftp.cs:line 2021
#28303
Posted: 02/07/2014 04:09:41
by Vsevolod Ievgiienko (EldoS Corp.)

For me it looks like the 2nd file doesn't exist on the server or disappears during download process and "No such file" message is returned by the server. Please try to login to the server suing our sample from \EldoS\SecureBlackbox.NET\Samples\C#\SFTPBlackbox\Desktop\Client\SimpleSFTPClient folder andtry to download needed files one by one. If some error happens then post the log produced by the sample here.
#28304
Posted: 02/07/2014 04:17:09
by Peder Skanning (Basic support level)
Joined: 02/26/2013
Posts: 22

All files exists on the server before and after batch execution. I send the log from two batch executions. Subsequent I have started the program in manual mode and downloaded the first file withuot problems.
Log:
Quote
02/07/2014 09:38:58 - ******* Starting batch mode *******
02/07/2014 09:38:59 - Server key received and validated OK.
02/07/2014 09:39:00 - Transfer All
02/07/2014 09:39:00 - Preparing batch download
02/07/2014 09:39:03 - Ready for download of file 2014-126_E-ARKIV.AFP
02/07/2014 09:39:03 - Downloading 2014-126_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-126_E-ARKIV.AFP
02/07/2014 09:39:03 - Downloaded 0 of 78628
02/07/2014 09:39:03 - Downloaded 8192 of 78628
02/07/2014 09:39:03 - Downloaded 16384 of 78628
02/07/2014 09:39:03 - Downloaded 24576 of 78628
02/07/2014 09:39:04 - Downloaded 32768 of 78628
02/07/2014 09:39:04 - Downloaded 40960 of 78628
02/07/2014 09:39:04 - Downloaded 49152 of 78628
02/07/2014 09:39:04 - Downloaded 57344 of 78628
02/07/2014 09:39:04 - Downloaded 65536 of 78628
02/07/2014 09:39:04 - Downloaded 73728 of 78628
02/07/2014 09:39:09 - Downloaded 78628 of 78628
02/07/2014 09:39:09 - Transfer All
02/07/2014 09:39:09 - Preparing batch download
02/07/2014 09:39:10 - Ready for download of file 2014-134_E-ARKIV.AFP
02/07/2014 09:39:10 - Downloading 2014-134_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-134_E-ARKIV.AFP
02/07/2014 09:39:15 - Downloaded 0 of -1
02/07/2014 09:39:20 - Downloaded 8192 of -1
02/07/2014 09:39:20 - Download error for 2014-134_E-ARKIV.AFP with exception No such file
02/07/2014 09:39:20 - Transfer All
02/07/2014 09:39:20 - Preparing batch download
02/07/2014 09:39:22 - Ready for download of file 2014-134_E-ARKIV.AFP
02/07/2014 09:39:22 - Downloading 2014-134_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-134_E-ARKIV.AFP
02/07/2014 09:39:22 - Download error for 2014-134_E-ARKIV.AFP with exception SFTP component not connected
02/07/2014 10:51:05 - ******* Starting batch mode *******
02/07/2014 10:51:05 - Server key received and validated OK.
02/07/2014 10:51:06 - Transfer All
02/07/2014 10:51:06 - Preparing batch download
02/07/2014 10:51:06 - Ready for download of file 2014-126_E-ARKIV.AFP
02/07/2014 10:51:06 - Downloading 2014-126_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-126_E-ARKIV.AFP
02/07/2014 10:51:07 - Downloaded 0 of 78628
02/07/2014 10:51:07 - Downloaded 8192 of 78628
02/07/2014 10:51:07 - Downloaded 16384 of 78628
02/07/2014 10:51:07 - Downloaded 24576 of 78628
02/07/2014 10:51:07 - Downloaded 32768 of 78628
02/07/2014 10:51:07 - Downloaded 40960 of 78628
02/07/2014 10:51:07 - Downloaded 49152 of 78628
02/07/2014 10:51:07 - Downloaded 57344 of 78628
02/07/2014 10:51:07 - Downloaded 65536 of 78628
02/07/2014 10:51:07 - Downloaded 73728 of 78628
02/07/2014 10:51:12 - Downloaded 78628 of 78628
02/07/2014 10:51:12 - Transfer All
02/07/2014 10:51:12 - Preparing batch download
02/07/2014 10:51:12 - Ready for download of file 2014-134_E-ARKIV.AFP
02/07/2014 10:51:12 - Downloading 2014-134_E-ARKIV.AFP to C:\Multidata\RunWinScp\SftpUtil\bin\Debug\E-arkiv\2014-134_E-ARKIV.AFP
02/07/2014 10:51:18 - Downloaded 0 of -1
02/07/2014 10:51:23 - Downloaded 8192 of -1
02/07/2014 11:14:38 - TCP connection opened.
02/07/2014 11:14:38 - Server key received and validated OK.
02/07/2014 11:14:39 - Authentication type 16 failed.
02/07/2014 11:14:39 - Authentication succeeded.
02/07/2014 11:14:39 - SSH Connection started.
02/07/2014 11:14:39 - Sftp connection started.
02/07/2014 11:14:39 - Opening directory .
02/07/2014 11:14:39 - Directory opened.
02/07/2014 11:14:40 - File list received.
02/07/2014 11:14:40 - Closing active handle.
02/07/2014 11:15:25 - Starting file download 2014-126_E-ARKIV.AFP
02/07/2014 11:15:26 - File received
02/07/2014 11:15:26 - Closing active handle.
02/07/2014 11:15:29 - TCP connection closed.
Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.

Reply

Statistics

Topic viewed 4565 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!