EldoS | Feel safer!

Software components for data protection, secure storage and transfer

TElSimpleFTPSClient.DownloadFiles() Error Handling Example

Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.
#37240
Posted: 07/12/2016 16:39:19
by Jeremy Leininger (Standard support level)
Joined: 07/12/2016
Posts: 5

I am looking for an example of error handling for the TElSimpleFTPSClient.DownloadFiles() method in v15.0.294. Specifically, I want to know how to gracefully handle FTP command errors, how to recover from them, and how to retry/continue the download.

Background:
I have created a console application that allows our enterprise to use a scheduling application called ISE to download files. Nearly every download/upload for our vendor connections have little or no problem. However, when we try to download files from Citrix Sharefile (sharefileftp.com) using FTPS (on port 990, with implicit SSL mode, and passive mode), we are encountering consistent problems. Specifically, we are getting Unaccepted server reply (error code is 213) and also sometimes (error code is 425) before it. After that, the files download with a 0 size and the connection seems to stop authenticating.


#37241
Posted: 07/12/2016 17:17:14
by Jeremy Leininger (Standard support level)
Joined: 07/12/2016
Posts: 5

Additional information:
Attached: Wireshark test captures.

Code snippet (C#):
///
/// Process the FTPS server data.
///

/// <param name="ftpsServer">The FTPS server data.</param>
/// <returns>The number of files copied for this server.</returns>
public int Process(FTPSServer ftpsServer)
{
// Report the status back to the console.
Console.WriteLine("FTPS Client started.");

// Report the status back to the console.
if (this.debug)
{
Console.WriteLine("FTPSServer Address: {0}", ftpsServer.Address);
Console.WriteLine("FTPSServer Port: {0}", ftpsServer.Port ?? 990);
Console.WriteLine("FTPSServer Username: <hidden>");
Console.WriteLine("FTPSServer Password: <hidden>\n");
}

// Use a simple FTPS client object.
using (TElSimpleFTPSClient telSimpleFTPSClient = new TElSimpleFTPSClient())
{
// Initialize the FTPS client.
this.InitializeFTPSClient(telSimpleFTPSClient);

// Set the FTPS client properties for this FTPS server.
telSimpleFTPSClient.Address = ftpsServer.Address;
if (ftpsServer.Port.HasValue)
{
telSimpleFTPSClient.Port = ftpsServer.Port.Value;
switch (telSimpleFTPSClient.Port)
{
case 990:
telSimpleFTPSClient.SSLMode = TSBSSLMode.smImplicit;
break;

default:
break;
}
}

telSimpleFTPSClient.Username = ftpsServer.Username;
telSimpleFTPSClient.Password = ftpsServer.Password;
telSimpleFTPSClient.Versions = SBSSLConstants.Unit.sbTLS1 + SBSSLConstants.Unit.sbTLS11 + SBSSLConstants.Unit.sbTLS12;
telSimpleFTPSClient.UseSSL = true;
telSimpleFTPSClient.EncryptDataChannel = true;
telSimpleFTPSClient.PassiveMode = true;

// Load the certificate cache.
foreach (X509Certificate x509Certificate in ftpsServer.Certificates)
{
TElX509CertificateEx newTElX509CertificateEx = new TElX509CertificateEx();
switch (x509Certificate.X509CertificateFileFormat.Format)
{
case "DER":
newTElX509CertificateEx.LoadFromBuffer(x509Certificate.FileBuffer);
this.telMemoryCertStorage.Add(newTElX509CertificateEx);
break;

case "PEM":
newTElX509CertificateEx.LoadFromBufferPEM(x509Certificate.FileBuffer, x509Certificate.Password);
this.telMemoryCertStorage.Add(newTElX509CertificateEx);
break;

case "PFX":
newTElX509CertificateEx.LoadFromBufferPFX(x509Certificate.FileBuffer, x509Certificate.Password);
this.telMemoryCertStorage.Add(newTElX509CertificateEx);
break;

case "SPC":
newTElX509CertificateEx.LoadFromBufferSPC(x509Certificate.FileBuffer);
this.telMemoryCertStorage.Add(newTElX509CertificateEx);
break;
}
}

// Report the status back to the console.
Console.WriteLine("Opening connection.");

// Open the connection to the FTPS server.
telSimpleFTPSClient.Open();

// Login to the FTPS server.
telSimpleFTPSClient.Login();

// Report the status back to the console.
Console.WriteLine("Connection opened.");

// Report the status back to the console.
if (this.debug)
{
Console.WriteLine("Iterating through the FileTransferSettings.");
}

// Determine if there is at least one file transfer setting group.
if (ftpsServer.FileTransferSettingGroups.Count > 0)
{
// Initialize the last server group name with the first server group name.
this.LastServerGroupName = ftpsServer.FileTransferSettingGroups.First().Name;
}

// Iterate through each task for this FTPS server.
foreach (FileTransferSettingGroup fileTransferSettingGroup in ftpsServer.FileTransferSettingGroups)
{
// Initialize the total setting process files copied to zero.
this.TotalSettingProcessFilesCopied = 0;

// Determine if the current setting group name doesn't match the last server group name.
if (fileTransferSettingGroup.Name != this.LastServerGroupName)
{
// Output the total files copied for the setting group.
Console.WriteLine("Total files copied for vendor '{0}', server '{1}', and setting group '{2}': {3}", this.vendorName, ftpsServer.Address, this.LastServerGroupName, this.TotalSettingGroupFilesCopied);

// Set the last server group name to the current server group name.
this.LastServerGroupName = fileTransferSettingGroup.Name;

// Initialize the total setting group files copied to zero.
this.TotalSettingGroupFilesCopied = 0;
}

// Report the status back to the console.
if (this.debug)
{
Console.WriteLine("FileTransferSetting:");
Console.WriteLine("\tFileTransferType: {0}", fileTransferSettingGroup.FileTransferType.Type);
}

// Determine which type of task is to be performed.
switch (fileTransferSettingGroup.FileTransferType.Type)
{
case "Download":
// Report the status back to the console.
if (this.debug)
{
Console.WriteLine("\tRemotePath: {0}", fileTransferSettingGroup.RemotePath);
Console.WriteLine("\tMask: {0}", fileTransferSettingGroup.Mask);
Console.WriteLine("\tLocalPath: {0}", fileTransferSettingGroup.LocalPath);
Console.WriteLine("\tFileTransferMode: {0}", fileTransferSettingGroup.FileTransferMode.ModeName);
Console.WriteLine("\tFileCopyMode: {0}", fileTransferSettingGroup.FileCopyMode.ModeName);
Console.WriteLine("\tCaseSensitive: {0}", fileTransferSettingGroup.CaseSensitive);
Console.WriteLine("\tCaseConversion: {0}", fileTransferSettingGroup.CaseConversion.ModeName);
Console.WriteLine("\tRecursive: {0}\n", fileTransferSettingGroup.Recursive);
}

// Download the files.
telSimpleFTPSClient.DownloadFiles(fileTransferSettingGroup.RemotePath, fileTransferSettingGroup.Mask, fileTransferSettingGroup.LocalPath, (TSBFileTransferMode)Enum.Parse(typeof(TSBFileTransferMode), fileTransferSettingGroup.FileTransferMode.Mode.ToString()), (TSBFileCopyMode)Enum.Parse(typeof(TSBFileCopyMode), fileTransferSettingGroup.FileCopyMode.Mode.ToString()), fileTransferSettingGroup.CaseSensitive, (TSBCaseConversion)Enum.Parse(typeof(TSBCaseConversion), fileTransferSettingGroup.CaseConversion.Mode.ToString()), fileTransferSettingGroup.Recursive);
break;

case "Upload":
// Report the status back to the console.
if (this.debug)
{
Console.WriteLine("\tLocalPath: {0}", fileTransferSettingGroup.LocalPath);
Console.WriteLine("\tMask: {0}", fileTransferSettingGroup.Mask);
Console.WriteLine("\tRemotePath: {0}", fileTransferSettingGroup.RemotePath);
Console.WriteLine("\tFileTransferMode: {0}", fileTransferSettingGroup.FileTransferMode.ModeName);
Console.WriteLine("\tFileCopyMode: {0}", fileTransferSettingGroup.FileCopyMode.ModeName);
Console.WriteLine("\tCaseSensitive: {0}", fileTransferSettingGroup.CaseSensitive);
Console.WriteLine("\tCaseConversion: {0}", fileTransferSettingGroup.CaseConversion.ModeName);
Console.WriteLine("\tRecursive: {0}\n", fileTransferSettingGroup.Recursive);
}

// Upload the files.
telSimpleFTPSClient.UploadFiles(fileTransferSettingGroup.LocalPath, fileTransferSettingGroup.Mask, fileTransferSettingGroup.RemotePath, (TSBFileTransferMode)Enum.Parse(typeof(TSBFileTransferMode), fileTransferSettingGroup.FileTransferMode.Mode.ToString()), (TSBFileCopyMode)Enum.Parse(typeof(TSBFileCopyMode), fileTransferSettingGroup.FileCopyMode.Mode.ToString()), fileTransferSettingGroup.CaseSensitive, (TSBCaseConversion)Enum.Parse(typeof(TSBCaseConversion), fileTransferSettingGroup.CaseConversion.Mode.ToString()), fileTransferSettingGroup.Recursive);
break;
}

// Output the total files copied for the process.
Console.WriteLine("Total files copied for vendor '{0}', server '{1}', setting group '{2}', and process {3}: {4}", this.vendorName, ftpsServer.Address, this.LastServerGroupName, fileTransferSettingGroup.FileTransferType.Type.ToLower(), this.TotalSettingProcessFilesCopied);
}

// Output the total files copied for the setting group.
Console.WriteLine("Total files copied for vendor '{0}', server '{1}', and setting group '{2}': {3}", this.vendorName, ftpsServer.Address, this.LastServerGroupName, this.TotalSettingGroupFilesCopied);

// Close the connection to the FTPS server.
telSimpleFTPSClient.Close();

// Report the status back to the console.
Console.WriteLine("FTPS Client ended.");

// Return the total number of files copied for the server.
return this.TotalServerFilesCopied;
}
}
#37242
Posted: 07/12/2016 17:18:47
by Jeremy Leininger (Standard support level)
Joined: 07/12/2016
Posts: 5

Attached: Wireshark test captures.


[ Download ]
#37243
Posted: 07/13/2016 15:17:02
by Vsevolod Ievgiienko (EldoS Corp.)

Thank you for contacting us.

Could you please try to connect to the problematic server using our sample and post its log here. You can also try different connection options using this sample and check if some combination works.

The sample is located in \EldoS\SecureBlackbox.NET\Samples\C#\FTPSBlackbox\Desktop\Client\SimpleFTPS folder.
#37244
Posted: 07/13/2016 15:45:00
by Jeremy Leininger (Standard support level)
Joined: 07/12/2016
Posts: 5

Connection properties that work. I have never had a problem connecting. The problem, I believe, is dropping the connection during download and not being able to recover gracefully.

Connection log...
<<<220 ftp-ec2-2.sharefile.com FTP Server Ready (SSL)

>>>USER <redacted>
<<<331 Hello <redacted>, please enter your password.

>>>PASS <redacted>
<<<230-Connection established from 71-87-13-50.static.eucl.wi.charter.com [71.87.13.50].
230-You are connected as <redacted>.
230 Welcome to the FILO FTP site.

>>>PBSZ 0
<<<200 OK.

>>>PROT P
<<<200 Data connections set to secure (SSL) mode

>>>FEAT
<<<211-Extensions supported:
EPSV
MDTM
PASV
REST STREAM
SIZE
UTF8
PBSZ
PROT
X-NOVELLABS
X-CITRIX
211 End.


#37245
Posted: 07/14/2016 03:54:44
by Vsevolod Ievgiienko (EldoS Corp.)

Quote
The problem, I believe, is dropping the connection during download

And was it the sample log for such situation? Please try to download a few files and post its log here.
#37248
Posted: 07/14/2016 10:12:21
by Jeremy Leininger (Standard support level)
Joined: 07/12/2016
Posts: 5

OK, I will. And I appreciate you trying to "solve" this one problem I am having.

How do you download all the files in the directory with the sample tool? Ie. What do you put for the command parameter?

Also, the sample doesn't use TElSimpleFTPSClient.DownloadFiles(), it uses TElSimpleFTPSClient.Receive(). Is that then a fair comparison?

If I only download one file at a time, what is that going to tell us?

Should I download one file at a time until I see the error?


Ultimately, the original question still remains. Using the TElSimpleFTPSClient.DownloadFiles() method, how do I gracefully handle FTP command errors, how do I recover from them, and how do I retry/continue the download?
#37255
Posted: 07/16/2016 16:25:57
by Eugene Mayevski (EldoS Corp.)

I apologize for the delay in answering, but you have not linked your support access ticket, as described in the licensing email, and so your support status was indicated as basic. Please see my comments below.

Quote
Connexus Credit Union wrote:
Also, the sample doesn't use TElSimpleFTPSClient.DownloadFiles(), it uses TElSimpleFTPSClient.Receive(). Is that then a fair comparison?


You probably need to do minor modifications to the sample in order to get the proper log.

Quote
Connexus Credit Union wrote:
Ultimately, the original question still remains. Using the TElSimpleFTPSClient.DownloadFiles() method, how do I gracefully handle FTP command errors, how do I recover from them, and how do I retry/continue the download?


You do this by using DownloadFile() or DownloadStream() method and writing your own implementation of DownloadFiles().

The components offer you a flexible and extensible tool. The high-level methods like those methods that perform group operation fit the needs of most customers, but in some cases their flexibility is not enough. This is exactly where lower-level methods and even classes come to help.

You can check the source code (although it is in Pascal, not in C#) for how that method is implemented. You can easily rewrite it in C# and add your own error handling.

But the issue hides deeper. Error 213 is reported as invalid because it's probably not valid according to the standard. I.e. the client sends the command and knows, what responses may be given. And the server sends 213 in response. If the result code is not among the allowed ones, then it's reported as erroneous.

Unfortunately wireshark and similar logs are of zero use here - you are dealing with secure connection, which is explicitly designed to prevent looking into the dataflow.

To be able to tell you more, we do need to see the text log of the connection, that includes this error code. It's up to you how you create this log - either by modifying the sample or by adding logging capabilities to your own application.
You can use the sample application to see how it logs what's sent and received (via the corresponding events).


Sincerely yours
Eugene Mayevski
Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.

Reply

Statistics

Topic viewed 983 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!