EldoS | Feel safer!

Software components for data protection, secure storage and transfer

SFTP Upload incomplete but no error

Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.
#14077
Posted: 08/02/2010 10:05:15
by Charles DeWeese (Standard support level)
Joined: 04/17/2008
Posts: 52

Using SBB Pro v8.0.17 when uploading a file via the SFTP component if user exceeds the disk quota or the disk is full the SFTP component isn't able to detect this, the file continues to send until all the data has been sent and then it indicates the upload was successful.

I don't know if the SFTP specs support an error reporting for this situation but if the file isn't uploaded completely obviously the user needs to be warned of this situation, however I don't see any great way to solve this since no error is thrown.

One thought would be to check the file size after upload and compare the source and target size but this doesn't seem to be ideal.

When testing FileZilla I noticed during the transfer it throws an error
"Error: error while writing: failure"

How do I get the SBB SFTP component to trigger a similar error? The OnError event is never triggered.
#14078
Posted: 08/02/2010 10:23:52
by Ken Ivanov (EldoS Corp.)

Thank you for reporting the problem. We will try to reproduce the issue in our environment and get back to you as soon as possible.

I assume that you are using TElSimpleSFTPClient and its UploadFile/UploadStream method, aren't you?
#14079
Posted: 08/02/2010 10:34:30
by Charles DeWeese (Standard support level)
Joined: 04/17/2008
Posts: 52

Yes that is correct, I also tested with TElSftpClient because I thought it might be a limitation of TElSimpleSFTPClient.
#14088
Posted: 08/03/2010 05:39:11
by Mykola Olshevsky (Basic support level)
Joined: 07/07/2005
Posts: 450

Hi. I just tried to reproduce this issue - and everything goes fine, component receives 'failure' message and throws an exception to outer code.
I used ElSimpleSFTPClient demo.
#14090
Posted: 08/03/2010 07:38:28
by Charles DeWeese (Standard support level)
Joined: 04/17/2008
Posts: 52

When I came across this issue it was in a custom program I made using the TElSftpClient component directly, the uploads are asynchronized and it uses SBB3Compatible = true. This seems to be the factor in all of this.

The original ElSimpleSFTPClient demo seems to worked flawlessly, At one point I must of modified it to set SBB3Compatible to true.

As soon as the SBB3Compatible flag is set the problem I experienced is visible.

The problem is that that when doing asynchronized uploads it seems to me that the SBB3Compatible property needs to be true, otherwise the data sometimes arrives on the server in the wrong order corrupting the upload.

I was trying to avoid using a blocking method to upload the files which is why I opted to upload them asynchronized.

Perhaps I'm going about this wrong but my goal was to upload the files in a non-blocking way which is why I wanted to do it asynchronized.
#14093
Posted: 08/03/2010 10:04:36
by Mykola Olshevsky (Basic support level)
Joined: 07/07/2005
Posts: 450

In such case you should use OnError event handler, it is fired when server returns 'failure'. However, it is also fired with 'end of file' message, when there is an end of directory listing.
#14094
Posted: 08/03/2010 10:56:23
by Charles DeWeese (Standard support level)
Joined: 04/17/2008
Posts: 52

Exactly my thought, but the problem I am experiencing is that as far as I can tell the OnError event isn't being triggered when SBB3Compatible is true when sending a file.

Looking at the source code within SBSFTP.pas

procedure TElSftpClient.HandleTransferManagerFinish(Sender: TObject);

The only event being triggered here DoSuccess(FLastSyncComment), nowhere is DoError() and since the LastComment and LastError defined within this procedure isn't set globally the procedure DoError() would need to be called here or those variables would need to be passed along somewhere so that an error can be generated. which does not appear to be the case.

.. and (perhaps an unrelated bug) if you look at the code above it looks like it should be using LastComment which is passed from TElSftpTransferManager and not FLastSyncComment
#14098
Posted: 08/03/2010 12:44:07
by Ken Ivanov (EldoS Corp.)

The only purpose of introducing SBB3Compatible property was the ease of migration from SBB3 to SBB4. The code related to this property has not been updated for a long term, and we do not recommend switching this property on in versions other than SBB4.

Let's have a deeper look into the issue you referenced above:
Quote
The problem is that that when doing asynchronized uploads it seems to me that the SBB3Compatible property needs to be true, otherwise the data sometimes arrives on the server in the wrong order corrupting the upload.

It is very likely to be a bug of the server. SFTP supports strict ordering of packets, and it is technically impossible for the packets to arrive in wrong order. Could you please describe this issue in more detail?
#14113
Posted: 08/05/2010 10:33:49
by Charles DeWeese (Standard support level)
Joined: 04/17/2008
Posts: 52

The corruption issue was a problem I experienced when I first started coding my project, that was roughly 2yrs ago, It was specific to a certain SFTP server and it occurred with zlib-delayed being enabled. Its quite possible that it was an issue with the server and I'm not 100% sure if it was sending the data out of order since its very difficult to debug the SFTP protocol, I recall when doing a binary compare on the uploaded file there was segments of data out of order or in some cases missing from the uploaded file.

I could be mistaken but I'm fairly sure that SBB3Compatible is true by default when using TElSftpClient directly. Only in TElSimpleSFTPClient is SBB3Compatible is it set to false.

I managed to solve my problem by slightly changing
procedure TElSftpClient.HandleTransferManagerFinish(Sender: TObject);

I replaced the block of code:
Code
    else if (Direction = tdUpload) then
          DoSuccess(FLastSyncComment);

with:
Code
    else if (Direction = tdUpload) then
     begin
          if LastErr = 0 then
              DoSuccess(LastComment)
          else
              DoError(LastErr, LastComment);
     end;


Please correct me if I'm wrong but it seems like LastComment should be used instead of FLastSyncComment

With this change in place everything works flawlessly for me and solves the problem I experienced.
#14116
Posted: 08/05/2010 14:08:30
by Ken Ivanov (EldoS Corp.)

Hmm, you are right. I am sorry for misleading you in the previous post. Your fixes seem to be correct as well; thank you for pointing us at that piece of code.

There have been introduced a number of compatibility improvements to the zlib-delayed implementation for last two years. Last testings did not expose any problems with third-party zlib-delayed-capable implementations, so it is very likely that the issue you were experiencing has been already resolved.
Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.

Reply

Statistics

Topic viewed 3079 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!