EldoS | Feel safer!

Software components for data protection, secure storage and transfer

ReadDirectory() method fails for LARGE listing in WSFTP 7.1

Also by EldoS: Callback File System
Create virtual file systems and disks, expose and manage remote data as if they were files on the local disk.
#10539
Posted: 07/07/2009 17:16:28
by Eric Heimerman (Basic support level)
Joined: 07/07/2009
Posts: 3

I am having a similar problem to [URL=http://www.eldos.com/forum/read.php?FID=7&TID=1832&MID=10313&phrase_id=406946#message10313]this thread[/URL], but I am not downloading a large file, I'm retrieving a large directory listing with the ReadDirectory() method. The directory contains over 5000 JPG images, of about 230MB.

I am able to connect and authenticate just fine, but as soon as I try to read that folder with 5000 images in it, the WS_FTP SSH service crashes on the server, and the client disconnects with error 105.

I've installed the latest version of WS_FTP Server (7.1) on a fresh Windows 2003 Server (64bit) with the PostgreSQL backend.

I downloaded the latest beta version of SecureBlackBox because of the reported improved WS_FTP compatibility. The webpage says it should be 7.1.159, but when I right-click the SecureBlackBox.dll and look at properties, it shows version 7.1.0.158.

I have tried the suggestions in the linked thread about setting AutoAdjustTransferBlock to false, PipelineLength to 1, DownloadBlockSize to 32768, but those settings did not work in my case. Even setting ASCIIMode to true did not help.

I have tried using both the SimpleSFTPClient demo program and the SophisticatedClient demo program. The simple demo also returned error code 105 and crashed the WS_FTP service. The sophisticated demo never reported an error, but did not list the files and also resulted in the service crashing.

Here's the log from WS_FTP when the service crashes:

Quote
07/07/2009 15:43:34 SSH 10.4.1.144:8022 formsb01.cm.pam 04032828 192.168.62.105:1678 msbautomation Success 0 Normal Completed publickey Authentication. User logged in
07/07/2009 15:43:34 channel 10.4.1.144:8022 formsb01.cm.pam 04032828 192.168.62.105:1678 msbautomation Information 0 Normal SFTP subsystem started in channel s_Id: 45, c_Id: 1, c_Window: 32749, c_MaxPacket: 262144, s_Window: 300000, s_MaxPacket: 30000
07/07/2009 15:43:58 SSH 10.4.1.144:8022 formsb01.cm.pam 04032828 192.168.62.105:1678 msbautomation Operating System Error 0 Error OnPacketReceive, Attempted to read or write protected memory. This is often an indication that other memory is corrupt.: at SSHServerAPI.Transport.Core._ProcessReadPacket(SSH2BufferStream _Packet, Byte[] PacketMAC, Byte[] ReadMAC) at SSHServerAPI.Transport.Core._OnPacketRecv(IAsyncResult ar)


I have tried connecting with both WinSCP and Anyclient, and both work fine. Here is the session log from WinSCP, if that helps (next post):
#10540
Posted: 07/07/2009 17:24:32
by Eric Heimerman (Basic support level)
Joined: 07/07/2009
Posts: 3

Quote

2009-07-07 16:16:55.567 --------------------------------------------------------------------------
2009-07-07 16:16:55.567 WinSCP Version 4.1.9 (Build 416) (OS 5.2.3790 Service Pack 2)
2009-07-07 16:16:55.567 Login time: Tuesday, July 07, 2009 4:16:55 PM
2009-07-07 16:16:55.567 --------------------------------------------------------------------------
2009-07-07 16:16:55.567 Session name: msbautomation@formsb01.cm.pam
2009-07-07 16:16:55.567 Host name: formsb01 (Port: 8022)
2009-07-07 16:16:55.567 User name: msbautomation (Password: No, Key file: Yes)
2009-07-07 16:16:55.567 Tunnel: No
2009-07-07 16:16:55.567 Transfer Protocol: SFTP (SCP)
2009-07-07 16:16:55.567 Ping type: -, Ping interval: 30 sec; Timeout: 15 sec
2009-07-07 16:16:55.567 Proxy: none
2009-07-07 16:16:55.567 SSH protocol version: 2; Compression: No
2009-07-07 16:16:55.567 Bypass authentication: No
2009-07-07 16:16:55.567 Try agent: Yes; Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: No
2009-07-07 16:16:55.567 Ciphers: aes,blowfish,3des,WARN,arcfour,des; Ssh2DES: No
2009-07-07 16:16:55.567 SSH Bugs: -,-,-,-,-,-,-,-
2009-07-07 16:16:55.567 SFTP Bugs: -,-
2009-07-07 16:16:55.567 Return code variable: Autodetect; Lookup user groups: Yes
2009-07-07 16:16:55.567 Shell: default, EOL: 0
2009-07-07 16:16:55.567 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes
2009-07-07 16:16:55.567 LS: ls -la, Ign LS warn: Yes, Scp1 Comp: No
2009-07-07 16:16:55.567 Local directory: default, Remote directory: home, Update: No, Cache: Yes
2009-07-07 16:16:55.567 Cache directory changes: Yes, Permanent: Yes
2009-07-07 16:16:55.567 DST mode: 1
2009-07-07 16:16:55.567 --------------------------------------------------------------------------
2009-07-07 16:16:55.582 Looking up host "formsb01"
2009-07-07 16:16:55.582 Connecting to 10.4.1.144 port 8022
2009-07-07 16:16:55.582 Server version: SSH-2.0-WS_FTP-SSH_7.1
2009-07-07 16:16:55.582 We claim version: SSH-2.0-WinSCP_release_4.1.9
2009-07-07 16:16:55.598 SSPI: acquired credentials for: eheimerman@CM.PAM
2009-07-07 16:16:55.598 Constructed service principal name 'host/formsb01'
2009-07-07 16:16:55.598 Enabling GSSKEX for this target
2009-07-07 16:16:55.629 Using SSH protocol version 2
2009-07-07 16:16:55.629 Using Diffie-Hellman with standard group "group14"
2009-07-07 16:16:55.629 Doing Diffie-Hellman key exchange with hash SHA-1
2009-07-07 16:16:55.770 Host key fingerprint is:
2009-07-07 16:16:55.770 ssh-rsa 1024 30:0b:b7:78:14:5a:3c:aa:32:de:2f:78:37:2f:f6:b8
2009-07-07 16:16:55.770 Initialised AES-256 CBC client->server encryption
2009-07-07 16:16:55.770 Initialised HMAC-SHA1 client->server MAC algorithm
2009-07-07 16:16:55.770 Initialised AES-256 CBC server->client encryption
2009-07-07 16:16:55.770 Initialised HMAC-SHA1 server->client MAC algorithm
2009-07-07 16:16:55.770 Reading private key file "\\nasmsb1\shared\AppSupport\Automate\includes\automate-test.ppk"
2009-07-07 16:16:55.770 Using username "msbautomation".
2009-07-07 16:16:55.801 Offered public key
2009-07-07 16:16:55.801 Offer of public key accepted
2009-07-07 16:16:55.801 Authenticating with public key "imported-openssh-key"
2009-07-07 16:16:55.864 Access granted
2009-07-07 16:16:55.864 Opened channel for session
2009-07-07 16:16:55.864 Started a shell/command
2009-07-07 16:16:55.864 --------------------------------------------------------------------------
2009-07-07 16:16:55.864 Using SFTP protocol.
2009-07-07 16:16:55.864 Doing startup conversation with host.
2009-07-07 16:16:55.864 Type: SSH_FXP_INIT, Size: 5, Number: -1
2009-07-07 16:16:55.864 Type: SSH_FXP_VERSION, Size: 5, Number: -1
2009-07-07 16:16:55.864 SFTP version 4 negotiated.
2009-07-07 16:16:55.864 We will use UTF-8 strings when appropriate
2009-07-07 16:16:55.864 Getting current directory name.
2009-07-07 16:16:55.864 Getting real path for '.'
2009-07-07 16:16:55.864 Type: SSH_FXP_REALPATH, Size: 10, Number: 16
2009-07-07 16:16:55.895 Type: SSH_FXP_NAME, Size: 28, Number: 16
2009-07-07 16:16:55.895 Real path is '/users'
2009-07-07 16:16:55.895 Listing directory "/users".
2009-07-07 16:16:55.895 Type: SSH_FXP_OPENDIR, Size: 15, Number: 267
2009-07-07 16:16:55.895 Type: SSH_FXP_HANDLE, Size: 10, Number: 267
2009-07-07 16:16:55.895 Type: SSH_FXP_READDIR, Size: 10, Number: 524
2009-07-07 16:16:55.910 Type: SSH_FXP_NAME, Size: 133, Number: 524
2009-07-07 16:16:55.910 Type: SSH_FXP_READDIR, Size: 10, Number: 780
2009-07-07 16:16:55.910 Type: SSH_FXP_STATUS, Size: 28, Number: 780
2009-07-07 16:16:55.910 Status/error code: 1
2009-07-07 16:16:55.910 Type: SSH_FXP_CLOSE, Size: 10, Number: 1028
2009-07-07 16:16:55.910 Startup conversation with host finished.
2009-07-07 16:16:59.473 Cached directory change via "/users/eheimerman" to "/users/eheimerman".
2009-07-07 16:16:59.473 Getting current directory name.
2009-07-07 16:16:59.473 Listing directory "/users/eheimerman".
2009-07-07 16:16:59.473 Type: SSH_FXP_OPENDIR, Size: 26, Number: 1291
2009-07-07 16:16:59.488 Type: SSH_FXP_STATUS, Size: 17, Number: 1028
2009-07-07 16:16:59.488 Discarding reserved response
2009-07-07 16:16:59.488 Type: SSH_FXP_HANDLE, Size: 10, Number: 1291
2009-07-07 16:16:59.488 Type: SSH_FXP_READDIR, Size: 10, Number: 1548
2009-07-07 16:16:59.488 Type: SSH_FXP_NAME, Size: 16330, Number: 1548
2009-07-07 16:16:59.488 Type: SSH_FXP_READDIR, Size: 10, Number: 1804
2009-07-07 16:16:59.504 Type: SSH_FXP_NAME, Size: 16315, Number: 1804
2009-07-07 16:16:59.504 Type: SSH_FXP_READDIR, Size: 10, Number: 2060
2009-07-07 16:16:59.520 Type: SSH_FXP_NAME, Size: 16318, Number: 2060
2009-07-07 16:16:59.520 Type: SSH_FXP_READDIR, Size: 10, Number: 2316
2009-07-07 16:16:59.520 Type: SSH_FXP_NAME, Size: 16315, Number: 2316
2009-07-07 16:16:59.520 Type: SSH_FXP_READDIR, Size: 10, Number: 2572
2009-07-07 16:16:59.535 Type: SSH_FXP_NAME, Size: 16339, Number: 2572
2009-07-07 16:16:59.535 Type: SSH_FXP_READDIR, Size: 10, Number: 2828
2009-07-07 16:16:59.535 Type: SSH_FXP_NAME, Size: 16311, Number: 2828
2009-07-07 16:16:59.535 Type: SSH_FXP_READDIR, Size: 10, Number: 3084
2009-07-07 16:16:59.551 Type: SSH_FXP_NAME, Size: 16311, Number: 3084
2009-07-07 16:16:59.551 Type: SSH_FXP_READDIR, Size: 10, Number: 3340
2009-07-07 16:16:59.551 Type: SSH_FXP_NAME, Size: 16311, Number: 3340
2009-07-07 16:16:59.551 Type: SSH_FXP_READDIR, Size: 10, Number: 3596
2009-07-07 16:16:59.567 Type: SSH_FXP_NAME, Size: 16311, Number: 3596
2009-07-07 16:16:59.567 Type: SSH_FXP_READDIR, Size: 10, Number: 3852
2009-07-07 16:16:59.582 Type: SSH_FXP_NAME, Size: 16311, Number: 3852
2009-07-07 16:16:59.582 Type: SSH_FXP_READDIR, Size: 10, Number: 4108
2009-07-07 16:16:59.582 Type: SSH_FXP_NAME, Size: 16311, Number: 4108
2009-07-07 16:16:59.582 Type: SSH_FXP_READDIR, Size: 10, Number: 4364
2009-07-07 16:16:59.598 Type: SSH_FXP_NAME, Size: 16311, Number: 4364
2009-07-07 16:16:59.598 Type: SSH_FXP_READDIR, Size: 10, Number: 4620
2009-07-07 16:16:59.598 Type: SSH_FXP_NAME, Size: 16311, Number: 4620
2009-07-07 16:16:59.598 Type: SSH_FXP_READDIR, Size: 10, Number: 4876
2009-07-07 16:16:59.613 Type: SSH_FXP_NAME, Size: 16311, Number: 4876
2009-07-07 16:16:59.613 Type: SSH_FXP_READDIR, Size: 10, Number: 5132
2009-07-07 16:16:59.613 Type: SSH_FXP_NAME, Size: 16311, Number: 5132
2009-07-07 16:16:59.613 Type: SSH_FXP_READDIR, Size: 10, Number: 5388
2009-07-07 16:16:59.629 Type: SSH_FXP_NAME, Size: 16311, Number: 5388
2009-07-07 16:16:59.629 Type: SSH_FXP_READDIR, Size: 10, Number: 5644
2009-07-07 16:16:59.629 Type: SSH_FXP_NAME, Size: 16311, Number: 5644
2009-07-07 16:16:59.629 Type: SSH_FXP_READDIR, Size: 10, Number: 5900
2009-07-07 16:16:59.629 Type: SSH_FXP_NAME, Size: 16311, Number: 5900
2009-07-07 16:16:59.629 Type: SSH_FXP_READDIR, Size: 10, Number: 6156
2009-07-07 16:16:59.645 Type: SSH_FXP_NAME, Size: 16311, Number: 6156
2009-07-07 16:16:59.645 Type: SSH_FXP_READDIR, Size: 10, Number: 6412
2009-07-07 16:16:59.645 Type: SSH_FXP_NAME, Size: 16311, Number: 6412
2009-07-07 16:16:59.645 Type: SSH_FXP_READDIR, Size: 10, Number: 6668
2009-07-07 16:16:59.660 Type: SSH_FXP_NAME, Size: 9843, Number: 6668
2009-07-07 16:16:59.660 Type: SSH_FXP_READDIR, Size: 10, Number: 6924
2009-07-07 16:16:59.660 Type: SSH_FXP_STATUS, Size: 28, Number: 6924
2009-07-07 16:16:59.660 Status/error code: 1
2009-07-07 16:16:59.660 Type: SSH_FXP_CLOSE, Size: 10, Number: 7172
2009-07-07 16:17:11.863 Closing connection.
2009-07-07 16:17:11.863 Sending special code: 12
2009-07-07 16:17:11.863 Sent EOF message
#10541
Posted: 07/07/2009 18:20:00
by Eric Heimerman (Basic support level)
Joined: 07/07/2009
Posts: 3

I just threw a dozen or so text files in the same folder, and attempted to call the ReadDirectory() method with the Mask parameter, but got the exact same result.
#10542
Posted: 07/08/2009 00:16:07
by Ken Ivanov (EldoS Corp.)

Thank you for contacting us.

Let's continue the conversation in the Helpdesk. I have created a ticket for you.
#10980
Posted: 09/07/2009 10:08:38
by Frank Munsberg (Standard support level)
Joined: 06/04/2009
Posts: 47

May I ask, if there was any conclusion to this error?
We are also experiencing a problem where the WS_FTP 7.1 Server SSH Service is crashing and this does sound related.
#10981
Posted: 09/07/2009 10:25:19
by Ken Ivanov (EldoS Corp.)

Thank you for contacting us.

Mr. Heimerman's issue (WS_FTP's long listings issue, to be exact) has been resolved in SBB 7.1, i.e. the currently available build does not expose that problem. So first of all please install the latest build and check if the upgrade helps. If you are already running the latest build, please provide us the details of your problem (the exact steps that lead to the server crash would be enough as a first step).
Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.

Reply

Statistics

Topic viewed 4129 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!