EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Running out of Connections

Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.
#26241
Posted: 08/28/2013 03:59:36
by Stewart Joyce (Standard support level)
Joined: 08/13/2013
Posts: 10

I have been having issues experiencing connection timeouts (Error Code 10060) as well as Resource temporarily unavailable (Error Code 10035).

I am sending out 30 messages to an external endpoint and note using the utility CurrPorts that 19 connections are made and in the BizCrypto log file 19 messages are sent before I start getting Connection Timeouts.

I have also used FileZilla to send out the 30 messages to the external endpoint and all are sent successfully. This tells me that there are no issues with our firewall or that the endpoint is restricting the number of connections.

The 11 messages that do not get transmitted go into a retry process of waiting 5 minutes until they are tried again. They again fail and I get Connection Timeout errors occurring.

It doesn't appear that the connections are being reused and are hanging around for a period of time preventing further transmission of messages. This is preventing us sending out large numbers of messages in a short burst, 100s are envisaged in the future but we are having problems sending out 30 at the moment.

I have the Socket Timeout set to 60000 milliseconds.
#26256
Posted: 08/31/2013 11:09:08
by Ken Ivanov (EldoS Corp.)

Hello Stewart,

Could you please provide us some details about the issue:

1. What exactly adapter are you using? I assume it's the SFTP one, but we need to be sure.

2. Are you sending all the messages as single-message batches, or there are some subsets of messages sent within the same batches?

3. What is the frequency and sizes of the messages being sent out? Are any of the messages sent (or attempted to be sent) in parallel?
#26263
Posted: 09/02/2013 07:10:05
by Stewart Joyce (Standard support level)
Joined: 08/13/2013
Posts: 10

1. Yes we are using the SFTP Adapter

2. See screen shots in attachment.


#26270
Posted: 09/02/2013 16:44:27
by Ken Ivanov (EldoS Corp.)

Hello Stewart,

Thank you for the adapter configuration details; they will be quite helpful on further stages. Still, could you please answer questions 2 and 3, as they are really important for understanding the circumstances of your BizTalk environment.
#26289
Posted: 09/04/2013 03:17:09
by Stewart Joyce (Standard support level)
Joined: 08/13/2013
Posts: 10

Ken

2. We receive into BizTalk a number of messages 3 times a month which are then required to be sent via SFTP to an external company. Each message is sent individually by a separate SFTP call and there is no batching of messages.

3. The sizes of the messages are fairly consistent in that their format cannot grow i.e. no repeating groups. The size is around 20kb. Up to 1000 messages are received 3 times a month and are released to BizTalk in one go by a batch process on another system. We then process the messages and send in parallel via SFTP the messages to the external company.
#26346
Posted: 09/06/2013 17:45:56
by Ken Ivanov (EldoS Corp.)

Hello Stewart,

Thank you for the details.

1000 messages is a really huge number if each message is sent individually (within a dedicated batch) and most of them are sent out in parallel. In non-batched mode a separate SSH/SFTP connection is established for every single message, and the majority of servers would hardly hold more than 20 connections at a time (not to consider firewalls and IPS which are likely to treat such a situation as a DoS attack).

Please consider packing your messages into batches and using one of the batch modes supported (either greedy or transactional, depending on your requirements), and/or send the messages sequentially. The goal is to reduce the number of parallel SFTP connections established to the server at any given moment.
#26675
Posted: 10/02/2013 04:33:16
by Ed Price (Basic support level)
Joined: 10/02/2013
Posts: 1

Ken - thanks for your reply to Stewart above. Regarding packing into batches, which connection properties do you need to change to switch to batch mode?
#26676
Posted: 10/02/2013 04:40:57
by Ken Ivanov (EldoS Corp.)

Hello Ed,

Thank you for contacting us.

Batching is controlled with the Delivery Type property. You will find the explanation about the differences between transactional and greedy delivery types here.

Please feel free to continue the topic should you need any further assistance.

Ken
Also by EldoS: Callback File System
Create virtual file systems and disks, expose and manage remote data as if they were files on the local disk.

Reply

Statistics

Topic viewed 6079 times

Number of guests: 2, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!