EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Would like to slow down FTPS adapter.

Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.
Posted: 07/02/2009 09:59:36
by Brian Swenson (Standard support level)
Joined: 07/02/2009
Posts: 7

I'm looking for a way to slow down the FTPS adapter, both sending and receiving. We have 24000+ files to send to an external FTP server. We have been dropping the files into a receive location 200 at a time. The problem is BizTalk processes these files too quickly and opens too many connections on the external FTP server. This in turn is causing the vendors FTP server to crash. Once the files are processed, by the vendor, then we need to pick them back up from their FTP server. Is there a way to have BizTalk only monitor their FTP server say every 10 minutes instead of real time? They would like us to not have so many connection to their FTP server. Thanks!
Posted: 07/02/2009 12:09:34
by Ken Ivanov (EldoS Corp.)

Thank you for contacting us.

First of all, let's clarify the scheme of your business process. Do I understand you right that your workflow looks like the following:
1) A number of files is put [manually?] to some local location,
2) FILE adapter monitors that local location and posts the found message(s) to the orchestration,
3) The orchestration passes the received message(s) to BizCrypto FTPS adapter, which uploads the message(s) to the vendor's server,
4) Another receive location monitors the same server looking for the processed files and downloads them?
Posted: 07/02/2009 15:36:35
by Brian Swenson (Standard support level)
Joined: 07/02/2009
Posts: 7

Let me help clarify the process flow:
1) We have a local folder where a business person drops files manually (at least for now).
2) We pick up the files using the BizTalk File adapter.
3) A send port, with filter based on receive port, picks up these messages from messagebox and sends them to another local folder using the Eldos PGP encryption send adapter.
4) Next we have a second receive location watching for these newly encrypted files and it picks them up again using the BizTalk File adapter.
5) Then a second send port, with filter based on receive port, send these files to external vendor using Eldos FTPS send adapter.

Once option is to slow the initial dropping of the files down from 200 to 10 or lower. In general, I am wondering if there is a good way to have the FTPS send adapter restrict the number of files (connections) it sends to the external FTP site. Also curious to know if it can poll the external FTP server less frequently... I'm guessing I can use the Polling Interval in the Bizcrypto FTPS configuration settings?
Posted: 07/03/2009 02:56:30
by Ken Ivanov (EldoS Corp.)

Thank you for the explanation.

First, why don't you use OpenPGP pipeline instead of adapter to protect the file? It will allow you encrypt and upload files using a single send port. The pipeline will encrypt the data obtained from the File receive adapter and pass it to FTPS adapter for uploading.

I assume that the server-side problem is caused not by the receive but by the send adapter. Receive adapter uses a single control connection to download all the available files, so it is unlikely to be the reason for the server crash. At the same time, send adapter uses a separate control connection for each file, so a big number of files on input makes it establish a big number of control connections to server.

You can limit the number of parallel control connections to server by setting "Number of messages in batch" property of the File adapter to 1 (Configure -> Batching tab). This might be enough to resolve the problem.

We actually do have plans of implementing persistent mode for the send adapter, so that a single persistent connection would be used to upload all the files to server. However, it is likely to be available in 7.2 version.



Topic viewed 4440 times

Number of guests: 1, registered members: 0, in total hidden: 0


Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!