EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Closing a socket

Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.
Posted: 05/25/2011 10:45:36
by Ian  (Basic support level)
Joined: 02/23/2010
Posts: 26

In my message handler - there are times when depending on the message code or the data itself, I want to basically close/shutdown the socket quickly - so no response is sent back to the message source. What's the best/cleanest way to do that (which will force the client to re-connect)...

ie. where normally I would respond to a message by doing something

I want to do something like


By the way, how do I get the socket from the MCMessage pointer when I'm inside a message handler?

Do I use transport->GetMessageByID(mcmessage->MsgID)? Then how do I get the socket associated with this particular message?

Posted: 05/25/2011 11:05:53
by Eugene Mayevski (EldoS Corp.)

MsgConnect is not connection-oriented and sockets are just a backend transport. The message know nothing about sockets, neither should the applications. So simple answer to all of your questions is "you can't do this cause it's not the way MsgConnect was designed".

Sincerely yours
Eugene Mayevski
Posted: 05/25/2011 18:11:11
by Ian  (Basic support level)
Joined: 02/23/2010
Posts: 26

Ok, so I should get the client to do the connect, send whatever it needs to, then drop the connection by de-activating the transport?

That does work for me, I was hoping I could get the server to do it as well!

Posted: 05/26/2011 01:05:08
by Eugene Mayevski (EldoS Corp.)

If you tell me what the reason for forceful disconnection is, maybe I'd be able to give some advice. So far it doesn't make much sense to me.

Sincerely yours
Eugene Mayevski
Posted: 05/31/2011 07:10:57
by Ian  (Basic support level)
Joined: 02/23/2010
Posts: 26

All the clients are on 3G or GPRS connections. I am finding that many of them aren't correctly disconnecting from the cell tower or somewhere in the network path, so consequently, the server is maintaining what it thinks is a valid connection, while continuing to accept what looks like a new connection (but is in fact, a previously disconnected device that hasn't correctly completed the FIN handshake).

So what happens is the server is picking up lots of spurious new connections from the same source IP but different ports. Eventually it may or may not close those previous connections, but again, die to problems or delays in the 3G/GPRS network, sometimes the FIN handshake doesn't get completed, so all those sockets on the MessageConnect end doesn't ever get closed, but remains open - so I end up with a lot of thread connections that never do anything...

So now my customer is up to 8000+ devices, but I don't get past about 3000 or so, as I start running out of memory/threads. These devices are supposed to connect, send data, disconnect, so in reality, there should only be 2000-2500 devices simultaneously active, but I can the thread count going way past that, with of course no actual network activity on them.

That's why I need to detect connections that appear to be hanging around a while )or are sending a particular type of message) and forcefully diconnect them to allow the existing threads to be re-used correctly...

Any help would be greatly appreciated!
Posted: 06/05/2011 09:58:10
by Ian  (Basic support level)
Joined: 02/23/2010
Posts: 26

ok, never mind - I set the inactivity (idle) timeout and had message connect automatically disconnect devices that were idle for a period of time - allowing me to re-use the threads...

I've set the initial thread size to 2500, gave them a stack size of 256K each (use linker options)...

Now it's humming away, memory consumption goes to about 160MB, then remains pretty stable at that point...

I could do more active/simultaneous connections - but there's not much point and this way, memory use is (relatively) low, thread count is high but not too high and the CPU use for the processing I do is less than 5% most of the time..

I'll wait till you have the IOCP version to push the server more!

This way, Message Connect is handling about one message per second, 24 hours a day... All good!
Posted: 06/05/2011 11:51:32
by Eugene Mayevski (EldoS Corp.)

There's a thread pool inside and if you have many connections, you can play with pool size to minimize thread allocation and disposal.

Sincerely yours
Eugene Mayevski
Posted: 06/05/2011 20:28:48
by Ian  (Basic support level)
Joined: 02/23/2010
Posts: 26

Yup - that's what I did - initialised thread pool - only change I made to the code is to explicitly add the parameter STACK_SIZE_PARAM_IS_A_RESERVATION to the CreateThread() call and put in a stack reservation size - this way it can differ from the global setting set by the linker options.

Otherwise, everything else I can control using all the various setXXX() API calls in the transport and messenger classes...

After all this time, I'm still "discovering" APIs I hadn't used before - but now I've had time to trace the code and see what's happening and what some of them actually do!
Posted: 07/03/2011 09:15:04
by Ian  (Basic support level)
Joined: 02/23/2010
Posts: 26

Just an update...

I'm now running MessageConnect (and my app) in 64 bit mode. Gives me a little more memory breathing room... My final configuration is running 2500 connections (threads), each with 128K stack. I have the disconnect on idle set to one hour. I'm averaging about 60,000 transactions per day using a single instance of the server and has basically been up for several weeks now non-stop. Memory consumption is stable and network and CPU consumption are both well under 3.5% (on the comms side - of course slightly higher CPU consumption on the database side)...

We have approx 8100 devices connecting throughout the day and about 10 users online administering and monitoring the system - all comms using MessageConnect...

I'm planning on running a Linux based load balancer on the WAN interface and up to about 5 instances of the server app on the internal LAN interface. This will give me the scalability I'm currently missing until your IOCP version is ready! LOL...!!

So for those deciding on whether or not MessageConnect can handle the traffic, scale, efficiency and reliability (on Windows no less) - here's proof!

We've kept all our competitors away, with the fact none of them can match the number of connections per box and the low memory and CPU requirements for the volume of transactions we're doing!
Posted: 07/03/2011 09:37:03
by Eugene Mayevski (EldoS Corp.)

Thank you very much for detailed information. Indeed IOCP version suffers from certain delays, but if the current one works after tune-up, that's great!

Sincerely yours
Eugene Mayevski
Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.



Topic viewed 6190 times

Number of guests: 1, registered members: 0, in total hidden: 0


Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!