EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Min/MaxThreadCount thread pooled, or cycled?

Also by EldoS: SecureBlackbox
200+ components and classes for digital security, signing, encryption and secure networking.
Posted: 05/10/2016 20:36:20
by Marius Storm-Olsen (Priority Standard support level)
Joined: 10/01/2015
Posts: 3


I'm wondering if when you set a Min and MaxThreadCount for the callbackfilter, will the threads created to support the callback handling be static, or can they be terminated, created again, so you end up with a cycling set of "MaxThreadCount" number of varying threads?

I'm asking because I would like to use ZeroMQ for communication, but it requires a separate inproc "socket" per thread, so I would like to lazily create a new one only when I hit a new thread, up to MaxThreadCount. However, if the threads can be torn down, and new ones created, there will be a cycling effect, and I might end up spending more time creating new sockets than I would like.

Also, my internal engine for the callbackfilter usage has a static amount of threads, and it would be great if the threads didn't cycle, as I could then allocate each internal static worker thread to each thread from CallbackFilter.

So, any details on how threads are created/destroyed/increased/decreased in CallbackFilter would be greatly appreciated!

Posted: 05/11/2016 02:37:31
by Volodymyr Zinin (EldoS Corp.)


Usually the worker thread pool contains the currently used threads (threads that are calling the callbacks) plus several more threads that are waiting for new requests. In the case a lot of waiting threads exists then some of them are terminated. If it is required more threads to process requests then new ones are created. But the number of threads are always in the span of Min and MaxThreadCount.
Posted: 05/11/2016 03:17:06
by Eugene Mayevski (EldoS Corp.)

To add to Volodymyr's answer, in your case it makes sense to employ a pool of sockets and/or threads and not bind them to CallbackFilter internal worker threads, but rather use some locks / monitors . All in all, I understand it so that you don't need strict binding of threads/sockets, but rather you need to guarantee that one socket is used by one thread.

Sincerely yours
Eugene Mayevski
Posted: 05/11/2016 14:06:44
by Marius Storm-Olsen (Priority Standard support level)
Joined: 10/01/2015
Posts: 3

Well, in the case of ZMQ (http://zguide.zeromq.org/page:all#Multithreading-with-ZeroMQ), the development paradigm is to avoid locking mechanisms completely, and just use the inproc messaging scheme between threads. However, this assumes that the ZMQ socket is created on the thread which passes the message.

If CBF cycles threads at random times, I won't be able to rely on that, and would require locking individual threads just to pass a message. And I wouldn't know when to destroy a previous socket created on the thread which has disappeared from the pool.

Is there a guarantee that there will be no thread cycling if I set Min/Max to the same? So, if I set Min=8, Max=8, I will always have the exact same 8 threads?
Posted: 05/11/2016 18:20:35
by Marius Storm-Olsen (Priority Standard support level)
Joined: 10/01/2015
Posts: 3

I guess most ideally would be to have CBF callbacks for the creation/deletion of CBF worker threads, allowing me to create/destroy application-specific TLS or assign a worker thread specific pointer (much like the PVOID* UserContext on I/O callbacks).

That way I could easily
MyStruct *threadData = (MyStruct*)cbFlt.GetWorkerThreadData();
on a callback function, to get thread-specific contexts.
Posted: 05/11/2016 23:21:47
by Eugene Mayevski (EldoS Corp.)

Yes, we have this idea in the wishlist for Callback File System, and we can implement it for all products. I have added the task to the ToDo list.

Sincerely yours
Eugene Mayevski
Also by EldoS: Callback File System
Create virtual file systems and disks, expose and manage remote data as if they were files on the local disk.



Topic viewed 2424 times

Number of guests: 1, registered members: 0, in total hidden: 0


Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!