EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Critical Problem - CBFS product

Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.
#3828
Posted: 09/20/2007 13:05:01
by Yevgeny Safovich (Basic support level)
Joined: 09/20/2007
Posts: 6

Hello,

We've already contacted you discussing the possibility to purchase your CBFS product to be used in a solution developed for our company - Comverse.
Since then we were evaluating the product.

Today we have encountered a critical problem that prevents us from using the CBFS for our needs.
Please advise if there is an solution for this problem that can be provided within the next couple of days. (If not, we will have to look for alternatives, which is very challenging in the timeframes of our project.)


Problem Description:

Ok, here's a quote from CBFS documentation:
"Callback functions are always serialized when they use the same file. In other words, if there are two write or read operations on the same file, the callback functions are called first for the first operation, then, after the first callback has returned, for the next operation and so on. The calls are performed by the same thread.
However if there are two pending operations for different files, the callbacks can be called in two ways - either serialized (as described above) or in parallel from two different threads. The default behaviour is to call the callbacks in parallel, and this can be changed to sequential call using SerializeCallbacks property."

The problem is - you do not guarantee that it will be called from separate threads, you say it may be called like this (Ronen from my group has discussed this with you on the forum). In reality (at least on my laptop and on several other PCs here) you either serialize all of the calls to a single thread, or use a thread pool of one or two threads to do the callbacks. That means if we have a blocking operation in one of the callbacks that takes a long time, then all of the explorer windows that work with the virtual drive are frozen till the blocking operation completes!

We need a change in the driver behaviour to call our callbacks from different thread for each file and to use a reasonably sized thread pool (of 20 threads or so) so user won't have to wait.

Otherwise we can NOT use the Eldos driver.


I will truely appreciate a prompt response.

Thank you.
#3829
Posted: 09/20/2007 13:17:36
by Eugene Mayevski (EldoS Corp.)

Yes, the pool exists and is controlled by the internal variable, which is currently initialized to the number of processors (that's why you get 1 or 2). It's not a problem to add a property which specifies the pool size. However, it's possible that the OS will be in blocking state anyway, i.e. it won't be able to issue 20 requests in parallel (even for different files). This won't happen always, but I would treat this as a possibility.


Sincerely yours
Eugene Mayevski
#3830
Posted: 09/20/2007 13:41:12
by Yevgeny Safovich (Basic support level)
Joined: 09/20/2007
Posts: 6

Thank you for the immediate response!

1. Can you elaborate what are the circamstances in which the OS may be in a blocking state? What does this depend on? Is there a special reason you limited the thread pool for a the amount of the processors?

2. Is it possible to get a trial version with the property you mentioned to check this option? What is the soonest when you can provide such a version? We are working almost around the clock, including the weekends, to hold the schedules of our project. If you provide us the version, this will allow us to decide on our next steps and to define the direction.

Thanks a lot in advance!

Best Regards,
Yevgeny Safovich


#3831
Posted: 09/20/2007 14:03:16
by Eugene Mayevski (EldoS Corp.)

1. I said "possible". Only Microsoft kernel developers can tell you for sure.
1a. We don't answer the questions "why it's made so", sorry. Curiosity is an option for registered developers.
2. There will be a beta of CBFS 1.1 with security and streams support pretty soon (in about a week). The new property will go to that beta version.


Sincerely yours
Eugene Mayevski
#3832
Posted: 09/20/2007 16:27:44
by Yevgeny Safovich (Basic support level)
Joined: 09/20/2007
Posts: 6

1. Can you provide a more precise date for the beta release (this will allow us to plan our development tasks better)
2. When the final version of CBFS 1.1 is going to be released (including this feature)?
3. Can we be certain that in "normal" circumstances it will be possible to perform multiple long running operation on the same CBFS drive (using the Property you mentioned)? Is there a way we can check this in our application? (Maybe you can provide us a preliminary alpha version?)


Thank you.

Best Regards,
Yevgeny Safovich
#3845
Posted: 09/21/2007 10:35:21
by Volodymyr Zinin (EldoS Corp.)

We are going to make the new build (which will include the new features) of the product in the next week.

Quote
Yevgeny Safovich wrote:
Can we be certain that in "normal" circumstances it will be possible to perform multiple long running operation on the same CBFS drive (using the Property you mentioned)?

In the certain "normal" circumstances it's still possible that an OS will be in a "blocking state" until a certain request is processed. Sure, in the future versions of the product we are going to reduce amount of such situations, but it is not a trivial task.
#3849
Posted: 09/21/2007 12:21:06
by Eugene Mayevski (EldoS Corp.)

0. Please forgive me my possibly harsh words. With the amount of work we have, we need to be as short and precise with words as possible. No time for sentiments, you know.
1. As soon as possible. From our experience, *any* estimated time must be doubled, so I don't want to set the dates. We have work for 3 work days before the beta. This will surely lead to a full week. Pushing the developers won't help, and I can't do the job of over a dozen of programmers myself to speed up things.
2. The new version introduces additional callbacks and a couple of minor properties (including "yours"). We have several requests for these callbacks and we want the users to test them. The previous functionality should not be broken with new beta, so as soon as we have positive feedback on the new callbacks, we will release 1.1 to public. This can take 4-6 weeks. This is a realistic estimate.
3. Vladimir has written the whole essay for me to understand what the situation is about. In brief, we need to introduce the new property for you and you will check with your scenarios. 10-20 threads in the pool should give some speed improvement, however there exist combinations of requests when some blocking will occur.
Example: two Windows Explorer windows navigate to different subfolders of the same folder. To do this, each navigation opens a subfolder. Opening a subfolder means opening all parent folders of this subfolder. Now, if the two subfolders have the common parent, two open operations will influence each other. It will be your task to optimize "open directory" operation so that two parallel requests for directory opening are not processed twice in full, but are cached in some way.
In general, all our products are designed with smart programmer in mind. We offer flexibility, on top of which the developer is able to build as efficient system as he needs. There will be some work involved, of course.


Sincerely yours
Eugene Mayevski
#3863
Posted: 09/26/2007 06:25:17
by Petr Chmelar (Basic support level)
Joined: 07/30/2007
Posts: 24

Would it be possible to add a property whether streams are supported or not? So that if I set the property i.e. StreamsSupported = false it would behave in the same way as it behaves now (i.e. the callbacks will not get stream operation requests)?

Thanks,
Petr Chmelar

Quote
Eugene Mayevski wrote:
1. I said "possible". Only Microsoft kernel developers can tell you for sure.
1a. We don't answer the questions "why it's made so", sorry. Curiosity is an option for registered developers.
2. There will be a beta of CBFS 1.1 with security and streams support pretty soon (in about a week). The new property will go to that beta version.
#3864
Posted: 09/26/2007 07:04:44
by Volodymyr Zinin (EldoS Corp.)

Sure. This feature will be optional.
#3892
Posted: 09/29/2007 05:03:21
by Yevgeny Safovich (Basic support level)
Joined: 09/20/2007
Posts: 6

Hello!

Can we get the new build of this product with the property to control the amount of threads?

Thank you!
Yevgeny Safovich
Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.

Reply

Statistics

Topic viewed 16304 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!