EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Evaluation issues on Vista x64

Also by EldoS: SecureBlackbox
200+ components and classes for digital security, signing, encryption and secure networking.
#14276
Posted: 08/23/2010 11:44:44
by Eugene Mayevski (EldoS Corp.)

Do your crashes happen with sample applications? Can you please check? Lots of things were changed in version 3, and it's unlikely that the old issues re-appeared in version 3. I guess there's something in your code causing a problem, but we need a reliable way to reproduce it.


Sincerely yours
Eugene Mayevski
#21685
Posted: 09/24/2012 15:35:27
by Jose Battig (Basic support level)
Joined: 09/24/2012
Posts: 10

We are having the same issue when listing directories. I can't tell how many hours I have burned personally together with a developer in my Company.
We have put code in all pieces we use/wrote that validate not reusing objects that have been previously destroyed (we have been thinking so far the problem is in our code).
Now we began thinking that the overrun is either Delphi's code (I've seen them with Strings but back in Delphi 5 when using some strange casting that semantically are valid) or Eldos internal code overrunning the Delphi heap.
I don't have a decoupled sample that reproduces the issue, neither I have access to Eldos source (we have not bought the product *yet*) so I can suggest where the issue might be.
#21687
Posted: 09/24/2012 16:04:40
by Jose Battig (Basic support level)
Joined: 09/24/2012
Posts: 10

Quote
Jose Battig wrote:
We are having the same issue when listing directories. I can't tell how many hours I have burned personally together with a developer in my Company.
...issue might be.


Something interesting worth adding.
When serializing calls, either using the SerializeCalls property or via a critical section on the callbacks, the system seams to perform in a more stable manner though I can't tell the overrun or corruption doesn't happen anymore...
#21688
Posted: 09/24/2012 16:23:01
by Jose Battig (Basic support level)
Joined: 09/24/2012
Posts: 10

A couple of more things.
We noticed that the memory corruption issue is more prevalent the more enumerate Directory callback is called.
Finally, something interesting.
When serializing the calls using cbfs provided property, the problem seems to be solved.
When we serialize via a critical section on every calls, the application is less prone to make this issue appear, but nevertheless we can still cause it.

Something interesting worth noting.

On the callback prototype:


Code
procedure TForm.FCbFSEnumerateDirectory(Sender: TObject; DirectoryInfo: TCbFsFileInfo; var EnumerationContext: Pointer;
  Mask: TCBString; Index: Integer; Restart: Boolean; var FileFound: Boolean; var FileName: TCBString; var FileNameLength: Word;
  var ShortFileName: TCBString; var ShortFileNameLength: Byte; var CreationTime, LastAccessTime, LastWriteTime: TDateTime;
  var EndOfFile, AllocationSize, FileId: Int64; var FileAttributes: Cardinal);


I wonder if
Code
DirectoryInfo: TCbFsFileInfo
is potentially cached internally within cbfs guts, and there's some concurrent access from multiple worker threads to this object without proper serialization?

Also, when calling
Code
procedure TForm.FCbFSGetFileInfo(Sender: TObject; FileName: TCBString; var FileExists: Boolean; var CreationTime, LastAccessTime, LastWriteTime: TDateTime; var EndOfFile, AllocationSize, FileId: Int64; var FileAttributes: Cardinal; var LongFileName: TCBString; var LongFileNameLength: Word);


I imagine that callback fills in the DirectoryInfo or maybe some kind of FileInfo objects that again, should be properly serialized if they are being cached on the cbfs side.
#21692
Posted: 09/25/2012 00:22:47
by Eugene Mayevski (EldoS Corp.)

Thank you for detailed report. While it is always possible that memory corruption takes place, nobody else complained about the issue (and we have hundreds of CBFS users already), so it's specific to either your implementation or to Delphi (maybe even to particular Delphi version). It's necessary that you manage to reproduce the issue with some of samples (probably by modifying them) or by reducing your code to the test case which we can run. Without it we can't say much.

Regarding enumeration - it's a complex subject where it is easy to make a mistake. DirectoryInfo in CBFS 3 is a file context - it is allocated in a call to OpenFile and closed when the directory is closed. Now, there can be several enumerations performed at the same time, and your code must care about this.

It's possible (I need to check this with developers) that OnEnumerateDirectory is called from different threads for the same enumeration operation. In general, this must not be so, but we need to double-check.

Finally, all events have been reworked in CBFS 4 (as we added another set of contexts) and we recommend moving to CBFS 4 in evaluation.


Sincerely yours
Eugene Mayevski
#21702
Posted: 09/25/2012 09:42:34
by Volodymyr Zinin (EldoS Corp.)

Quote
Eugene Mayevski wrote:
It's possible (I need to check this with developers) that OnEnumerateDirectory is called from different threads for the same enumeration operation. In general, this must not be so, but we need to double-check.

Enumeration for the same directory is always sequential. I.e. the next OnEnumerateDirectory call is performed only when the previous one has been finished. But the thread which calls OnEnumerateDirectory can be different. I.e. any free worker thread is chosen to call the callback.
In the case the SerializeCallbacks property is set to true only one worker thread is created and used to call all the callbacks. In another case the quantity of worker threads is specified by the ThreadPoolSize property.
Also there can be several enumerations for the same directory at the same time (sure the OnEnumerateDirectory callbacks are still called sequentially). For an example - different programs perform the same directory enumeration. In this case EnumerationContext is different for different enumerations.

About a possible bug in our Delphi code - we have rechecked it but didn't find anything suspicious. It would be the best if you could reproduce the problem with one of CallbackFS samples (sure modified as it's required) and gave us that sources.
Thanks.
Also by EldoS: CallbackProcess
A component to control process creation and termination in Windows and .NET applications.

Reply

Statistics

Topic viewed 10495 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!