EldoS | Feel safer!

Software components for data protection, secure storage and transfer

.net Large object heap

Also by EldoS: Solid File System
A virtual file system that offers a feature-rich storage for application documents and data with built-in compression and encryption.
#17745
Posted: 10/05/2011 09:36:49
by Pol R (Basic support level)
Joined: 06/22/2011
Posts: 10

Hi,
I used eldos and notice that in some cases the CBFS "try to write" or "request to read" buffers larger than c# large object heap which cause memory fragmentation and other problems.

Are you aware to this issue? How do you handle it?

Thanks,
#17835
Posted: 10/12/2011 08:30:03
by Eugene Mayevski (EldoS Corp.)

Not sure that I understand the question.

The driver attempts to read or write as much data (in one call) as the application wants. It knows nothing about large object heap, neither do we.


Sincerely yours
Eugene Mayevski
#17839
Posted: 10/12/2011 09:22:03
by Eugene Mayevski (EldoS Corp.)

Let me explain some details. The driver maps the memory block which it received from requesting application back to the user space. This is unmanaged memory. .NET event handler must receive a managed block. So the .NET API code allocates a managed buffer of required size and passes it to the event handler. There seems to be no way to turn unmanaged memory block into managed array of bytes. If there is some - please let us know and we'll gladly use it.


Sincerely yours
Eugene Mayevski
#17850
Posted: 10/13/2011 03:58:18
by Ohad B (Basic support level)
Joined: 10/13/2011
Posts: 1

Hi,
Let me try to clarify it.
LOH is a part the garbage collector mechanism in .net

http://msdn.microsoft.com/en-us/magazine/cc534993.aspx
http://stackoverflow.com/questions/686950/large-object-heap-fragmentation

To summarize the issue , when you allocate in .net buffers larger than 85K bytes this memory is allocated in the LOH area which usually cause performance degradation over time and eventually out of memory exception even thou the process is far of reaching it maximum capacity.
The standard way to solve it is to use pre-allocated object pool - this solution fit when you repeatedly send buffers of the same size which I don't think is exactly the case here.
It seems like in your case you should divide the original buffer to small buffers smaller than 85K in your un managed code and than send these smaller buffer one by one to the managed code ( of course while reading you should do the opposite and assemble all the parts you got from the user mode to one buffer)

Thanks,
Ohad
#17851
Posted: 10/13/2011 06:21:27
by Eugene Mayevski (EldoS Corp.)

Thank you for clarification. I will add your suggestion to the wishlist and we will most likely implement it in future. I can't say that it will be fast (i.e. not within a week, probably), but will be done.


Sincerely yours
Eugene Mayevski
Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.

Reply

Statistics

Topic viewed 1495 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!