EldoS | Feel safer!

Software components for data protection, secure storage and transfer

WRite to stream / flush

Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.
#12499
Posted: 02/19/2010 11:54:48
by Nicolas Bourgogne (Standard support level)
Joined: 02/18/2010
Posts: 5

Hi All,

I have started playing around with SOlFS (Standard Edition) to replace my existing Structured Storage-based implementation.

I seem to have everything working fine, but I want to make sure I got the mechanics right.

In the help, the "Flush" method is listed, and it has also come up in the forum today. I can't quite figure out when I should call this function. So far, I have had no problem with simply writing to the various streams, and closing my storage when I am done with it.
I have not tested for interruption/crashes/failures yet mind you.

Could someone point me to the proper way of writing data to streams and disk, including the use or non-use of transactions? I was not able to find details on that on the website or the help.

Coming from Structured Storage, the data is held in streams in memory until the stream/storage is committed to disk, hence my confusion.

Thanks,


Nicolas
#12500
Posted: 02/19/2010 12:14:27
by Eugene Mayevski (EldoS Corp.)

Quote
Nicolas Bourgogne wrote:
Coming from Structured Storage, the data is held in streams in memory until the stream/storage is committed to disk, hence my confusion.


Nearly the same happens with SolFS. So you need to call Flush to flush all buffers to the disk for data safety.

Transactions (in fact they are closer to journaling concept rather than to transactions) can be used to further improve integrity of storage logical structure. They don't guard you from media error (i.e. if the backend storage is physically or logically damaged, only backup will save the data), but guards against corruption of SolFS structures in the storage.


Sincerely yours
Eugene Mayevski
#12501
Posted: 02/19/2010 12:31:56
by Nicolas Bourgogne (Standard support level)
Joined: 02/18/2010
Posts: 5

Thanks for the quick response Eugene.

So, if I understand correctly, I should call Flush, for safety, before I free the SolFSStream, or after a write to a SolFSStream that I wnat to ensure gets committed to disk. Is that correct?

What happens if I don't call Flush (as is the case now)? I assume there is an automated, deferred call - seeing as my data so far always gets saved to disk. Is there a way to suspend the flushing of SolFSStreams to disk until I explicitly call it, or do I need to open the storage in callback mode to achieve this? If callback mode is required, are there any samples available somewhere?

Sorry if these are basic questions, I just want to make sure I understand what I'm doing and what my options are :o)

Thanks!


Nicolas
#12502
Posted: 02/19/2010 12:52:59
by Eugene Mayevski (EldoS Corp.)

Quote
Nicolas Bourgogne wrote:
So, if I understand correctly, I should call Flush, for safety, before I free the SolFSStream, or after a write to a SolFSStream that I wnat to ensure gets committed to disk. Is that correct?


This won't hurt, yes.
Quote
Nicolas Bourgogne wrote:


What happens if I don't call Flush (as is the case now)?


If the storage is closed correctly, then the data will be flushed on close. If the application dies or computer is turned off, then the data in the cache is lost.

Quote
Nicolas Bourgogne wrote:
Is there a way to suspend the flushing of SolFSStreams to disk until I explicitly call it


And what the reason for this would be?


Sincerely yours
Eugene Mayevski
#12503
Posted: 02/19/2010 12:57:07
by Eugene Mayevski (EldoS Corp.)

The developer reminded me, that Flush is also performed when the stream is closed. If you want to defer all flushes, you would need to use callback mode, yes. It's quite trivial to implement callback mode. Just read description of OnFile* events in the help file.


Sincerely yours
Eugene Mayevski
#12504
Posted: 02/19/2010 13:48:14
by Nicolas Bourgogne (Standard support level)
Joined: 02/18/2010
Posts: 5

Quote
Eugene Mayevski wrote:

Quote
Nicolas Bourgogne wrote:
Is there a way to suspend the flushing of SolFSStreams to disk until I explicitly call it


And what the reason for this would be?


This would be for operations that are part of a larger one, when I want all the changes to be flushed, or non at all. But this should be covered by the "Flush on close".

Is there a command to flush all the open streams, or does Flush need to be called implicitly by each stream?

Quote
Eugene Mayevski wrote:
The developer reminded me, that Flush is also performed when the stream is closed. If you want to defer all flushes, you would need to use callback mode, yes. It's quite trivial to implement callback mode. Just read description of OnFile* events in the help file.


I think I will experiment with CallBack mode over the weekend, and get a better grasp of it.

Thanks again!


Nicolas
#12507
Posted: 02/20/2010 02:11:40
by Eugene Mayevski (EldoS Corp.)

Quote
Nicolas Bourgogne wrote:
This would be for operations that are part of a larger one, when I want all the changes to be flushed, or non at all. But this should be covered by the "Flush on close".


As flush is done for each stream, when it's closed, at the moment the only solution is to use callback mode and do all writes, initiated after transaction begins, to some temporary place. Then, when you are done writing all streams, call Flush on the storage (this will write more data to your temporary place) and then save the data from the temporary place to the real storage.

We will consider implementing this scheme in next version of SolFS.


Sincerely yours
Eugene Mayevski
Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.

Reply

Statistics

Topic viewed 3076 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!