EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Multiple Reads and Writes on Same File

Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.
#35109
Posted: 11/25/2015 23:18:06
by Devender Reddy (Basic support level)
Joined: 11/25/2015
Posts: 12

Hi,

I have a requirement, where my application has 'N' number of threads and all/some these threads may end up writing/reading the same file from the same bucket. So to accomplish this task, I did the following things:

- Created a single SolFSStorage object in write mode, so that all the threads will use this same SolFSStorage object for either reading/writing the files in that bucket.

Code
   SolFSStorage* m_solFsHandle = new SolFSStorage((TCHAR*)bucketPath.c_str(),
                                               false, false, '\\', false);
   if (NULL != m_solFsHandle) {
      return solFsHandle;
   }


- On each read/write request, each thread will open their own dataStream object and try to read/write the same/different file from the same bucket. This particular operation happens in Parallel and based on the read/write the isWrite flag is set to false/true.

Code
      try {
         dataStream = new SolFSStream(m_solFsHandle, (TCHAR *)fileName.c_str(),
                                     isWrite, !isWrite, isWrite, false,
                               false, (TCHAR *)eldosKey.c_str(),
                                     SOLIDFS_ENCRYPTION, 0,
                                     SOLIDFS_COMPRESSION,
                                     SOLIDFS_COMPRESSION_LEVEL,
                                     PAGES_PER_CLUSTER);
         return dataStream;
      } catch (ESolFSError *err) {
         errCode = err->ErrorCode();
         errString = ErrorStrings[-errCode - 1];

         if (errSharingViolation != errCode) {
            break;
         }

         sleep <<= 1;
         Sleep(sleep * 1000);
      }


With this operation, I am still getting sometimes errSharingViolation/errStorageReadOnly error codes. So can you please let me know, what went wrong here.

Thanks in advance for taking your time and looking into this issue.

NOTE: On the nit pick side, I noticed that the operation spelling in the error codes is wrong.

TEXT("Can not perform this opeartion on active storage"),
TEXT("Can not perform this opeartion on not active storage"),
#35110
Posted: 11/26/2015 03:03:22
by Alexander Plas (EldoS Corp.)

Hello

The errStorageReadOnly error means that the storage was opened in read only mode. SolFS first attempts to open the storage in read/write mode and if it is impossible opens it in read only mode. Check please appropriate property after storage opening in order to avoid this error.

The errSharingViolatio error may occurs when you try to recreate already existing and opened file. Please try not pass true as CreateNew parameter in stream constructor.
#35111
Posted: 11/26/2015 03:30:44
by Eugene Mayevski (EldoS Corp.)

Could you please clarify if all calls are made in threads within a single process? The reason I am asking is that Application Edition of SolFS doesn't let you open the storage in read-write mode from several processes in parallel.

On a side note it would help a lot if you used CODE button located above the text entry box (alternatively you can write [ CODE ] and [ /CODE ] tags by hand) to mark the beginning and the end of the code blocks in your messages. This would enable syntax highlighting and line numbering on the code and make it easier for analysis.


Sincerely yours
Eugene Mayevski
#35114
Posted: 11/26/2015 17:35:41
by Devender Reddy (Basic support level)
Joined: 11/25/2015
Posts: 12

Hi,

Thanks for editing the code block in nice format.

My entire application is running under process and all the threads are running under this same process. Most of the times, my requirement is to support multiple readers and single writer.

Thanks
Dev
#35117
Posted: 11/27/2015 01:40:17
by Eugene Mayevski (EldoS Corp.)

This scenario should work correctly.

errSharingViolation and errStorageReadOnly are different codes returned in different situations.

errSharingViolation means that you try to open the file (within a storage) in read-write mode, while it has been opened with more restrictive mode before.

errStorageReadOnly basically says the same about the whole storage.

The first error could happen due to insufficient guarding of the files, eg. due to not using SingleWriterMultipleReaders pattern. The reason of the second error (errStorageReadOnly) is not clear, though I can't exclude the variant that this error code is used somewhere mistakenly instead of errSharingViolation . This is something we can check by looking through the code.

It would be great if the problem could be reproduced with some runnable test case. Then we'd be able to debug it easily.


Sincerely yours
Eugene Mayevski
#35124
Posted: 11/27/2015 06:40:48
by Alexander Plas (EldoS Corp.)

Hello

According to your code you pass isWrite value into CreateNew parameter in SolFSStream constructor. So, if isWrite == true, SolFS tries to recreate the file. If the file already exists and it is opened by any other thread, an errSharingViolation error occurs. You have to pass false into CreateNew parameter.
#35130
Posted: 11/27/2015 18:31:22
by Devender Reddy (Basic support level)
Joined: 11/25/2015
Posts: 12

As suggested by Alaxender, it seems to be working fine after changing the createNew flag to false and read/write enabled options to true.

Once again thanks for your quick responses.


Code
   try {
      dataStream = new SolFSStream(m_solFsHandle,
                                             (TCHAR *)fileName.c_str(),
                              false, true, true,
                                             false, false,
                    (TCHAR *)eldosKey.c_str(),
                              SOLIDFS_ENCRYPTION, 0,
                              SOLIDFS_COMPRESSION,
                              SOLIDFS_COMPRESSION_LEVEL,
                              PAGES_PER_CLUSTER);
   } catch (ESolFSError *err) {
      std::wstring errStr = ErrorStrings[-(err->ErrorCode()) - 1];
      LOG_ERROR(L"Error while opening dataStream for file %s. Reason: %s(%d)",
           fileName.c_str(), errStr.c_str(),
                          err->ErrorCode());
      CLOSE_STREAM(dataStream);
      return NULL;
   }
Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.

Reply

Statistics

Topic viewed 2515 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!