EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Decrypt to Stream without loading the whole fil

Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.
#28582
Posted: 02/26/2014 11:48:58
by Frank Salinas (Basic support level)
Joined: 02/26/2014
Posts: 2

I'm using SecureBlackbox 10.0.233 and calling it from C# and .Net 4.0 in Windows. I'm trying to figure out how to decrypt a PGP encrypted zip file to a stream so that I can pass the decrypted stream to a class that can unzip the contents directly from it. I tried using the DecryptAndVerify method in TElPGPReader passing it an output stream through its OutputStream property. For the count parameter to DecryptAndVerify I cannot send any other value but zero or I get an "Unexpected End Of Data" error. From what I read in the documentation passing zero in the Count parameter loads the whole file into memory and decrypts it. After the call to DecryptAndVerify I have to then rewind the stream and pass it to my unzipper class to get the process to work. However, I'm worried about the memory usage because I will be decrypting very large files in multiple simultaneous threads. Is it possible to decrypt the file to a stream without having to load the whole thing into memory before passing the output stream to my unzipper? Here is a snippet showing what I'm doing right now which does work but using a lot of memory in the process.

Code
TElPGPReader pgpreader = new TElPGPReader();
pgpreader.KeyPassphrase = _config.PGPDecryptSites.Passphrase;
pgpreader.DecryptingKeys = _keyring;

pgpreader.OutputStream = new MemoryStream();
inputFileStream = new FileStream(fileToDecrypt, FileMode.Open);
pgpreader.DecryptAndVerify(inputFileStream, 0);
pgpreader.OutputStream.Position = 0;

_unzipper.UnzipFromStream(pgpreader.OutputStream);


Thanks.
#28583
Posted: 02/26/2014 12:20:17
by Eugene Mayevski (EldoS Corp.)

Thank you for detailed description of your task.

Streams in .NET are not "pipes", so you can't easily send the data to the consumer without decrypting the complete data first. If you have the data larger than possible amount of available memory, your best option is to decrypt to temporary file stream and then process it.


Sincerely yours
Eugene Mayevski
#28584
Posted: 02/26/2014 14:15:46
by Frank Salinas (Basic support level)
Joined: 02/26/2014
Posts: 2

Is this something specific to PGP decryption that prevents the streams from behaving like Pipes? I've done this with other streams, for example getting a stream from a web request and unzipping the response on the fly without waiting to get the the whole response from the web server first. I'm just trying to understand in more detail what the problem is. On a somewhat unrelated note, how do you use the Count parameter of DecryptAndVerify. Like I said in my original post anything other than zero or the full length of the file has always produced an error for me.

Thanks.
#28589
Posted: 02/26/2014 16:27:20
by Ken Ivanov (EldoS Corp.)

Hello Frank,

TElPGPReader always processes streams as a pipe, without loading the whole file to memory. The component reads protected data from the input stream chunk-by-chunk and writes chunks of unprotected data to the output stream. This way, if your ZIP processor supports chunk-by-chunk processing, you could implement your own descendant of the System.IO.Stream class (that will be passing data written to it up to your ZIP component), and assign an object of that stream class to TElPGPReader's OutputStream property, effectively making a pipeline:

Code
void Write(byte[] Buf, int StartIndex, int Count)
{
  _unzipper.WriteChunk(Buf, StartIndex, Count);
}

Reply

Statistics

Topic viewed 1306 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!