EldoS | Feel safer!

Software components for data protection, secure storage and transfer

In-Line Compression and Encryption to/from Amazon S3

Also by EldoS: CallbackDisk
Create virtual disks backed by memory or custom location, expose disk images as disks and more.
#15474
Posted: 01/12/2011 08:59:15
by Eric Lenington (Standard support level)
Joined: 12/06/2010
Posts: 37

It is possible to add file compression "in-line" to the S3 storage class, so that I can start with a clear-text, uncompressed file, and compress and encrypt it, resulting in an encrypted, compressed file on S3 (and of course, the opposite when getting the file)? Of course I can do this now as two separate steps (compress first, then send the file to S3 using in-line encryption), but I'd like to do both of these operations at "the same time" on the stream.
#15478
Posted: 01/12/2011 09:52:59
by Ken Ivanov (EldoS Corp.)

Thank you for contacting us.

I think we will be able to add compression functionality to one of the first SBB 9 builds (within 4-6 weeks, I suppose).
#15480
Posted: 01/12/2011 11:45:25
by Eric Lenington (Standard support level)
Joined: 12/06/2010
Posts: 37

That's excellent news! Can you give me a "preview" of what you're planning (i.e. what kind of compression, will it be in-line on the stream, etc.)?
#15481
Posted: 01/12/2011 12:07:08
by Ken Ivanov (EldoS Corp.)

Yes, compression and decompression will work transparently "on-the-fly". I.e. you would not need to introduce any changes to your existing code (maybe with the only exception of property value adjusting).

I also need to warn you that it will not be possible to use block read operations on compressed objects due to nature of compression.
#15482
Posted: 01/12/2011 12:47:41
by Eric Lenington (Standard support level)
Joined: 12/06/2010
Posts: 37

Understood on the block-read; that makes perfect sense. No matter for us--we treat all S3 objects as opaque, and always read/write the entire object at once.
#15483
Posted: 01/12/2011 12:52:30
by Eric Lenington (Standard support level)
Joined: 12/06/2010
Posts: 37

PS - I assume that a selection of compression algorithms (each with appropriate "compression factor" settings) will be available (including an appropriate HTTP header identifying the algorithm used)? Among others, it would make sense to support GZIP (with the standard GZIP header), so that browsers and other non-Eldos clients (assuming no encryption, of course) could GET the objects automatically in that format.
#15484
Posted: 01/12/2011 15:06:11
by Ken Ivanov (EldoS Corp.)

Unfortunately, we cannot guarantee instant support for GZIP, as the algorithm itself is in development at the moment. Zlib and Bzip2 will be supported though. As soon as GZIP is ready, we will extend CloudBlackbox with it.
#15944
Posted: 02/28/2011 20:21:46
by Eric Lenington (Standard support level)
Joined: 12/06/2010
Posts: 37

Just touching base on this topic...are you getting close to releasing this feature?
#15947
Posted: 02/28/2011 20:42:44
by Ken Ivanov (EldoS Corp.)

The code is in general ready and is passing various QA tests at the moment. Sorry for being a little late with this.
#16511
Posted: 05/25/2011 18:28:40
by Eric Lenington (Standard support level)
Joined: 12/06/2010
Posts: 37

Hello--
How's the testing coming on this feature?
--Eric
Also by EldoS: CallbackDisk
Create virtual disks backed by memory or custom location, expose disk images as disks and more.

Reply

Statistics

Topic viewed 2105 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!