EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Specify server side encryption for S3 upload?

Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.
#29648
Posted: 05/31/2014 16:21:22
by Brennon Thompson (Basic support level)
Joined: 05/28/2014
Posts: 11

I'm currently evaluating the trial version of SB to build a test app that will upload an object to an S3 bucket. My requirements specify that Amazon shall apply server side encryption to the uploaded object. According to the S3 documentation, that is accomplished by adding the following to the request header:

"x-amz-server-side-encryption: AES256"

I've gotten this working when I build the entire HTTP request by hand (i.e., not using SBB) but I'm not sure how to accomplish it using SBB. I added an event handler for TElHTTPSClient.OnPreparedHeaders() and then added the info above to the headers with:

Headers.Add("x-amz-server-side-encryption: AES256");

When I do this, I get back the following error from my call to m_DataStorage.WriteObject():

"Object write error: Connection lost (error code is 10053)"

If I remove this information from the headers, the upload works successfully (but then the server encryption isn't being applied either). I assume that I'm doing something wrong here but I'm not sure what.
#29653
Posted: 06/02/2014 05:14:52
by Ken Ivanov (EldoS Corp.)

Hi Brennon,

'x-amz-' headers are treated by Amazon web services a bit differently to standard HTTP headers (the former should be covered by the digital signature that accompanies the request), so you can't add these in 'last minute' way via the OnPreparedHeaders event. Instead, please add the header to the TElHTTPSClient.RequestParameters.CustomHeaders list before calling WriteObject(), so that it was included in the signature calculation routine.

Remember to clean-up the contents of the CustomHeaders list after making every WriteObject() call, as the list is not cleared automatically.

Ken
#29655
Posted: 06/02/2014 09:25:00
by Brennon Thompson (Basic support level)
Joined: 05/28/2014
Posts: 11

Thanks for the reply Ken. I'm probably missing something obvious here but it's not clear to me how to add something to TElHTTPSClient.RequestParameters.CustomHeaders? Is there a method to do this?
#29657
Posted: 06/02/2014 09:32:55
by Eugene Mayevski (EldoS Corp.)

What programming language are you using?


Sincerely yours
Eugene Mayevski
#29660
Posted: 06/02/2014 10:30:52
by Brennon Thompson (Basic support level)
Joined: 05/28/2014
Posts: 11

C#
#29661
Posted: 06/02/2014 10:38:31
by Eugene Mayevski (EldoS Corp.)

It should be something like
Code
TElHTTPSClient.RequestParameters.CustomHeaders.Add("Name: value")


Sincerely yours
Eugene Mayevski
#29662
Posted: 06/02/2014 10:52:51
by Brennon Thompson (Basic support level)
Joined: 05/28/2014
Posts: 11

:-( I was missing a reference to SecureBlackbox.HTTPCommon. Thanks.
#29664
Posted: 06/02/2014 20:22:29
by Brennon Thompson (Basic support level)
Joined: 05/28/2014
Posts: 11

So I've been able to add x-amz-server-side-encryption to my S3 PUT request and the upload is working properly however, when I view the object in the S3 management console, the server side encryption is not enabled. According to the S3 documentation, when you add x-amz-server-side-encryption to the request, the response headers will also include x-amz-server-side-encryption and I don't see that happening here. If I build the HTTPWebRequest and HTTPWebResponse by hand, it works as expected. Is there something else that I need to be doing to make this work with SecureBlackbox?

Here's the relevant code:

Code
private TElHTTPSClient m_Client = new TElHTTPSClient();
m_Client.RequestParameters.CustomHeaders.Add("x-amz-server-side-encryption", "AES256");

private TElAWSS3DataStorage m_DataStorage = new TElAWSS3DataStorage();
m_DataStorage.HTTPClient = m_Client;
m_DataStorage.WriteObject(bucketName, keyName, f, null);

Debug.WriteLine("m_Client.ResponseHeaders.Count = " + m_Client.ResponseHeaders.Count);
foreach (string header in m_Client.ResponseHeaders)
{
   Debug.WriteLine(header);
}


The response headers I see are:

HTTP/1.1 200 OK
x-amz-id-2: B8r6dcAQ3giruO35eacIIvrV3CAZXt7+Sy6rxP338rwyZ34ti9Tt0oruxIbRUkSx
x-amz-request-id: 2A0D25AD1D2659F1
Date: Tue, 03 Jun 2014 01:22:20 GMT
ETag: "40e3695c90312e071c268780a81eb2fd"
Content-Length: 0
Connection: close
Server: AmazonS3
#29672
Posted: 06/03/2014 09:05:37
by Ken Ivanov (EldoS Corp.)

Hi Brennon,

That was my fault, sorry. You should actually provide the custom header not via the TElHTTPSClient.CustomHeaders property, but via the WriteObjects()'s Headers parameter. That is, you should create an instance of SecureBlackbox TElStringList class before uploading the object, add the above header line to it, and then pass it to the WriteObject() method.

Sorry for misleading you.

Ken
#29678
Posted: 06/03/2014 18:54:43
by Brennon Thompson (Basic support level)
Joined: 05/28/2014
Posts: 11

Thanks for the information Ken. I was using this overload for WriteObject:

Code
public void WriteObject(string BucketName, string Key, Stream Data, TElCustomDataStorageSecurityHandler Handler);


In order to pass in headers, it looks like I'll need to use:

Code
public void WriteObject(string BucketName, string Key, Stream Data, string ContentType, string ContentDisposition, TElStringList Metadata, TElStringList Headers, TElCustomDataStorageSecurityHandler Handler);


In addition to headers, this overload obviously has several additional parameters like ContentType, ContentDisposition and Metadata that I don't currently pass in and I'm not sure what the correct values should be for them. If I pass in String.Empty or null for these parameters, will they default to something internally in WriteObject()? If not, can you tell me how I determine what values I should be passing in?
Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.

Reply

Statistics

Topic viewed 2321 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!