EldoS | Feel safer!

Software components for data protection, secure storage and transfer

HTTP Streaming

Also by EldoS: Callback File System
Create virtual file systems and disks, expose and manage remote data as if they were files on the local disk.
#11208
Posted: 10/01/2009 09:40:57
by Sven Polewacz (Standard support level)
Joined: 04/29/2009
Posts: 3

The advertisememt promises chunked encoding for HTTP(S). But the interface of TElHTTPSClient doesnt support any method to get access to the request stream. All I can see is the use of multi-part-requests in the context of file transfers, but this is NOT chunked encoding. So, a request or response always contains a ContentLength header. I want to stream some data over HTTP, so content-length-fixed encoding is no option.

How to create "real" chunked encoded messages?
#11209
Posted: 10/01/2009 10:11:17
by Eugene Mayevski (EldoS Corp.)

Chunked encoding for the client means that the client handles responses coming from the server in chunks. This mode is supported only for HTTP 1.1, so if connection happens using HTTP 1.0 or if the server doesn't support chunked encoding, then no chunked encoding is used. Also, if the server supports chunked encoding, it can provide or refrain from providing total length of the data being served.

As for "streaming" of chunked requests - frankly speaking, this is a very special thing which I never saw in use within 10 years. If you explain, what exactly functionality you want to get, it's likely that we will implement it.


Sincerely yours
Eugene Mayevski
#11222
Posted: 10/02/2009 07:07:42
by Sven Polewacz (Standard support level)
Joined: 04/29/2009
Posts: 3

Chunked POST with .NET HttpWebRequest:

HttpWebRequest wr = (HttpWebRequest)HttpWebRequest.CreateDefault(new Uri("http://192.168.80.250:5000/test/hallo"));
wr.SendChunked = true;
wr.AllowWriteStreamBuffering = false;
wr.Method = "POST";
Stream s = wr.GetRequestStream();
... sending data into requeststream
s.Close();
wr.GetResponse();

This produces (Wireshark output):

POST /test/hallo HTTP/1.1
Transfer-Encoding: chunked // chunked instead of ContentLength
Connection: Keep-Alive
Host: 192.168.80.250:5000

HTTP/1.1 100 Continue

3E8 // start chunk 1
S0030000FC
S2180D0000182F0D000066480D0001FA4E0D00029A550D000374
S224...32A0
S224...31A0
S224...0019
S224...0084
S224...0083
S224...00CC
S224...007A
S224...00CE
S224...00ED
S224...3214
S224...3175
S224...0046
3E8 // start chunk 2
S224...009A
S224...0015
S224...0028
...
#11226
Posted: 10/02/2009 08:54:06
by Eugene Mayevski (EldoS Corp.)

Thank you for detailed information. Can you please explain a little, what is the goal of using this approach? Chunked encoding is used by the servers to speed up serving of clients, but has little sense for clients. Also, architecture of the component is synchronous (unlike .NET's WebRequest) and changing it to asynchronous like you describe would require certain amount of work.


Sincerely yours
Eugene Mayevski
#11227
Posted: 10/02/2009 09:08:42
by Sven Polewacz (Standard support level)
Joined: 04/29/2009
Posts: 3

I want to stream "real-time" data to a server. Its a long-running POST with the goal to optimize the bandwith and latency. The problem is: I cant determine the amount of bytes when I start the request. I thought about a high "dummy" value for ContentLength but I worry about HTTP protocol violations at server side when I close the request before CotentLength is reached.

> Also, architecture of the component is synchronous (unlike .NET's WebRequest)
> and changing it to asynchronous like you describe would require certain
> amount of work.

I noticed that. I hoped there is a "secret" way to get control over the underlying stream. Another way to overcome the synchronous nature of the component could be a split of the Post() method. Something like

// prepare header, encoding, etc.
client.Start();
do
{
data = ReadXBytesOfRequestBody();
client.Send(data);
} while (endOfData)
client.Close();

Another idea: A Post() method with a callback parameter. The callback is called to return the data for the internal send loop.
#11228
Posted: 10/02/2009 09:15:22
by Eugene Mayevski (EldoS Corp.)

Thank you again. We've added the task to ToDo list. It will be implemented in SecureBlackbox 8.0 which we plan for December. You will get free upgrade to it when it's released.


Sincerely yours
Eugene Mayevski
Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.

Reply

Statistics

Topic viewed 1576 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!