EldoS | Feel safer!

Software components for data protection, secure storage and transfer

TElMessagePart does not Clone DataStream

Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.
#22464
Posted: 11/13/2012 07:22:56
by Niklas Kjellander (Standard support level)
Joined: 09/30/2008
Posts: 49

Hi,

It seems like the Clone method of TElMessagePart does not clone the DataStream property, why is that?

Regards,

Niklas Kjellander
#22465
Posted: 11/13/2012 07:36:17
by Vsevolod Ievgiienko (EldoS Corp.)

Thank you for reporting the problem.

We'll fix TElMessagePart.Clone method.
#22466
Posted: 11/13/2012 07:44:45
by Niklas Kjellander (Standard support level)
Joined: 09/30/2008
Posts: 49

Thanks!

I got another question/issue for you. When you use SetData of TElMessagePart and supply a stream, the size of the stream is limited to Int32.MaxValue. How do you assign data to the message part that is larger than this?
#22467
Posted: 11/13/2012 07:51:10
by Eugene Mayevski (EldoS Corp.)

I am not sure whether cloning would be correct there. IIRC DataStream can reference potentially huge stream and it can be not desirable to duplicate that stream. Probably that was a reason not to make a copy.


Sincerely yours
Eugene Mayevski
#22468
Posted: 11/13/2012 08:01:49
by Vsevolod Ievgiienko (EldoS Corp.)

For now data length is limited to Int32.MaxValue. We'll need to review our code to remove this limitation. You can add a request to our wish-list: https://www.eldos.com/sbb/wishlist.php
#22469
Posted: 11/13/2012 08:07:33
by Eugene Mayevski (EldoS Corp.)

MIME is not suitable for operations with huge files. All in all MIME is about mail, and mail has limits on message size (on most servers the limit is 3 to 25 Mb). MIMEBlackbox was not designed with 64-bit data sizes in mind and there's no reason to provide such support.

When I mentioned huge streams above, I was thinking about "huge" sizes for mobile platforms, which have severe constraints on per-process memory available. For example, in Android the process could use only 16 Mb of memory (later this limit was increased to 32Mb and I don't know current state). On Windows CE (which we still support via .NET CF) there exist similar limitations.


Sincerely yours
Eugene Mayevski
#22470
Posted: 11/13/2012 09:00:43
by Niklas Kjellander (Standard support level)
Joined: 09/30/2008
Posts: 49

I also noticed something else here. When saving a mail message to file I got a null reference exception when a message parts DataStream was a FileStrean (file size 170 mb).
Stack trace
Quote
at SBMIME.TElMessagePart.internalAssemble(Stream Destination, Byte[] Charset, TElHeaderEncoding HeaderEncoding, Byte[] BodyEncoding, Byte[] AttachEncoding, String AText, Boolean bDataIsText, Boolean bAssembleSource)
at SBMIME.TElMessagePart.Assemble(Stream Destination, Byte[] Charset, TElHeaderEncoding HeaderEncoding, Byte[] BodyEncoding, Byte[] AttachEncoding, Boolean bAssembleSource)
at SBMIME.TElMultiPartList.Assemble(Stream Destination, Byte[] Charset, TElHeaderEncoding HeaderEncoding, Byte[] BodyEncoding, Byte[] AttachEncoding, Boolean bAssembleSource)
at SBMIME.TElMessage.AssembleMessage(Stream Destination, Byte[] Charset, TElHeaderEncoding HeaderEncoding, Byte[] BodyEncoding, Byte[] AttachEncoding, Boolean bAssembleSource)
at SBMIME.TElMessage.AssembleMessage(Stream Destination, String Charset, TElHeaderEncoding HeaderEncoding, String BodyEncoding, String AttachEncoding, Boolean bAssembleSource)
at SBMIME.TElMessage.AssembleMessage(Stream Destination, Boolean bAssembleSource)


Is there any known limit of how large files that can be attached?

/Niklas
#22471
Posted: 11/13/2012 09:03:31
by Eugene Mayevski (EldoS Corp.)

There are no artificial limits there but I think that maybe you've come across low memory problem - memory was not allocated but the inner exception has been suppressed. Is this reproducible on some small test case?

On a side note, can you describe what you are doing (in regards to MIME)? Knowing this would help us understand better your needs and maybe find some alternative ways to doing some things that don't work as expected.


Sincerely yours
Eugene Mayevski
#22472
Posted: 11/14/2012 01:38:42
by Niklas Kjellander (Standard support level)
Joined: 09/30/2008
Posts: 49

The scenario is quite simple. I want to add a file as an attachment to a mail message and serialize the message to disk. Maybe this was not really what you wanted me to describe (I'm not sure what you mean by "in regards to MIME")?

Anyway this is reproducable with the code below. The size of the file is 177 012 736 bytes and when I run the test application I have about 650 mb of free physical memory.

Code
TElMessage message = new TElMessage(new TElMultiPartList(), "MyTestApplication");
TElMessagePart part = new TElMessagePart(message, null);
FileStream data = File.Open(@"d:\temp\large.file", FileMode.Open, FileAccess.Read, FileShare.Read);
part.DataStream = data;
part.DataSize = (int)data.Length;
part.SetContentType("application", true);
part.SetContentSubtype("octet-stream", false);
part.SetContentDisposition("attachment", true);

message.MainPart.AddPart(part, false, -1);

using (FileStream target = File.Open(@"d:\temp\mail.eml", FileMode.Create, FileAccess.Write, FileShare.None))
{
   message.AssembleMessage(target, false);
}
#22473
Posted: 11/14/2012 01:43:02
by Niklas Kjellander (Standard support level)
Joined: 09/30/2008
Posts: 49

By the way, are you going to clone the DataStream property or not when cloning a message part? I feel that this thread kind of lost it focus on the initial topic.
Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.

Reply

Statistics

Topic viewed 1470 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!