EldoS | Feel safer!

Software components for data protection, secure storage and transfer

LoadFromStream, SaveToStream performance, alternative cryto libs..libs

Also by EldoS: RawDisk
Access locked and protected files in Windows, read and write disks and partitions and more.
#18507
Posted: 12/19/2011 07:18:03
by Željko Tanović (Standard support level)
Joined: 12/16/2011
Posts: 4

Hi,

We're currently in an evaluation stage of your product (SecureBlackBox for .NET) and have run into a performance problem at completely unexpected place. What we want to do is sign an XML document using XADES. We have implemented a prototype process with the help of using provided examples like this:

- TElXMLDOMDocument.LoadFromStream (stream, "utf-8", true);
- Do signing
- TElXMLDOMDocument.SaveToStream (mstream, SBXMLDefs.Unit.xcmCanon, "utf-8");

Expected volume of documents is large, some documents are dozens of MBs in size. We expected that cryptographic operations would be a bottleneck. However - loading and saving to stream seem to be taking around 40% and 30% of CPU time respectively ( on a large document ), while the complete signing process takes about 15% ( measured using RedGate profiler ).

Could you recommend an alternative solution to Load and Save procedures, or a redesigned process that could take less time to parse the document, or.... ?

I'm attaching profiler call graph image as an example....

Also - on an unrelated note, is there a way to offload the signing process to a user specified CSP or PKCS11 library ?

Thanks in advance,

Željko


#18509
Posted: 12/19/2011 08:07:16
by Eugene Mayevski (EldoS Corp.)

Quote
Željko Tanović wrote:
I'm attaching profiler call graph image as an example....


That is normal. You are probably missing the fact that XML signing involves parsing and transforming the XML document. Cryptographic operations don't take lots of time, unlike parsing. Generation/saving could *probably* be faster if you use memory stream for saving (and then save the memory stream to the disk as a separate operation). Well, the same applies to loading - copy the document to memory stream and load from memory (if you are not doing this yet).

Quote
Željko Tanović wrote:
Also - on an unrelated note, is there a way to offload the signing process to a user specified CSP or PKCS11 library ?


Yes. If you need to use certificates from that CSP or PKCS#11, then you use corresponding TElWinCertStorage or TElPKCS11CertStorage respectively. If you want something more tricky, then you can create your own cryptoprovider class and pass it to the signer.


Sincerely yours
Eugene Mayevski
#18511
Posted: 12/19/2011 08:21:29
by Vsevolod Ievgiienko (EldoS Corp.)

You can also try to implement loading/saving operations using our TElReadCachingStream/TElWriteCachingStream classes declared in SecureBlackbox.Base.SBStreams. The code will look like this:
Code
TElReadCachingStream s = new TElReadCachingStream();
s.Stream = stream;
s.CacheSize = cache_size_in_bytes;
XMLDocument.LoadFromStream(s);
#18512
Posted: 12/19/2011 08:22:15
by Dmytro Bogatskyy (EldoS Corp.)

Quote
Could you recommend an alternative solution to Load and Save procedures, or a redesigned process that could take less time to parse the document, or.... ?

Could you describe what data types mainly used in those xml documents. For example, the xml documents consist of many small elements or they contain big text nodes?
In second case you can try to use delayed loading mode. See LoadFromStream method with DataToLoad parameter:
http://www.eldos.com/documentation/sb...tream.html
Note, in this case you shouldn't close a stream.
Quote
Also - on an unrelated note, is there a way to offload the signing process to a user specified CSP or PKCS11 library ?

To sign using user specified CSP you can create your own provider or simply to use distributed signing methods:
For this you would need to replace GenerateSignature and Save method with GenerateSignatureAsync and InitiateAsyncSign methods. InitiateAsyncSign method will return an async state that contains a hash value that should be signed. How to process async state you can look at PDFBlackbox\ASPNet_Distributed or Azure samples.
Then you would need to call CompleteAsyncSign(XMLDocument, State) to complete signing. You can save an xml document to disk and then load it between InitiateAsyncSign and CompleteAsyncSign calls.
As for PKCS#11, you can load a certificate from PKCS#11 storage and use it for signing, it is supported by a component.
#18513
Posted: 12/19/2011 08:23:50
by Željko Tanović (Standard support level)
Joined: 12/16/2011
Posts: 4

Quote
Eugene Mayevski wrote:
That is normal. You are probably missing the fact that XML signing involves parsing and transforming the XML document. Cryptographic operations don't take lots of time, unlike parsing. Generation/saving could *probably* be faster if you use memory stream for saving (and then save the memory stream to the disk as a separate operation). Well, the same applies to loading - copy the document to memory stream and load from memory (if you are not doing this yet).


Unfortunately I am doing this already :) And nope, I'm not missing the fact that library needs to parse the document... But it's performance in doing it seems to be way below .NET XML engine for some reason. The thing is that before initiating signing process I already have the document parsed in .NET's XMLDomDocument object. To pass it to your library for signing, I'm saving it to memory stream ( what takes very little time, btw.... ), then loading into TElXMLDOMDocument object for processing.... And in the end getting the second memory stream out of it. Doing this 50 times on a 150 - 200 kb docs takes more than 20 secs....

Quote

Yes. If you need to use certificates from that CSP or PKCS#11, then you use corresponding TElWinCertStorage or TElPKCS11CertStorage respectively. If you want something more tricky, then you can create your own cryptoprovider class and pass it to the signer.


Thanks. Do you maybe have a mock code that does that ? I need to delegate calculating digests and signing to a hardware device ( unfortunatelly I'm not sure at this moment what kind of API it provides)
#18514
Posted: 12/19/2011 08:34:20
by Eugene Mayevski (EldoS Corp.)

Quote
Željko Tanović wrote:
Doing this 50 times on a 150 - 200 kb docs takes more than 20 secs....


Then you have to live with this ...

Quote
Željko Tanović wrote:
I need to delegate calculating digests and signing to a hardware device ( unfortunatelly I'm not sure at this moment what kind of API it provides)


The approach with distributed signing mentioned by Dmytro above would be the best in your case.


Sincerely yours
Eugene Mayevski
#18525
Posted: 12/20/2011 03:03:02
by Željko Tanović (Standard support level)
Joined: 12/16/2011
Posts: 4

Hi guys,

Thanks for your suggestions. I'll try them out and see what comes out... To answer Dmytro's question,

Quote
For example, the xml documents consist of many small elements or they contain big text nodes?


It contains a lot of small elements and virtually no attributes at all. In a lot of cases element content is smaller than element name.

Thanks again,

Željko
#18526
Posted: 12/20/2011 06:22:24
by Dmytro Bogatskyy (EldoS Corp.)

Quote
It contains a lot of small elements and virtually no attributes at all. In a lot of cases element content is smaller than element name.

Thank you for the description. I will check for the possibility to improve parsing speed in this situation.
Also by EldoS: MsgConnect
Cross-platform protocol-independent communication framework for building peer-to-peer and client-server applications and middleware components.

Reply

Statistics

Topic viewed 1874 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!