Speed measurement (so-called "benchmarking") is among the first actions that each developer performs when evaluating any communication library. However incomplete, inadequate or inaccurate benchmarking can lead to selection of inappropriate library. What steps should be done and what questions should be answered to ensure that the measurement is correct and affects real situation?
Before defining the strategy of benchmarking one needs to understand the basics of how SSL works. SSL (Secure Socket Layer) is a layer on top of TCP in OSI stack of network layers. It is generally possible to use SSL with any protocol that keeps packet order and does reliable delivery, but most implementations are socket-based. The application or internet protocol developer can create his own data exchange and put it on top of SSL, thus getting data security and authentication transparently. In most cases it is possible to "insert" SSL between application-level protocol and TCP.
SSL-secured communication consists of 3 steps:
- SSL handshake. During this step parties exchange security information (certificates, session keys) and define whether actual data exchange can take place. When security information is being exchanged, keys for asymmetric encryption algorithm are generated. Generation of keys is a slow operation.
- Encrypted data exchange. During this step parties encrypt the application-level protocol data into secured packets and send these packets to each other. Encryption is done using symmetric encryption algorithms, which are fast.
- "Goodbye" step (optional). Most applications ignore this step and just close internet connection. However this step can be used in future to perform "session resume" operation which saves time on consequent connects (due to reuse of pre-generated asymmetric keys).
SSL can use several symmetric and asymmetric algorithms. Some algorithms are faster than other. SSL always uses one symmetric encryption algorithm, one asymmetric encryption algorithm and one hashing algorithm. The set of algorithms used is called "cipher suite". The list of cipher suites to use is specified by the application. The SSL implementation then tries to select the right cipher suite from the list using the information received from other party. Certain cipher suites can be used only when certificate is specified for the party, other cipher suites are used for ad-hoc security (using Diffie-Hellman algorithm to exchange the session key).
There exist several versions of SSL - SSL2, SSL3 and TLS1. They can use different cipher suites. For example, SSL2 can't use anonymous cipher suites - server side of the communication must have a certificate.
Now when you are acquainted with SSL (at least a little :), we can come closer to the actual topic of this article.
- First thing to remember is "don't compare apples to oranges". If you measure speed of different libraries, always specify the same version of SSL and the same cipher suites for all libraries. When selecting the cipher suites, take into account the design of your system, i.e. whether client and/or server certificates are used. Remember, that SSL2 is outdated and less secure version that should be used only for fallback modes, i.e. when the unknown party doesn't support better versions.
- Next you need to define, what kind of data your system is going to transfer. If your system is supposed to transfer small data blocks (for example up to 8Kb), speed of handshake is more important than overall speed. To measure speed of handshake you just need to establish SSL connection and forcefully close connection right after SSL connection is established. Don't transfer any data at all. Remember to check all cipher suites that can be used in your system - different implementations of SSL have different speed on different cipher suites. On the other hand, if your system is about to transfer larger data blocks or even huge files, then speed of handshake shouldn't be considered a factor which should be taken into account.
- If you perform certificate validation in code, turn it off because your code can significantly influence the results.
- Do not perform one-time measurements. Each measurement should be taken several times (the more the better) and average should be calculated. Add random intervals between measurement cycles. Some SSL implementations might cache some information and random intervals will show "clear performance", without caching.
- Check if the library supports goodbye step and session resuming. Resuming of sessions can save time on handshake phase. This is especially important for multi-tier applications without persistent connections (i.e. when for each request a separate connection is established).
- Check what platform the library is written for and how it is executed: implementations written in .NET managed code or Java byte-code will be slower than native-code implementations.
After you've completed your tests, throw them away. Seriously. All native SSL implementations have similar speed which can vary in particular scenarios, but in general libraries are very close in speed. Security and ease of use (which leads to better security) are also very important things. And if your ultimate goal is to save a fraction of second per connection, you will lose time and money patching your code, improving security and avoiding bugs of fast and insecure implementations. Remember, that SSL is Secure Socket Layer, and not Speedy Socket Layer.