Evaluating the Performance of SPDY-enabled Web Servers

As you may already know, SPDY (pronounced “SPeeDY”) is a new networking protocol introduced by Google and designed to reduce the web latency associated with HTTP. With SPDY, web pages load up to 64 percent faster than HTTP alone, according to Google. It accomplishes this by adding a session layer between HTTP and SSL that supports concurrent, interleaved streams over a single TCP connection. The initial draft of HTTP/2.0, the future of the web, is based on SPDY, which is a welcome step forward considering that HTTP/1.0 was released in 1996. SPDY holds great potential for mobile devices, for which latency is more of an issue and the market is catching on. As evidence of this, Microsoft recently announced the next version of Internet Explorer will support the new protocol.

Today, SPDY can be deployed on Apache by installing a single module (mod-spdy, the Apache SPDY module). No changes to website content are required. SPDY is now enabled by default on many of the latest modern browsers, including Chrome, Firefox, Opera, and Android Browser. (You can check your own browser’s support by visiting isspdyenabled.com).

So, simply by deploying the SPDY module on your Apache server, you can provide a large percentage of your users with better response times! More importantly, the response times of a website are correlated with the site’s conversion rate and bounce rate—so SPDY can have a significant positive impact on your business!

With all the advantages of SPDY, you may be wondering if there is a price to pay on the server side. Moving from HTTP to SPDY adds encryption and compression overhead, which clearly requires more resources. It does, however, use fewer TCP connections. It wasn’t immediately clear to me if all these changes in aggregate would cause noticeable drawbacks on the server.

To get to the bottom of this question, I thought it would be interesting to get some real-world performance metrics on SPDY, in particular its effect on the server side. To obtain this information, I used the new SPDY support in NeoLoad to run load tests of SPDY and compare its performance against HTTP and HTTPS.

Test Configuration Details

My approach was to test a server with a fairly typical configuration—not one that is finely tuned for a specific purpose. This setup provides a basis for comparing HTTP, HTTPS, and SPDY. It’s important to note here that changes to the application or the server configuration will produce different results—perhaps vastly different.

Test setup:

Results and Analysis

First I wanted to examine the hit rate (hits per second), average response time (duration), and number of errors as the number of concurrent users increased.

Here are the results for the HTTP test:

spdy-http

The plot illustrates the following:

The table below compares these metrics for HTTP, HTTPS, and SPDY:

HTTP HTTPS SPDY
Maximum pages/s 16.3 pages/safter 3 minutes at 120 users 15.9 pages/safter 3 minutes at 120 users 98 pages/safter 14 minutes at 777 users
Page response time at 100 users 1.1s 1.3s 1.1s
Page response time at 120 users 1.4 s 1.5s 1.1s
Page response time at 200 users 7.1s 7.8s 1.1s
Page response time at 777 users 70.2s 72s 2.7s
First error 405 Users 225 Users 884 Users
Service-level agreement violation (< 3s page load time) 133 Users 133 Users 794 Users

Based on a service-level agreement (SLA) of less than 3s for page load time, my test server handled almost six times as many users with SPDY than HTTP!

More Users With Fewer Workers

To handle incoming requests in parallel, Apache web servers use worker threads to process each request. Because each thread consumes resources and memory, Apache lets the system administrator limit thread use, which has an effect on how incoming requests are handled.

My HTTP and HTTPS results are typical of a scenario known as thread starvation.

This is confirmed by the following graph, which shows busy workers in orange, page rate in green, and user load in blue:

spdy-worker

The number of busy Apache workers (threads) hits its maximum at the same time the page rate reaches its maximum.

In contrast to HTTP/S, SPDY uses a single connection for all requests. The principal benefit of this is the decreased latency seen by the client, resulting in reduced response times for users. There is another great benefit on the server side: SPDY clients consume one worker instead of six. As a result, it takes much longer (that is, many more users) for all workers to become busy:

spdy-http-workers

SPDY enables the server to handle more users with the same number of workers.

Server CPU and Memory Consumption

So far, I showed how SPDY affects Apache worker usage with my test configuration. Next, I wanted to see how CPU usage and memory were affected by SPDY.

Here is a plot showing system idle time (low when the CPU is used intensively) for HTTP/HTTPS/SPDY:

spdy-linux-cpu-idle

No surprise here. As most HTTP/HTTPS requests are waiting to be served after the thread limit is reached, they don’t consume much CPU. For the SPDY tests, Apache is below the worker limit for much longer, resulting in a higher page rate and consequently more CPU used. The curve plateau is reached when all workers are busy (around 11 minutes).

To better understand the behavior before the worker limit is reached, I focused on the numbers at 2 minutes, 88 virtual users, and 12.5 pages/s. The CPU idle time measurements are an average of six values taken over 30 seconds:

CPU HTTP HTTPS SPDY
CPU Idle time at 2 minutes 93.33% 91.5% 92%

The values are comparable. SPDY consumes slightly less CPU than HTTPS and slightly more than HTTP for this load.

On the memory side, here’s the global overview:

spdy-memory

Again, it’s interesting to examine the memory used before the worker limit is reached. The numbers are an average of six measurements over 30 seconds.

System Memory 01:00 02:00 Difference = Consumed Memory
HTTP 3,357 MB 3,416 MB 59 MB
HTTPS 3,500 MB 3,579 MB 79 MB
SPDY 3,607 MB 3,631 MB 24 MB

The server consumed less memory when handling SPDY requests than when handling HTTP and HTTPS.

During this test, SPDY consumed just 41% of the memory consumed by HTTP, and just 30% of the memory consumed by HTTPS.

Conclusion

It’s no surprise that SPDY improves response times on the client side. That’s what it was designed to do. It turns out that SPDY also has advantages on the server side:

From these results, it is clear that Apache tuning performed for HTTP/S may not be appropriate after SPDY is enabled on the server. It’s important to take the time to re-evaluate server sizing and retune the server before you start handling SPDY requests from your users.

As I mentioned earlier, the test results I’ve shared here are specific to my server configuration and the web page I used for the tests. It’s important to understand how your server will perform under a realistic user load before you make SPDY available to the users on your site. I encourage you to test your own server and website using a load testing tool such as NeoLoad to see how SPDY will affect your users’ response times and your server’s performance.