Evaluating the Performance of SPDY-enabled Web Servers

As you may already know, SPDY (pronounced “SPeeDY”) is a new networking protocol introduced by Google and designed to reduce the web latency associated with HTTP. With SPDY, web pages load up to 64 percent faster than HTTP alone, according to Google. It accomplishes this by adding a session layer between HTTP and SSL that supports concurrent, interleaved streams over a single TCP connection. The initial draft of HTTP/2.0, the future of the web, is based on SPDY, which is a welcome step forward considering that HTTP/1.0 was released in 1996. SPDY holds great potential for mobile devices, for which latency is more of an issue and the market is catching on. As evidence of this, Microsoft recently announced the next version of Internet Explorer will support the new protocol.

Today, SPDY can be deployed on Apache by installing a single module (mod-spdy, the Apache SPDY module). No changes to website content are required. SPDY is now enabled by default on many of the latest modern browsers, including Chrome, Firefox, Opera, and Android Browser. (You can check your own browser’s support by visiting isspdyenabled.com).

So, simply by deploying the SPDY module on your Apache server, you can provide a large percentage of your users with better response times! More importantly, the response times of a website are correlated with the site’s conversion rate and bounce rate—so SPDY can have a significant positive impact on your business!

With all the advantages of SPDY, you may be wondering if there is a price to pay on the server side. Moving from HTTP to SPDY adds encryption and compression overhead, which clearly requires more resources. It does, however, use fewer TCP connections. It wasn’t immediately clear to me if all these changes in aggregate would cause noticeable drawbacks on the server.

To get to the bottom of this question, I thought it would be interesting to get some real-world performance metrics on SPDY, in particular its effect on the server side. To obtain this information, I used the new SPDY support in NeoLoad to run load tests of SPDY and compare its performance against HTTP and HTTPS.

Test Configuration Details

My approach was to test a server with a fairly typical configuration—not one that is finely tuned for a specific purpose. This setup provides a basis for comparing HTTP, HTTPS, and SPDY. It’s important to note here that changes to the application or the server configuration will produce different results—perhaps vastly different.

Test setup:

• Server:
  • OS: Linux CentOS 6.3
  • Hardware:
    • Intel Core i3 CPU 540 @ 3.07GHz
    • 4GB RAM
  • Apache
    • Version 2.2.15
    • worker MPM: <IfModuleworker.c>
      StartServers 5
      MaxClients 600
      MinSpareThreads 50
      MaxSpareThreads 150
      ThreadsPerChild 50
      MaxRequestsPerChild 0
      </IfModule>
    • SPDY module (mod-spdy-beta-0.9.3.3-386.x86_64). Default SPDY optimization flags were used (uncommented suggested values):
      • SpdyMaxThreadsPerProcess 30
      • SpdyMaxStreamsPerConnection 100
• Application:
  • 1 web page: 1 HTML 9.5kB + 50 PNG icons (32×32).
  • The web page and all resources were served by the same server.
• Load testing tool: NeoLoad 4.1.2
• Tests:
  • Ramp-up from 1 to 1000 users for 20 minutes.
  • Scenario: 1 page call, wait for 5s, restart.
  • Chrome browser simulated, with 6 parallel connections to the server.
  • Emulated latency: 40ms in download, 45ms in upload.
  • Test HTTP, HTTPS, and SPDY under the same conditions.
  • Aggregation period of 30s for graphs. This smoothes variations to simplify comparison between several tests.

Results and Analysis

First I wanted to examine the hit rate (hits per second), average response time (duration), and number of errors as the number of concurrent users increased.

Here are the results for the HTTP test:

The plot illustrates the following:

• At 3 minutes, the maximum number of page per second is reached.  This maximum rate (16 pages/s or 827 request/s) is reached at 120 virtual users (VU), and the rate decreases as more concurrent users are added.
• At the same time (the 3-minute mark), response time starts to increase significantly.
• The first errors (which investigation revealed to be timeouts) appear at 405 users.

The table below compares these metrics for HTTP, HTTPS, and SPDY:

Based on a service-level agreement (SLA) of less than 3s for page load time, my test server handled almost six times as many users with SPDY than HTTP!

More Users With Fewer Workers

To handle incoming requests in parallel, Apache web servers use worker threads to process each request. Because each thread consumes resources and memory, Apache lets the system administrator limit thread use, which has an effect on how incoming requests are handled.

My HTTP and HTTPS results are typical of a scenario known as thread starvation.

This is confirmed by the following graph, which shows busy workers in orange, page rate in green, and user load in blue:

The number of busy Apache workers (threads) hits its maximum at the same time the page rate reaches its maximum.

In contrast to HTTP/S, SPDY uses a single connection for all requests. The principal benefit of this is the decreased latency seen by the client, resulting in reduced response times for users. There is another great benefit on the server side: SPDY clients consume one worker instead of six. As a result, it takes much longer (that is, many more users) for all workers to become busy:

SPDY enables the server to handle more users with the same number of workers.

Server CPU and Memory Consumption

So far, I showed how SPDY affects Apache worker usage with my test configuration. Next, I wanted to see how CPU usage and memory were affected by SPDY.

Here is a plot showing system idle time (low when the CPU is used intensively) for HTTP/HTTPS/SPDY:

No surprise here. As most HTTP/HTTPS requests are waiting to be served after the thread limit is reached, they don’t consume much CPU. For the SPDY tests, Apache is below the worker limit for much longer, resulting in a higher page rate and consequently more CPU used. The curve plateau is reached when all workers are busy (around 11 minutes).

To better understand the behavior before the worker limit is reached, I focused on the numbers at 2 minutes, 88 virtual users, and 12.5 pages/s. The CPU idle time measurements are an average of six values taken over 30 seconds:

The values are comparable. SPDY consumes slightly less CPU than HTTPS and slightly more than HTTP for this load.

On the memory side, here’s the global overview:

Again, it’s interesting to examine the memory used before the worker limit is reached. The numbers are an average of six measurements over 30 seconds.

The server consumed less memory when handling SPDY requests than when handling HTTP and HTTPS.

During this test, SPDY consumed just 41% of the memory consumed by HTTP, and just 30% of the memory consumed by HTTPS.

Conclusion

It’s no surprise that SPDY improves response times on the client side. That’s what it was designed to do. It turns out that SPDY also has advantages on the server side:

• Compared to HTTPS, SPDY requests consume less resources (CPU and memory) on the server.
• Compared to HTTP, SPDY requests consume less memory but a bit more CPU. This may be good, bad, or irrelevant depending on which resource (if either) is currently limiting your server.
• Compared to HTTP/S, SPDY requires fewer Apache worker threads, which increases server capacity. As a result, the server may attract more SPDY traffic.

From these results, it is clear that Apache tuning performed for HTTP/S may not be appropriate after SPDY is enabled on the server. It’s important to take the time to re-evaluate server sizing and retune the server before you start handling SPDY requests from your users.

As I mentioned earlier, the test results I’ve shared here are specific to my server configuration and the web page I used for the tests. It’s important to understand how your server will perform under a realistic user load before you make SPDY available to the users on your site. I encourage you to test your own server and website using a load testing tool such as NeoLoad to see how SPDY will affect your users’ response times and your server’s performance.