Performance Calendar

Speed engineers rely on business buy-in to get the job done. Telling a clear story will gain you resources and help when it’s time to balance speed against other marketing considerations.

The cost of a garbled message? Wasted time, frustrated users, and lost sales. Last year, the big sites got 23% slower and lost billions as a result.

We already know how to start the speed story simply. “Customers show up to our slow website, get thwarted, and move on without spending money”. But what happens next? How do you handle the exec who gut-checks by pulling up his cached site from a high-speed T3 line?

Boil it down

You need hard numbers to clearly explain the situation. Unfortunately, many speed measures aren’t intuitive or even very useful. By the time you’ve explained 95th percentile, you’ve lost your audience.

A single measure can explain how the site is performing across all your different pages and customers. It’s got to handle everyone’s intuitive feel that some pages are especially important. And this number should quickly give the gist, even if the details are tricky to compute. Hierarchal Average Load Time (HALT) combines the critical attributes:

Load times across different pages in the website
Different connection speeds, geographic locations, caching, etc. proportional to customer usage
Weighting by customer value, page value, and speed sensitivity

HALT is a single number that’s simple to grasp. However, the calculation takes some effort. HALT integrates many different measures, so let’s unpack it a little.

1 – Break your pages down

Different audiences react to speed differently. If your customers display different speed behavior by country, mobile/fixed, or any other trait, split your page-level data. Just make sure these factors are statistically significant.

2 – Start at the median

The calculation begins with median (not average) load times on each page. Averages get bumped around by this type of data, but the load time for the middle point isn’t skewed by a few 100-second outliers.

3 – Measure in proportion to the userbase

Your coworkers normally access a Potemkin Village version of the website. Cached files, fast connection, and servers next to the browser. It’s artificial, but can feel real when your arguments for performance optimization are being evaluated.

If 10% of our users happen to access the site from Topeka, Kansas on a dial-up modem with Internet Explorer during business hours, then 10% of measurements should replicate those conditions. Simple execution if you’re using Real User Measurement tools like Google Analytics. If RUM doesn’t work for you, approximate this sampling with careful setup in synthetic measurement tools like Web Page Test.

4 – Weight by importance

Let’s face it, your legal disclaimers are less important to your customers than your home page. A weighted average of all the median load times keeps everything in proportion.

So how do you judge each page’s importance for weighting?

Number of page views – higher volume pages matter more. RUM pulls this data automatically, but server logs can help you proportion synthetic testing.
Page value – what’s the average value of goals achieved (goods sold, videos watched, etc.) after visiting this page? If you don’t have an analytics package installed, poll your marketing team to take an educated guess.
Speed sensitivity – do customers care about speed on this particular page? Is it reflected in page value? For example, your Cart Checkout page might show much more sensitivity than your home page. Be careful – correlation is definitely not causation. If the numbers are unavailable, assume constant sensitivity.

Putting it together

        sum (median load time per page * # of page views * page value * speed sensitivity)
HALT = ------------------------------------------------------------------------------------
        sum (number of page views * page value * speed sensitivity)

Your HALT number will be measured in seconds – and hopefully trend downwards. If it isn’t headed the right way, start digging in:

Apply speed technology and coding changes across the board
Focus on the most important pages and customer demographics
Watch for shifts in the key pages and customers

Keep it clean

Load time makes sense to many people, but there are good arguments for different measurements like Speed Index, Time To Interact or Above-The-Fold Time.

Whatever you do, play it square. Some useful techniques can modify load times (like deferred loads). If they unfairly skew times, make sure to either highlight the change or adjust numbers in compensation.

Likewise, if you take shortcuts, make them obvious. If you don’t have all the information you’d like, no problem. But as you make changes to your numbers, highlight the differences.

Remember to use your words and numbers like poetry – pack them with meaning. Your story will sink in better and coworkers will start thinking speed to reach their goals.