Capability Reporting with Service Worker

By Ilya Grigorik on December 15, 2014

Some people, when confronted with a problem, think: “I'll use UA/device detection!” Now they have two problems...

But, despite all of its pitfalls, UA/device detection is a fact of life, a growing business, and an enabling business requirement for many. The problem is that UA/device detection often frequently misclassifies capable clients (e.g. IE11 was forced to change their UA); leads to compatibility nightmares; can't account for continually changing user and runtime preferences. That said, when used correctly it can also be used for good.

Browser vendors would love to drop the User-Agent string entirely, but that would break too many things. However, while it is fashionable to demonize UA/device detection, the root problem is not in the intent behind it, but in how it is currently deployed. Instead of "detecting" (i.e. guessing) the client capabilities through an opaque version string, we need to change the model to allow the user agent to "report" the necessary capabilities.

Granted, this is not a new idea, but previous attempts seem to introduce as many issues as they solve: they seek to standardize the list of capabilities; they require agreement between multiple slow-moving parties (UA vendors, device manufacturers, etc); they are over-engineered - RDF, seriously? Instead, what we need is a platform primitive that is:

Flexible: browser vendors cannot anticipate all the use cases, nor do they want or need to be in this business beyond providing implementation guidance and documenting the best-practices.
Easy to deploy: developers must be in control over which capabilities are reported. No blocking on UA consensus or other third parties.
Cheap to operate: compatible and deployable with existing infrastructure. No need for third-party databases, service contracts, or other dependencies in the serving path.

Here is the good news: this mechanism exists, it's Service Worker. Let's take a closer look...

Service worker is an event-driven Web Worker, which responds to events dispatched from documents and other sources… The service worker is a generic entry point for event-driven background processing in the Web Platform that is extensible by other specifications - see explainer, starter, and cookbook docs.

A simple way to understand Service Worker is to think of it as a scriptable proxy that runs in your browser and is able to see, modify, and respond to, all requests initiated by the page it is installed on. As a result, the developer can use it to annotate outbound requests (via HTTP request headers, URL rewriting) with relevant capability advertisements:

Developer defines what capabilities are reported and on which requests.
Capability checks are executed on the client - no guessing on the server.
Reported values are dynamic and able to reflect changes in user preference and runtime environment.

This is not a proposal or a wishlist, this is possible today, and is a direct result of enabling powerful low-level primitives in the browser - hooray. As such, now it's only a question of establishing the best practices: what do we report, in what format, and how to we optimize interoperability? Let's consider a real-world example...

E.g. optimizing video startup experience

Our goal is to deliver the optimal — fast and visually pleasing — video startup experience to our users. Simply starting with the lowest bitrate is suboptimal: fast, but consistently poor visual quality for all users, even for those with a fast connection. Instead, we want to pick a starting bitrate that can deliver the best visual experience from the start, while minimizing playback delays and rebuffers. We don't need to be perfect, but we should account for the current network weather on the client. Once the video starts playing, the adaptive bitrate streaming will take over and adjust the stream quality up or down as necessary.

The combination of Service Worker and Network Information API make this trivial to implement:

// register the service worker
navigator.serviceWorker.register('/worker.js').then(
    function(reg) { console.log('Installed successfully', reg) },
    function(err) { console.log('Worker installation failed', err) }
);

// ... worker.js
self.addEventListener('fetch', function(event) {
    var requestURL = new URL(event.request);

    // Intercept same origin /video/* requests
    if (requestURL.origin == location.origin) {
        if (/^\/video\//.test(requestURL.pathname)) {
            // append the MD header, set value to NetInfo's downlinkMax:
            // http://w3c.github.io/netinfo/#downlinkmax-attribute
            event.respondWith(
                fetch(event.request.url, {
                    headers: { 'MD': navigator.connection.downlinkMax }
                })
            );
            return;
        }
    }
});

Site installs a Service Worker script that is scoped to capture /video/* requests.
When a video request is intercepted, the worker appends the MD header and sets its value to the current maximum downlink speed. Note: current plan is to enable downlinkMax in Chrome 41.
Server receives the video request, consults the advertised MD value to determine the starting bitrate, and responds with the appropriate video chunk.

We have full control over the request flow and are able to add additional data to the request prior to dispatching it to the server. Best of all, this logic is transparent to the application, and you are free to customize it further. For example, want to add an explicit user override to set a starting bitrate? Prompt the user, send the value to the worker, and have it annotate requests with whatever value you feel is optimal.

Tired of writing out srcset rules for every image? Service Worker can help deliver DPR-aware 's: use content negotiation, or rewrite the image URL's. Note that device DPR is a dynamic value: zooming on desktop browsers affects the DPR value! Existing device detection methods cannot account for that.

Implementation best practices

Service Worker enables us (web developers) to define, customize, and deploy new capability reports at will: we can rewrite requests, implement content-type or origin specific rules, account for user preferences, and more. The new open questions are: what capabilities do our servers need to know about, and what's the best way to deliver them?

It will be tempting to report every plausibly useful property about a client. Please think twice before doing this, as it can add significant overhead to each request - be judicious. Similarly, it makes sense to optimize for interoperability: use parameter names and format that works well with existing infrastructure and services - caches and CDN's, optimization services, and so on. For example, the MD and DPR request headers used in above examples come from Client-Hints, the goals for which are:

To document the best practices for communicating client capabilities via HTTP request header fields.
Acts as a registry for common header fields to help interoperability between different services.
- e.g. you can already use DPR and RW hints to optimize images with resrc.it service.

Now is the time to experiment. There will be missteps and poor initial implementations, but good patterns and best practices will emerge. Most importantly, the learning cycle for testing and improving this infrastructure is now firmly in the hands of web developers: deploy Service Worker, experiment, learn, and iterate.

Ilya Grigorik is a web performance engineer and developer advocate at Google, where his focus is on making the web fast and driving adoption of performance best practices — follow on Twitter, Google+.