Concurrency in JavaScript | TypedArray.org

Just like with Flash, JavaScript code runs by default on the UI thread, and any expensive computation will usually affect the UI responsiveness. As you may know, at 60 fps, you have around 16ms (1000ms/60) per frame to do what you have to do (computations, rendering and other misc logic). If you exceed that budget, you will alter the frame rate and potentially make your content feel sluggish or worse, unresponsive.

Frame budget

Web Workers are now broadly available in most browsers even on mobile (caniuse.com stats for Web Workers) and give you the power of concurrency from within JavaScript. It will allow you to move expensive computations to other threads, to permit best responsive programming, and ideally open the doors in the future to true parallelization in JavaScript. Let’s have a look at the reasons why you may be interested into leveraging Web Workers.

Responsive programming: When you have an expensive computation and don’t want to block the UI.
Parallel programming: When you want to leverage multiple CPU cores by having computations running concurrently to solve a specific task.

We will see later on that parallel programming with Web Workers can be challenging, and that they are not really designed for that today unfortunately. But before we dive into the details of responsive programming and parallel programming, let’s start with some more details about Web Workers. Note that we will cover here dedicated Web Workers, not the shared Web Workers.

Workers and threads

A very common question around Web Workers is how they compare to low-level threads available in other languages like C++, Java or C#.

First off, Web Workers are way more high-level and heavier than threads, when newing a Web Worker, you are actually instantiating a new JS VM. As a result, instantiating a Web Worker does indeed create another low-level thread behind the scenes to power the VM that runs your code, but you never work with it directly. Therefore, it is important to remember that the cost of a single Worker in memory is pretty high, and same for its instantiation time. Based on that, you can imagine that instantiating many workers on a mobile browser is usually not an option, in fact, most browsers have a limitation in the number of workers that can be allocated for that reason.

Another major difference with threads is how data passing works, where nothing is shared. As we will see later with Web Workers, data is passed by default through message cloning, in other words, any data passed is cloned. Given the overhead of cloning for large objects, you can also transfer ownership when needed. That allows you to pass the reference of your object to another Web Worker, without cloning it, you just lose its reference from where you sent it. It is a pretty elegant approach and very simple to use and works well with the responsive use cases.

Workers and cores

On a multi-core environment, multiple threads can truly run concurrently if distributed over multiple cores. On a single core system, the CPU performs what is called context switching betweens the threads. It happens so fast that you cannot actually tell the threads are not running concurrently but they truly aren’t. This leads to another common question: can I leverage multi-core architectures with Web Workers ? In other words, can I choose specifically which core to leverage?

That is not in your control, the OS will decide if the thread utilized by the instantiated VM is spawn on another core or not. It is also not possible to query the number of cores available, even if libraries have emerged to guess that, but you ideally want to stay out of this and let the OS balance things best for you.

Life as a worker

You can use all of the JavaScript features inside a Worker, but the amount of browser APIs available is limited. The main restriction is that there is no way to display anything inside from a Worker, simply because there is no DOM available, which lives on the UI thread. Imaging making the DOM thread safe, that would be pretty tricky across workers. Hence why if you try to access the document object from within a worker, you will get the following runtime exception:

Uncaught ReferenceError: document is not defined

At this point, an error event will be dispatched by the worker, that can be listened to. As a result, it is impossible to render anything on screen from a Web Worker. To achieve this, you will have to send the object back to the main (UI) thread for display. A very classic scenario is anything related to image processing. Given how expensive image processing can be, it is very tempting to offload the expensive code to a Web Worker. In the initial Web Workers specification, it was not possible to pass an ImageData object between Web Workers, you had to pass an array of pixel colors. Recently, the specification got updated to allow message passing of ImageData objects, which is quite nice, given that many use cases rely on image/bitmap manipulation.

Another important use case is resource loading. You may want to load a remote resource from a worker, and that is possible through the XMLHttpRequest object. That is very useful if you want to do some expensive parsing of a resource that you are loading within a game, like parsing an object model, or more simply logging to a server some analytics you have been cooking within a background worker. You can find here more details about the functions available from within Web Workers.

Another thing you may try is to use the console to log things from within a worker, this is also not supported, but you can build a wrapper on top of the postMessage API that we will cover in a few minutes. David Flanagan has posted one there which works well. In addition, you can also use the Chrome Dev Tools which offer you a way to debug Web Workers.

Instantiating a Worker

To instantiate a worker, two approaches are available. The first one requires you to point to the JavaScript code needed to run in the background:

var worker = new Worker("background.js");

When the script logic is passed to the Worker object, the code is immediately ran. Simple and easy. The limitation here is that it relies on a separate file, you may be in a situation where you need to reduce as much as possible the file dependencies. Another technique allows you to remove such a dependency and create the worker logic through the createObjectURL API and a Blob object:

// worker logic var blob = new Blob(["self.addEventListener('message', messageReceived);function messageReceived(e) {self.postMessage('Doing pretty good here!');}"], {type: 'text/javascript'}); // webkit handling var URL = window.URL || window.webkitURL; // create the virtual file var code = URL.createObjectURL(blob); // create the worker var worker = new Worker(code);

// worker logic

var blob = new Blob(["self.addEventListener('message', messageReceived);function messageReceived(e) {self.postMessage('Doing pretty good here!');}"], {type: 'text/javascript'});

// webkit handling

var URL = window.URL || window.webkitURL;

// create the virtual file

var code = URL.createObjectURL(blob);

// create the worker

var worker = new Worker(code);

Note that the way you retrieve the string passed to the Blob object can vary. Obviously, passing the string like above is not very optimal. As Mozilla points out in their documentation, you can also store the JavaScript code inside a custom script tag and extract its content dynamically and pass it to the Blob object.

In the code below, the JavaScript code is located inside a custom script tag:

self.addEventListener('message', messageReceived);

function messageReceived(e) {

self.postMessage('Doing pretty good here!');

}

</script>

Did you notice the use of the keyword self? This is how we refer (from the background code) to the worker itself.

We retrieve the script using the querySelector API and pass it to the Blob object:

// extract the string

var logic = document.querySelectorAll("script[type=\"text\/js-worker\"]")[0].textContent;

// worker logic

var blob = new Blob([logic], {type: 'text/javascript'});

// webkit handling

var URL = window.URL || window.webkitURL;

// create the virtual file

var code = URL.createObjectURL(blob);

// create the worker

var worker = new Worker(code);

Relying on a separate file for the worker logic is the most common way to work with Workers but you now have seen different options in case you had to get rid of a dependent file.

Terminating a Web Worker

To terminate a Worker, just call the terminate API:

In the same way, a Worker can close itself through the close API:

Note that once terminated or closed, a Web Worker cannot be restored, it would have to be recreated. Now that our Worker object is created, let’s send some data back and forth to see how things work.

Sub Web Workers

If needed, a Worker can create a Worker within itself:

self.addEventListener('message', messageReceived); // a sub worker var subworker = new Worker ('Task.js'); subworker.addEventListener('message', subTaskReceived); function messageReceived(e) { // some logic } function subTaskReceived(e) { // some logic }

self.addEventListener('message', messageReceived);

// a sub worker

var subworker = new Worker ('Task.js');

subworker.addEventListener('message', subTaskReceived);

function messageReceived(e) {

// some logic

}

function subTaskReceived(e) {

// some logic

}

Passing data

To communicate, we use the postMessage API (not to be confused with window.postMessage). Any message sent with postMessage triggers the “message” event that dispatched by the Worker object:

// extract the string

var logic = document.querySelectorAll("script[type=\"text\/js-worker\"]")[0].textContent;

// worker logic

var blob = new Blob([logic], {type: 'text/javascript'});

// webkit handling

var URL = window.URL || window.webkitURL;

// create the virtual file

var code = URL.createObjectURL(blob);

// create the worker

var worker = new Worker(code);

// listen to the response from the Worker

worker.addEventListener('message', receiveMessage);

// pass some data

worker.postMessage('How is it going over there?');

// callback handling the response, data is available in the event object

function receiveMessage (e)

{

console.log (e.data);

}

Our code sends a message to the Worker, which returns a message back to the main thread. The code above outputs the following message:

In that case, we sent the string: ‘How is it going over there’, and the Worker returned ‘Doing pretty good here’. We sent different data to illustrate how simply message passing works. But in most cases you want to send similar data that you can retrieve on both sides. Some coordinates, a path that has been computed in a game, or more simply a binary blob that you want to process, like a PDF or an image that has been generated. To retrieve the data that is being passed, we use the data property of the event object.

In the code below, we update our worker logic to return the data back to the main thread:

self.addEventListener('message', messageReceived); function messageReceived(e) { self.postMessage(e.data); }

self.addEventListener('message', messageReceived);

function messageReceived(e) {

self.postMessage(e.data);

}

We still pass a simple string, but it is now sent back to the main thread:

// extract the string

var logic = document.querySelectorAll("script[type=\"text\/js-worker\"]")[0].textContent;

// worker logic

var blob = new Blob([logic], {type: 'text/javascript'});

// webkit handling

var URL = window.URL || window.webkitURL;

// create the virtual file

var code = URL.createObjectURL(blob);

// create the worker

var worker = new Worker(code);

// listen to the response from the Worker

worker.addEventListener('message', receiveMessage);

// we send some data

worker.postMessage("Some data passed back and forth!")

// callback handling the response, data is available in the event object

function receiveMessage (e)

{

console.log (e.data);

}

If we run the code above, the console outputs:

Some data passed back and forth!

Remember that we are not sharing anything here, our string got cloned implicitly. Now you may wonder, can I pass any kind of data? If you stick to all the primitive types available in the JavaScript language, like Number, String, Boolean, you will be fine. For composite data types, plain JSON objects are supported, in addition to the ImageData and TypedArray types.

In the code below, we pass a simple JSON object:

// a user var user = { name : "Bob", age: 30, city: "San Francisco" }; // we pass our object worker.postMessage ( user );

// a user

var user = { name : "Bob", age: 30, city: "San Francisco" };

// we pass our object

worker.postMessage ( user );

Here again, the object is cloned and accessible from our Worker code:

self.addEventListener('message', messageReceived); function messageReceived (e) { var user = e.data; var name = user.name; var age = user.age; var city = user.city; }

self.addEventListener('message', messageReceived);

function messageReceived (e)

{

var user = e.data;

var name = user.name;

var age = user.age;

var city = user.city;

}

If you try to pass an unsupported type, you will get a DataCloneError exception. In the code below, we try to pass a reference to the window object:

// attempt to pass the window object worker.postMessage(this.window);

// attempt to pass the window object

worker.postMessage(this.window);

Which triggers the following exception:

Uncaught Error: DataCloneError: DOM Exception 25

Same thing with an XHR object:

// attempt to pass an XHR object worker.postMessage(new XMLHttpRequest());

// attempt to pass an XHR object

worker.postMessage(new XMLHttpRequest());

Which also results in a runtime exception:

Uncaught Error: DataCloneError: DOM Exception 25

In the same way, passing a custom type is not supported. In the code below, we try to pass an Array of Ball objects:

var radius = 10;

var Ball = (function () {

function Ball(radius, x, y, destX, destY, context, balls) {

this.friction = .1;

this.color = 'green';

this.radius = radius;

this.x = x;

this.y = y;

this.destX = destX;

this.destY = destY;

this.context = context;

this.balls = balls;

}

return Ball;

})();

var BALLS_NUM = 100;

var balls = new Array();

for(var i = 0; i < BALLS_NUM; i++) {

var obj = new Ball(radius, Math.round(Math.random() * 1024), Math.round(Math.random() * 768), Math.random() * 1024, Math.random() * 768, context, balls);

balls.push(obj);

}

// attempt to pass an array of custom type (Ball) objects

worker.postMessage (balls);

This code will also trigger the same DataCloneError exception. As you can imagine, passing data through cloning has overhead. If you try to pass a big amount of data at a high frequency, cloning will not be the most efficient path. However, this approach has one merit, it is very safe. Given that everything is cloned, it is therefore impossible to get in a situation where data is corrupted or simply ending up with unsynchronized shared data.

Sending larger objects

In the following example, we create a 16mb byte array that we send to the worker, we capture the time it takes to allocate and transfer back and forth:

// extract the string var logic = document.querySelectorAll("script[type=\"text\/js-worker\"]")[0].textContent; // worker logic var blob = new Blob([logic], {type: 'text/javascript'}); // webkit handling var URL = window.URL || window.webkitURL; // create the virtual file var code = URL.createObjectURL(blob); // create the worker var worker = new Worker(code); // listen to the response from the Worker worker.addEventListener('message', receiveMessage); // capture current time var started = Date.now(); // Create a 16MB "file" and fill it. var uInt8View = new Uint8Array(1024*1024*16); // 16MB for (var i = 0; i < uInt8View.length; ++i) { uInt8View[i] = i; } console.log ( "Memory allocation : " + (Date.now() - started) + " ms" ); // capture current time started = Date.now(); // pass the bytearray (cloned) worker.postMessage(uInt8View.buffer); // callback handling the response, data is available in the event object function receiveMessage (e) { console.log ("Back/forth transfer : " + (Date.now() - started) + " ms"); console.log ("Byte array size : " + e.data.byteLength); }

// extract the string

var logic = document.querySelectorAll("script[type=\"text\/js-worker\"]")[0].textContent;

// worker logic

var blob = new Blob([logic], {type: 'text/javascript'});

// webkit handling

var URL = window.URL || window.webkitURL;

// create the virtual file

var code = URL.createObjectURL(blob);

// create the worker

var worker = new Worker(code);

// listen to the response from the Worker

worker.addEventListener('message', receiveMessage);

// capture current time

var started = Date.now();

// Create a 16MB "file" and fill it.

var uInt8View = new Uint8Array(1024*1024*16); // 16MB

for (var i = 0; i < uInt8View.length; ++i) {

uInt8View[i] = i;

}

console.log ( "Memory allocation : " + (Date.now() - started) + " ms" );

// capture current time

started = Date.now();

// pass the bytearray (cloned)

worker.postMessage(uInt8View.buffer);

// callback handling the response, data is available in the event object

function receiveMessage (e)

{

console.log ("Back/forth transfer : " + (Date.now() - started) + " ms");

console.log ("Byte array size : " + e.data.byteLength);

}

If we run this code on different devices and environments, we get the following results:

Platform	Time (ms)
iOS7 (Safari/iPhone5)	214
iOS6 (Safari/iPhone4S)	524
MacBook Pro (Chrome/10.8.4)	75

Note how expensive data cloning is on a recent device and OS. Let’s see if transfer of ownership can help performance.

Transferable objects

In a scenario where you need to pass a large amount of data and cloning could be too costly, you can rely on transferrable objects. To transfer an object, the postMessage API accepts a second optional second argument, which is an array containing the objects to transfer ownership from:

// transfer ownership to the worker worker.postMessage(uInt8View.buffer, [uInt8View.buffer]);

// transfer ownership to the worker

worker.postMessage(uInt8View.buffer, [uInt8View.buffer]);

With that change, we see that passing data becomes much more efficient, around a 3x performance boost on mobile:

Platform	Time (ms)
iOS7 (Safari/iPhone5)	80
iOS6 (Safari/iPhone4S)	162
MacBook Pro (Chrome/10.8.4)	37

Note that just before calling postMessage, our byte array is available, right after the transfer, the reference is no longer accesssible and trying to access it throws a runtime exception:

// outputs: 16777216 // access to the bytearray possible console.log ( uInt8View.buffer.byteLength ); // transfer ownership to the worker worker.postMessage(uInt8View.buffer, [uInt8View.buffer]); // triggers runtime exception: Uncaught TypeError: Cannot read property 'byteLength' of null console.log ( uInt8View.buffer.byteLength );

// outputs: 16777216

// access to the bytearray possible

console.log ( uInt8View.buffer.byteLength );

// transfer ownership to the worker

worker.postMessage(uInt8View.buffer, [uInt8View.buffer]);

// triggers runtime exception: Uncaught TypeError: Cannot read property 'byteLength' of null

console.log ( uInt8View.buffer.byteLength );

This technique can be useful in a scenario where you need to pass a big amount of data, like a big binary blob. Note that since the beginning, we passed data from the main thread to the Worker, but data could be passed the other way around. This time, we will create the memory buffer from the worker and send it to the main thread.

To achieve this, we modify our worker logic:

// Create a 16MB "file" and fill it. var uInt8View = new Uint8Array(1024*1024*16); // 16MB for (var i = 0; i < uInt8View.length; ++i) { uInt8View[i] = i; } self.postMessage(uInt8View.buffer, [uInt8View.buffer]);

// Create a 16MB "file" and fill it.

var uInt8View = new Uint8Array(1024*1024*16); // 16MB

for (var i = 0; i < uInt8View.length; ++i) {

uInt8View[i] = i;

}

self.postMessage(uInt8View.buffer, [uInt8View.buffer]);

Then, we listen to the message coming from the worker:

// extract the string

var logic = document.querySelectorAll("script[type=\"text\/js-worker\"]")[0].textContent;

// worker logic

var blob = new Blob([logic], {type: 'text/javascript'});

// webkit handling

var URL = window.URL || window.webkitURL;

// create the virtual file

var code = URL.createObjectURL(blob);

// create the Worker

var worker = new Worker(code);

// listen to the incoming message from the Worker

worker.addEventListener('message', receiveMessage);

function receiveMessage (e) {

console.log (e.data.byteLength);

}

If we run the code above, the console outputs:

We are now transferring our data from the background Web Worker to the main UI thread.

Responsive use cases

As we said earlier, Web Workers are commonly used today to perform expensive/synchronous tasks in the background. This allows you as a developer to perform expensive computations in the background, without causing the UI to lock so that your application stays always responsive.

Here is below a few use cases:

Generating or parsing/rendering a PDF.
Compressing an image to a custom file format (JPEG, etc.).
Compute an A* path inside a game.
Lookup a word in a database for a spell checker in a text editor.
Load and parse some resources inside a game.
Perform general complex computations in the background and log that remotely.
And many others…

For these, message cloning works just fine, because low-latency is acceptable. All you need is to pass your data back to the UI thread and that’s it, you don’t need to pass messages at a high frequency

Concurrency and the future of the web

As you can see, Web Workers are very easy to use and pretty powerful. The message passing model is simple and easy and they can be leveraged today on desktop and mobile to make your application more responsive and snappier. So, are we done with them or could they be improved? We will see in the following section that there is room for a few improvements that could really move the web forward.

Parallelization

So far, we have covered techniques to offload expensive computations to a background worker, using message cloning and transfer of ownership, but we have not discussed the parallelization use cases. Instead of relying on a single Web Worker to perform a computation in the background, the idea of parallelization is to have multiple Web Workers run in parallel and working on solving the same task but faster, given that we distribute the task to multiple threads. Microsoft has posted a good example about this for image manipulation.

As we have seen, two communication models currently exist for Web Workers, cloning and ownership transfer, unfortunately, both do not work well today for parallelization. The former does not, giving data ownership to one Web Worker at a time. The latter allows you to parallelize tasks by duplicating data when needed, but the cloning has real overhead and that can become limiting in some scenarios. In my tests, passing an array of 100 JSON objects over a few frames brought the performance to a crawl. Being able to truly share some memory (like a simple TypedArray) between workers would be fast, but it would expose some additional complexity to web developers, like having to deal with synchronization of data.

Most of the time, the idea of shared memory makes people worried about nasty things we usually don’t want to deal with, like data races and locks leading to memory corruption. I actually agree that within the context of low-level languages, this is pretty scary, but with a managed language like JavaScript, I would argue that it is not as bad. In addition, the main UI thread and Web Workers have been designed so that blocking cannot happen when one is waiting for the other and blocking the UI can already happen today with a simple while (true) sitting on the main thread.

So, really, I am not sure a shared memory model with Web Workers would necessarily hurt the web, but I agree that there is probably a better solution than exposing plain simple shared memory. This would expose to web developers some additional complexity, like dealing with conditions and mutexes, and there’s gotta be something better for web developers. Anyway, I am following discussions about bringing a better message passing model to Web Workers with great passion.

Shared resources (WebGL)

Web Workers also offer some great potential when it comes to GPU programming. The most exciting one to me, is the idea of shared resources for WebGL relying on Web Workers. Today, when you are programming with WebGL, all the initialization work required to initialize WebGL, like shader programs initialization and texture uploads happen on the UI thread. In other words, to see something on screen, WebGL requires you to upload your textures and programs (vertex/fragments shaders) and all of this still happens on the main UI thread. In addition. two things usually required when working with GPU APIs have always been slow historically and most importantly blocking operations: texture upload and texture readback.

This is a big limitation, as you obviously need to upload textures and this will once again lock the UI if expensive, making your game/application feel janky. Same thing for texture readback (which means copying the texture uploaded on the GPU back to RAM) to perform other computations on the CPU. With shared resources support for WebGL, multiple contexts could be created across Web Workers to perform some of the tasks mentioned earlier in other threads, allowing developers to produce lock-free, snappy, super responsive GPU experiences.

Additional Web Workers resources

I hope you enjoyed reading these additional details on Web Workers!

Posted on July 1, 2013 by Thibault Imbert · 21 comments Tags: asynchronous, concurrency, threading, workers.