Heavy-duty JavaScript processing

Very unfortunately, JavaScript, even its newer incarnations, have no sleep or wait or yield command, and they also have no way to ask the browser whether the DOM (Document Object Model), that has possibly been built or modified by your JavaScript program, has already been completely rendered to the screen.

Moreover, a long-running JavaScript program will be rudely interrupted by the browser, typically after 5 seconds, and the user will be asked whether he wants to kill it or continue, which is not good for our purposes of heavy-duty JavaScript computing.

If you are interested in details of this behavior, please read the following, excellent article for details:

What determines that a script is long-running?

Why would we want anything like a sleep or wait or yield command in the first place? One reason is that the use of JavaScript is exceeding all earlier expectations, and JavaScript is today used for purposes never foreseen by its original designers.

The problem shows up in heavy-duty JavaScript programs that render lots of data to the screen and also do lots of data processing in the background. While a JavaScript program runs, most browsers (except perhaps Opera) halt all screen rendering and wait until the JavaScript program finishes. No matter how well you finalize the in-memory DOM, the browser will not render it to the screen. You have to halt your JavaScript program first.

How to sleep, wait, or yield

There are only two ways to put a JavaScript program to sleep, short of a very ugly, processor-frying, endless loop, which can also fail due to most browsers' execution time limit of some 5 to 10 seconds:

1. The DOM (window) functions setTimeout(…) and setInterval(…)
2. Calling another program or function that can sleep

There are no easy, browser-independent solutions with the second choice, so for now we'll have to try to make do with the two JavaScript functions.

Breaking out

Unfortunately again, these two functions lead out of the function in which they are executed, so we lose our program flow status and have to start all over. For example, if one of these two functions is called inside any loops, we end up finding ourselves outside those loops.

The only redeeming fact is that we can still have access to local variables of the calling function through function parameters (except this, which we would have to preserve with the command var that = this; before calling.) The technical name for this wizardry, as far as objects are concerned, is closure. (Basic data type parameters are merely passed by value.)

But in many cases leaving your program flow is not a big problem. Either we are not in a loop, or it doesn't matter to start over, or we can wriggle ourselves back into the loop or loops with the help of some local variables, conveyed through function call parameters. My program Telly, for example, which does both lots of screen rendering and even more background data crunching, finds its way back into two nested loops after each precautionary interruption to evade the browsers' execution time limit.

So all we need to do is call the next function (or the same one recursively) like this:

function step1() {
  // Do some data processing here.
  setTimeout(step2, 20);
}

function step2() {
  // Do more data processing here.
}

In the 20 ms between the end of step1 and the start of step2 the browser can render the DOM to the screen.

The function step2 can have access to local variables of step1 through parameters, but in that case you have to do an extra step and wrap the actual function call in a usually nameless function for Internet Explorer, because IE cannot directly add parameters to a function reference:

function step1() {
  // Do some data processing here.
  var x = 7;
  setTimeout(function () { step2(x); }, 20);
}

function step2(x) {
  alert(++x); // Shows: 8
  // Do more data processing here.
}

Note that the basic, non-object data types are passed by value, so when we increment the variable x in step 2, we are not changing the variable x in step1, and we are therefore effectively not creating a closure.

If we have not changed much in the DOM and have only a little screen rendering to do, then this method should do fine. 20 ms are usually enough, given a clock granularity of some 18 ms, and if the browser really doesn't finish the rendering, it will have another chance every time we interrupt our processing again to avoid the execution time limit.

Likewise, if we don't have much background processing to do, we may not want to bother, as the rendering can wait until the background processing has been finished.

If, however, we have both a large amount of data to render to the screen and a lot of time-consuming background data processing to do, then those 20 ms may not be enough, depending how slow the computer is. You will see in some browsers (like Internet Explorer) that your screen rendering is interrupted, and the browser takes up JavaScript processing again, before the screen is fully rendered.

So we need to give the browser more time. Will 99 ms be enough? Or a whole second? We're in a quandary here, as we don't know how fast or slow the end user's computer is. It may be a mobile phone with a comparably slow processor and a tighter execution time limit, like only 1.5 s.

We also don't know how fast the browser is. Browser processing speed can vary widely, even from version to version, so attempts to predict that for each browser are futile.

No, none of that would not be a good solution. Instead we have to find a way to wait reliably for the actual rendering of the screen. We don't want to interrupt the processing on a fast computer for a whole second or two, just because we fear that the computer might be slow.

How to detect screen rendering

Asking the DOM for completion is of no use, because in most browsers DOM manipulation is quickly processed, but screen rendering happens after that, and it happens independently. JavaScript programs have no access to the screen rendering engine and cannot ask for its status. We have to devise a clever method to find out when the screen rendering is complete.

To be truly browser-independent, we cannot rely on testing existing browsers, because we cannot be sure whether future versions of these browsers or entirely new future browsers will always behave in the same way. We have to conceive of a method that will always work, no matter how the browser functions.

The following is experimental, but apparently works fine in Internet Explorer and Firefox and most likely in all other browsers as well. The basic idea is to create a DOM element way down the page, near the end of the DOM, then ask for a property that cannot be known before the actual rendering to the screen.

In my experiments I used a paragraph with an embedded span element that gets a few non-breaking spaces. It is absolutely positioned somewhere in the lower right behind all other elements and invisible on the user's screen. Then I ask for its inner width, which the browser cannot know before rendering. Let us call this element the "render-sensor".

In most contemporary browsers merely asking for such a rendering-dependent value already seems to trigger immediate screen rendering, but since we cannot rely on this for all present and future browsers, we prefer to rely on actually obtaining the rendering-dependent value from our sensor.

For simplicity I use jQuery $(…) functions, but of course the example could easily be converted to jQuery-free code.

renderThen(fn)

Here is one possible, experimental solution, a function that works similarly to setTimeout(…), but instead of a fixed number of milliseconds this one always waits until the screen is completely rendered before calling the other function. Let's call it renderThen(fn), meaning: first render to screen, then call the function fn.

function renderThen(fn) {
  $("body").append('\
<p style="margin: 0; padding: 0;\
 position: absolute; bottom: 0; right: 0;\
 z-index: -9999;">\
<span id="render-sensor"\
 style="margin: 0; padding: 0;\
 font-family: monospace; font-size: 10px;"\
>\xA0\xA0\xA0</span></p>');
  var renderSensor = $("#render-sensor");
  var counter = 0;
  var interval = setInterval(function () {
    if (renderSensor.length &&
        renderSensor.innerWidth() > 2 ||
        ++counter > 99) {
      clearInterval(interval);
      renderSensor.remove();
      fn();
    }
  }, 20);
}

The function keeps checking for the invisible paragraph to appear and for the span element inside it to assume some width. Whenever that happens, it stops and calls the function given to it as a parameter. Notes:

1. Some browsers may actually render, as soon as they are asked for something like an element width. That would be all the better, but we don't want to rely on this.
2. Be aware of the line-breaking syntax inside a string literal in JavaScript. The backslash at the very end of the line does that.
3. jQuery must be loaded for this function to work.
4. I position the render-sensor paragraph absolute and below the visible elements to make it as inconspicuous as possible. It should be totally invisible and have no influence on the sight of the screen whatsoever.
5. I use the generic monospace font to make sure the spaces have some considerable width.
6. setInterval(…) calls the function referenced in its first parameter repeatedly until it is cleared.
7. The counter makes sure we don't get into an endless loop, just because some exotic browser doesn't work as expected.

This is how you can use the function:

function step1() {
  // Do some data processing here.
  // Render to screen, then do step2:
  renderThen(step2);
}

function step2() {
  // Do more data processing here.
}

Note that, unlike the setTimeout(…) function the renderThen(…) function has no timeout value, as it always waits until the screen is fully rendered.

If you experiment with this and gain any new knowledge, please add a comment below.