The Callback Syndrome In Node.js

When starting out in event-driven programming, you often find yourself amazed at how it works. It is pretty much what it feels like when you discover loops or conditional statements. As you progress, it becomes obvious that the overuse of the new paradigm will most likely prevent anyone else from successfully going through your code. This article dissects the problem of asynchronous control flow in JavaScript and suggests a simple solution using closures.

What It Is

The callback syndrome describes a readability and maintainability anti-pattern found in the JavaScript code. On the whole, it means nesting callbacks, which is similar to nesting conditional statements or loops in sequential code. Still, the trouble starts when the level of nesting complicates the code to such an extent that understanding its functionality gets difficult and bugs start to creep in from unaddressed error conditions.

function startExecution() {
    doSomethingAsync(function(error, result) {
        doSomethingElseAsync(function(error, result) {
            moreAsync(function(error, result) {
                evenMoreAsync(function(error, result) {
                    // ... I think you got it
                });
            });
        });
    });
};

Why It Happens

Nesting callbacks in JavaScript is essential for long-running code like fetching data from a web API or for database queries. The core of Node.js uses callbacks for non-blocking operations. Most of the really bad nesting occurs when a lot of callbacks need to be executed in sequence, like in the diagram below.

callback-syndrome-cause

How It Can Be Stopped

My early experience with integrating APIs in Node.js pushed me to search for a solution that improved overall readability. The first thing I came across was the Async module. It relies on simple asynchronous control flow patterns used by the JavaScript community for quite some time. This article deals with understanding those patterns and the ability to implement them on your own.

In all these examples, I referenced a couple of simple functions that emulate common asynchronous behavior:

// prints text and waits one second
function doSomethingAsync(callback) {
    console.log('doSomethingAsync: Wait for one second.');
    setTimeout(function() { callback(); }, 1000);
}

// prints text and waits half a second
function doSomethingElseAsync(callback) {
    console.log('doSomethingElseAsync: Wait for half a sec.');
    setTimeout(function() { callback(); }, 500);
}

// prints text and waits two seconds
function moreAsync(callback) {
    console.log('moreAsync: Wait for two seconds.');
    setTimeout(function() { callback(); }, 2000);
}

// prints text and waits a second and a half
function evenMoreAsync(callback) {
    console.log('evenMoreAsync: Wait for a second and a half.');
    setTimeout(function() { callback(); }, 1500);
}

// prints text
function finish() { console.log('Finished.'); }

The first thing to do is identify the common patterns used when dealing with Async code in Node.js. Mainly, you have two types of control flow (series and parallel) and each of them comes with its own quirks and flavors.

A series control flow requires that a set of callbacks are executed, one after another. The main reason for which you want to mimic sequential code is because some steps in a workflow must be executed in sequence. Normally, you would most likely implement something like this:

// executes the callbacks one after another
function series() {
    var callbackSeries = [doSomethingAsync, doSomethingElseAsync, 
                          moreAsync, evenMoreAsync];

    function next() {
        var callback = callbackSeries.shift();
        if (callback) {
            callback(next);
        }
        else {
            finish();
        }
    }
    next();
};

// run the example
series();

Parallel control flow should be used when steps in a workflow are to be executed in parallel.

// execute callbacks in parallel
function parallel() {
    var callbacksParallel = [doSomethingAsync, doSomethingElseAsync, 
                             moreAsync, evenMoreAsync];
    var executionCounter = 0;
    callbacksParallel.forEach(function(callback, index) {
        callback(function() {
            executionCounter++;
            if(executionCounter == callbacksParallel.length) {
                finish();
            }
        });
    });
}

// run the example
parallel();

Both of the control structures above can be generalized for any set of callbacks. I’ve parametrized the calls to accept an array of callbacks and a final callback. I’ve also stored the results inside the closure, so that they are sent as parameters for the final callback.

The last step requires that you put the code into our own Node.js module. This is fairly simple.

// *** index.js in casync module folder

// execute callbacks in parallel// executes the callbacks one after another
function series(callbacks, last) {
    var results = [];
    function next() {
        var callback = callbacks.shift();
        if(callback) {
            callback(function() {
                results.push(Array.prototype.slice.call(arguments));
                next();
            });
        } else {
            last(results);
        }
    }
    finish();
}

// executes the callbacks one after the other
function series(callbacks, last) {
    var results = [];
    var result_count = 0;
    callbacks.forEach(function(callback, index) {
        callback(function() {
            results[index] = Array.prototype.slice.call(arguments);
            result_count++;
            if(result_count == callbacks.length) {
                last(results);
            }
        });
    });
}

module.exports = {
    series: series,
    parallel: parallel
};

// *** test-module.js in main folder

var casync = require('./casync');

// ... declare example async functions

// run the example
casync.parallel([doSomethingAsync, doSomethingElseAsync, 
                 moreAsync, evenMoreAsync], finish);

Alternatively, you could use Deferred Objects or promises which are implemented in a couple of modules or libraries in the JavaScript community like node-promise for Node.js. Also, for nesting a maximum of three callbacks, you might get away with leaving the callbacks nested.

In Short

Because it’s relatively new, Node.js has a lot of tricks and quirks that can be exploited to obtain better results. While this improvement may not seem like such a big deal, imagine a huge code-base written entirely with nested callbacks. In the long term, this approach will drastically improve the readability and maintainability of your code.

For more cool articles on Node.js, tune in regularly. The next one will help you increase overall performance.

4 Comments

You can post comments in this post.


  • Your final example is a bit confusing, I think you have some typos and comments reversed.

    20: // executes the callbacks one after the other

    This is actually the parallel version, and the function is still called series (like the other) which I suspect is a copy/paste typo.

    T 4 years ago Reply


    • You’re right, a small copy and paste mistake. On the 3rd line of the 4th code sample there should be only one comment and on the 4th line of the 4th code sample the name of the function should be parallel.

      Hope somebody notices and fixes this 😉

      Liviu B. 3 years ago Reply


  • Great breakdown and explanation. I’m just learning node and was scratching my head at a few things, this has been very helpful.

    SAm B 3 years ago Reply


  • I want to show you my new node module, called harvests. You could try harvests and get simpler syntax compared with promises and flow libraries.https://github.com/salboaie/harvests

    salboaie 3 years ago Reply


Post A Reply