Posted on by & filed under Content - Highlights and Reviews, Web Development.

In the Functional Programming: Taming Asynchronous Code article we looked at sequential asynchronous code and how to tame it using functional programming. We also introduced how to use asynchronous function composition using composite and partial function application using “ap” to reduce complexity. We will be re-using those techniques in this article, so I recommend that you read it before reading further.

Parallel code

We will now look at parallel code and how we can use asynchronous iterators to manage their complexity.

An example of parallel code would be the following:

The above code reads all of the files in the /tmp folder and then calls next with an array of the file content.

What it’s actually doing is two actions: one is reading the directory and the second is mapping all of the file names to the actual file content.

It’s not technically mapping the array because fs.readFile is asynchronous and returns in an arbitrary order. We are using files.push(file), which will just turn files into an array of the files in random order.

Parallel map

With parallel map we can represent our code as a composition of reading the directory and mapping the files to their content:

In this case iterators.map takes a list (either an object or array) and calls an iterator function (in this case fs.readFile) for each value in the list (each fileName in the array).

iterators assumes your iterator function applies to the standard asynchronous syntax of callback last, and that the callback is called with (err, data). When we write code that complies with this standard node pattern it simplifies a lot of the complexity.

iterators.map itself will take the result value from the callback and replace the initial value in the list with it. iterators.map also keeps a running count of whether all the asynchronous functions are finished. When they are finished, iterators.map will call the third parameter – your done callback – with the mapped list.

For more details read the implementation of iterators.map.

This article has provided some insight to using parallel code and how you can use asynchronous iterators to manage them.

Safari Books Online has the content you need

Take advantage of these node.js resources in Safari Books Online:

Node: Up and Running (Rough Cuts) introduces you to Node, the new web development framework written in JavaScript. You’ll learn hands-on how Node makes life easier for experienced JavaScript developers: not only can you work on the front end and back end in the same language, you’ll also have more flexibility in choosing how to divide application logic between client and server.
Read What Is Node? to get up to speed on Node.js with this concise overview.
Node for Front-End Developers shows you how to use this popular JavaScript framework to create simple server applications, communicate with the client, build dynamic pages, work with data, and tackle other tasks.
In Sams Teach Yourself node.js in 24 Hours, expert web developer George Ornbo guides readers through every step of creating custom server-side solutions with Node.js.

About this author

Jake Verbaten is a JavaScript evangelist and open source hacker. He’s published a ton of modules on npm and build multiple production applications using node.js. To learn more about Jake, check out his github profile.

Tags: asynchronous iterators, Functional programming, Node.js, parallel code, parallel map,

Comments are closed.