Posted on by & filed under Content - Highlights and Reviews, Programming & Development.

In our previous Using Streams in Node.js post, we introduced the concept of streams and what abstractions and tools Node.js has for consuming or producing them.

Streams have many usages, ranging from abstracting file access, to streaming Hypertext Transfer Protocol (HTTP) requests and response bodies. In this post, we’re going to see how we can make two or more Node.js processes talk to each other, building a communication protocol on top of streams. Be sure to read Pedro’s Chapter 9: Reading and Writing Streams of Data in Professional Node.js: Building Javascript Based Scalable Software to learn more about Node.js streams.

Using some of these methods, you can build a mesh of Node.js processes that are loosely coupled, talking to each other as they need remote services.

TCP Server

A simple way of having two Node.js processes communicate with each other is by using Transmission Control Protocol (TCP). TCP is a protocol designed to transport data from one network device to another in a reliable and ordered way. It allows two nodes to establish a communication channel and pass a stream of ordered bytes between them. Node.js offers a streaming interface for both connected peers.

To be able to accept TCP connections, you need to create a TCP Server using the net module:

Then we can make this server listen on a certain TCP port:

Once the server is listening, it accepts connections on that port and emits a connection event:

Every time a peer connects to our server we get a connection event. In the event listener function call, we get the server socket object, which represents the two-way connection to the other connected peer. This object is a duplex stream that is writable and readable. When you write to it, it sends that data to the remote peer, and when you read to it, you read the data that the remote peer has sent you.

Here we’re simply piping the stream back to itself. All of the bytes that the server gets on this connection are sent back to the peer, effectively making an echo server.

Here is the source code of the entire server:

You can save this file with the name 01_tcp_server.js and launch it using the command line (you can access all of the source code found in this article here):

You can then test this server using a command-line TCP client like telnet or nc:

TCP Client

Now we can create a TCP client that connects to our TCP server (02_tcp_client.js):

Here we’re using the net.connect function to create a TCP connection to our server. Then we’re setting the connection encoding to UTF-8 so that we can read back strings instead of raw byte buffers. Next, we create an interval, writing the current date to the stream for every second that passes. Shortly after 5 seconds have elapsed, we stop the recurring interval and end the stream, effectively ending the connection.

In the last chunk of code we set up a readable event listener for when we receive data that was sent back from the server, thus printing it out.

You can test all of this by launching this client script while having the server running:

Reconnect

If we want to establish a long-lasting connection, we may need to reconnect to the server if the network link goes down. For that, we can use the reconnect module. Let’s then create a package.json manifest where we state that dependency:

Let’s then download and install this missing dependency:

Here is our reconnecting client (03_reconnecting_tcp_client.js):

To test the reconnecting effect of the client, you can start off by having the server down, and then launching the client. You will have to launch it before the client terminates, though, which is 10 seconds after.

Event Emitter

We’ve seen how we can connect two processes together, but all we’ve sent were unstructured string messages. TCP is a byte streaming protocol and does not mark message boundaries. For that, we can use JSON Stream, which JSON-encodes JavaScript objects to be sent down the wire and decodes them once they reach the other side.

So JSON Stream provides an object stream, allowing us to read and write JavaScript objects instead of bytes or strings. On top of the JSON Stream protocol, we’ll add the emit stream module, which transforms an object stream into an event emitter and vice-versa. Actually, and to save us some work, we’re going to use the duplex-emitter module, which makes all the stream connecting for us. Let’s then add duplex-emitter to the package.json manifest:

Now, download and install it:

Let’s then change the server to use duplex emitter to transform the peer stream into an event emitter and then send an arbitrary ping event every second (04_server_events.js):

Here the server also listens for remote pong events and prints the two arguments that they come with.

Now let’s create a client that also does a similar thing and reacts to remote ping events, sending back a pong event (05_client_events.js):

You can then launch the server:

And, on a separate command-line window, launch the client:

You should see the server printing lines like:

And the client printing lines like:

Summary

Using TCP you can establish a raw stream between two Node.js processes. On top of this stream you can use the JSONStream and the emit-stream modules to turn that stream into an event emitter. By emitting events into that stream, you are transmitting that event to the remote peer. You can also listen for the remove events using the common event emitter API.

Using these techniques, we created a distributed Node.js-based application, making it less coupled and more flexible, scalable and reliable to changes.

Safari Books Online has the content you need

Check out these Node.js books available from Safari Books Online:

Node.js is a powerful and popular new framework for writing scalable network programs using JavaScript. Professional Node.js begins with an overview of Node.js and then quickly dives into the code, core concepts, and APIs. In-depth coverage pares down the essentials to cover debugging, unit testing, and flow control so that you can start building and testing your own modules right away.
In just 24 sessions of one hour or less, Sams Teach Yourself Node.js in 24 Hours will help you master the Node.js platform and use it to build server-side applications with extraordinary speed and scalability. Using this text’s straightforward, step-by-step approach, you’ll move from basic installation, configuration, and programming all the way through real-time messaging between browser and server, testing and deployment. Every lesson and case-study application builds on what you’ve already learned, giving you a rock-solid foundation for real-world success!
This book shows you how to transfer your JavaScript skills to server side programming. With simple examples and supporting code, Node Cookbook talks you through various server side scenarios often saving you time, effort, and trouble by demonstrating best practices and showing you how to avoid security faux pas.
Node Web Development gives you an excellent starting point straight into the heart of developing server side web applications with node. You will learn, through practical examples, how to use the HTTP Server and Client objects, the Connect and Express application frameworks, the algorithms for asynchronous execution, and use both SQL and MongoDB databases.

About the author

Pedro Teixeira is a geek, programmer, freelancer, and entrepreneur. Author of some Node.js modules, the Node Tuts screencast show, some books about Node.js and overall fervent proclaimer of the Node.js creed. He is also the Co-founder and Partner of The Node Firm, and the organizer of the Lisbon JavaScript Conference.

Tags: Event Emitter, HTTP, Javascript, JSON Stream, Node.js, Streams, TCP, telnet, TLS,

Comments are closed.