In NodeJS, stream module provides the capability to work with streams. Even if you haven’t used stream module explicitly there are a lots of underlying functionality in NodeJS applications which use streams. “Streams” is an easy concept, but it may sound very complex if you are unfamiliar with it. Therefore, I thought of describing a few key concepts in NodeJS streams in visuals so that it can be easily understood.

Information flows from one place to another place as a stream of bits. For example, this happens when two peers talking to each other via a network, or even when your application is communicating with the disk or a peripheral device. When such an I/O operation happens, the information is read from a device and is flowed towards an application, or vice versa.

However, it’s possible that one end of the above transaction is slower than the other for various reasons. Therefore, some of the data might need to be “buffered” in-between, while the receiver-end is ready to accept more data.
Have a look at the following picture where two faucets of different sizes are connected via a tank. The rate water flows from upstream is higher than the rate downstream can consume. Therefore, the tank has to temporarily store (“buffer”) the excess water while the downstream slowly consumes.

This is the fundamental idea of streams in NodeJS. stream module provides functionality to implement this behaviour when working with streaming data. There are two basic types of streams provided by NodeJS.

They are,

• Readable Streams
• Writable Streams

However, there are two additional types of streams which are hybrids of Readable and Writable streams, and serve special purposes.

• Duplex Streams
• Transform Streams

Let’s get into more details, and try to visualise each of these.

Readable Stream

A readable stream can be used to read data from an underlying source such as a file descriptor. Data can be stored in a Buffer within the readable stream if the application consumes slower than the operating system reads from the source.

A few of the most common readable streams in NodeJS are process.stdin, fs.createReadStream and IncomingMessage object in an HTTP server.

Writable Stream

A Writable stream is used to write data from an application to a certain destination. To prevent data loss or overwhelming of the destination in case the destination is slower than the writing application, the data may be stored in an internal Buffer.

Duplex Stream

As I mentioned earlier, a duplex stream is a hybrid of a Readable stream and a Writable stream. An application connected to a duplex stream can both read from and write to the duplex stream. The most common example of a duplex stream is net.Socket. In a duplex stream, read and write parts are independent and have their own buffers.

Transform Stream

A transform stream is an even special hybrid, where the Readable part is connected to the Writable part in some way. A common example would be a crypto stream created using Cipher class. In this case, the application writes the plain data into the stream and reads encrypted data from the same stream.