Redis Pipeline in Nodejs

Intro

Redis offers a feature called pipeline that allows you to bulk send commands. This can drastically improved performance if you are running queries that can be batched together. The reason for this is that you only need to traverse the network to redis once, then Redis will run all the commands and return the result. In this artcile, we will learn how to use redis pipelining with Nodejs.

Setting up Redis

For setting up Redis, I would recommend using a service for you in prod. Azure for example, has a great redis service that scales easily. However, you will want to learn redis and eventually how to scale it yourself. This will help with debugging cloud services or eventually, saving money and not using them.

We will start our intro to redis via using docker compose. Create a docker-compose.yml file and add the following.

version: "3.2"
services:
  redis:
    image: "redis:alpine"
    command: redis-server
    ports:
      - "6379:6379"
    volumes:
      - $PWD/redis-data:/var/lib/redis
      - $PWD/redis.conf:/usr/local/etc/redis/redis.conf
    environment:
      - REDIS_REPLICATION_MODE=master

Ensure you have docker installed and run

docker-compose up

Installing Redis Modules

There are two modules I see often used in nodejs. I will tend towards ioredis as it has built in support for promises and many other features in redis.

npm install ioredis

Writing the Code

Let's open up a new file, index.js. To use pipelining, we first a pipeline instance.

const pipeline = redis.pipeline()

Next, we can append our commands to this pipeline. For example, here are a series of counter operations. Each of these commands will be appeneded to our pipeline and not sent.

pipeline.set("count-user1", 1)
pipeline.del("count-user1-old-day")
pipeline.get("count-user1")
pipeline.incr("count-user1")
pipeline.get("count-user1")

Finally, to send all commands, we can use the exec function on the pipeline. Redis will execute all the commands and returns the results in an array.

const results = await pipeline.exec()
console.log(results)

// [
//   [ null, 'OK' ],
//   [ null, 0 ],
//   [ null, '1' ],
//   [ null, 2 ],
//   [ null, '2' ]
// ]

Below is the full code for context.

const Redis = require("ioredis")

const redis = new Redis()

async function main() {
    const pipeline = redis.pipeline()
    pipeline.set("count-user1", 1)
    pipeline.del("count-user1-old-day")
    pipeline.get("count-user1")
    pipeline.incr("count-user1")
    pipeline.get("count-user1")
    const results = await pipeline.exec()
    console.log(results)
}

(async () => {
    await main()
})()