Node Redis Pubsub

Intro

Redis provides a Pub/Sub api that scales well and allows for quick real time connections. Real time apps are very popular, so redis can help you solve these business problems. If you are familiar with the popular socket.io, you will see in their docs an option to use Redis as an adapter to scale. In this article, we will learn how to use PubSub in Redis with Nodejs.

Use Cases

• Sending DB updates to the client
• Displaying real time edits (like Google Docs)
• Real time chat

Just to give a more clear view, let's say you have the following setup with some microservices because you have scaled to many users

• One Server to write to the Database
• Two servers running web apis (like Express.js).
  • These severs both have socket.io to connect to a Front End (like React or Vue)
• A static website hosted on Netlfiy or AWS with just your React or Vue code
  • This website has a socket appliation to talk to Express

In this scenario, you can have a number of users connect to either API server app which allows you to handle more connections than a single server. Then, you can have redis pub/sub send quick communication between the three servers, the API servers and the DB. This allows you to quickly add more backend servers to scale while keeping connections to the client.

Setting up Redis

For setting up Redis, I would recommend using a service for you in prod. Azure for example, has a great redis service that scales easily. However, you will want to learn redis and eventually how to scale it yourself. This will help with debugging cloud services or eventually, saving money and not using them.

We will start our intro to redis via using docker compose. Create a docker-compose.yml file and add the following.

version: "3.2"
services:
  redis:
    image: "redis:alpine"
    command: redis-server
    ports:
      - "6379:6379"
    volumes:
      - $PWD/redis-data:/var/lib/redis
      - $PWD/redis.conf:/usr/local/etc/redis/redis.conf
    environment:
      - REDIS_REPLICATION_MODE=master

Ensure you have docker installed and run

docker-compose up

Installing Redis Modules

There are two modules I see often used in nodejs. I will tend towards ioredis as it has built in support for promises and many other features in redis.

npm install ioredis

Creating the Subscriber

We will need two methods for subscribing to a channel. The first is the subscribe method where we can pass one or more channels to listen to. Next, we will use the on method with the message parmeter to list for message events comming to our Redis connection.

const Redis = require("ioredis")
const redis = new Redis()

redis.subscribe("channel-1", "channel-2", (err, count) => {
  if (err) {
    console.error("Failed to subscribe", err
    return
  }

  console.log(`Subscribed to ${count} channels.`)
});

redis.on("message", (channel, message) => {
  console.log(`Received ${message} from ${channel}`)
})

Creating the Publisher

For publishing, we can use the publish method. The first parameter is the channel we would like to send to, the second parameter is the message we want to send.

const Redis = require("ioredis")
const redis = new Redis()

const message = {
    id: 1,
    text: 'db updated'
}

redis.publish("channel-1", JSON.stringify(message));

Pattern Matching Publish and Subscribe

Redis also provides a way to pattern match when using the publish and subsribe feature. Let's say we had the following publisher sending events for multiple users.

const Redis = require("ioredis")
const redis = new Redis()

const message = {
    id: 1,
    text: 'db updated'
}

redis.publish("user-1", JSON.stringify(message));
redis.publish("user-2", JSON.stringify(message));

We can subscribed to all user events using the psubscribe method which stands for pattern subsrcibe. Here we subscribe to user-* which means all channels with the prefix user-

const Redis = require("ioredis")
const redis = new Redis()

redis.psubscribe("user-*", (err, count) => {
  if (err) {
    console.error("Failed to subscribe", err
    return
  }

  console.log(`Subscribed to ${count} channels.`)
});

redis.on("message", (channel, message) => {
  console.log(`Received ${message} from ${channel}`)
})