Redis Bitmap in Python

Intro

Bitmaps are a pattern in redis, not actualy a data type as they are just using string, that can help save space when using redis. For example, you can store 4 billion users subscribed to a notification using only 512mb of memory with bitmaps. In this article, we will learn how to use Redis bitmaps in Python.

Setting up Redis

For setting up Redis, I would recommend using a service for you in prod. Azure for example, has a great redis service that scales easily. However, you will want to learn redis and eventually how to scale it yourself. This will help with debugging cloud services or eventually, saving money and not using them.

We will start our intro to redis via using docker compose. Create a docker-compose.yml file and add the following.

version: "3.2"
services:
  redis:
    image: "redis:alpine"
    command: redis-server
    ports:
      - "6379:6379"
    volumes:
      - $PWD/redis-data:/var/lib/redis
      - $PWD/redis.conf:/usr/local/etc/redis/redis.conf
    environment:
      - REDIS_REPLICATION_MODE=master

Ensure you have docker installed and run

docker-compose up

Installing Redis Modules

In python, the main used redis module is called redis-py and can be installed using the follows.

pip install redis

Writing the Code

Let's open up a new file, index.py and go through many of the common commands you will used with lists in redis.

Creating Bitmaps in Redis

To create a bit map, we can use the setbit command. We provide the command with a name for the key, and offset or poistion in the bitmpa, and the value which must be 0 or 1. For example, we have want to track eight different users and say that user 7 is subscribed, we can use the following.

# Set Bit
r.setbit("users-subscribed", 7, 0)

Also note, that all unset bits will be set to 0. So the command above can represent 8 users where user 7 is on or 1 and other values will default to 0 for off.

We can retrive the bit information in two ways, the first is using the getbit command, but we can also use the basic get command as bitmaps are simply strings underneath.

# Get Bit
result = r.getbit("users-subscribed", 7)
print(result) # 0

result2 = r.get("users-subscribed")
print(result2) # 0

For each of the examples below, I will use the following template to run all the commands. Here is my full index file. We will just replace the commands each time.

import redis

r = redis.Redis(host='localhost', port=6379, db=0)

# Set Bit
r.setbit("users-subscribed", 7, 1)

# Get Bit
result = r.getbit("users-subscribed", 7)
print(result) # 0

Bit Position

If we want to find the first bit turned off or on, we can use the bitpos function. The following will tell use the first bit that is on, 1, in our key.

# Get Bit
result = r.bitpos("users-subscribed", 1)
print(result) # 7

Bit Operation

The bitop command will allow you to run bit operations on bit maps. For example, you can use And, Or, XOR and NOT on two bitmap keys.

# Set Bit
r.setbit("users-subscribed-2", 6, 1)

# Bit Operation
result = r.bitop("AND", "newkey", "users-subscribed", "users-subscribed-2")
print(result) # 7

result2 = r.get("newkey")
print(result2) # 7

Bit Field

The bitfield command allows you to perform multiple operations on a bitmap, which is helpful for perfoamnce boosting. The syntax is a bit confusing but first we pass the key to the command, then a series of commands to operation.

The series of commands must have the formmat (command, format, offset, value). For example, below we increment the 5bit signed integer offset 100 by 4 and then we get offset 2 in an unsigned interger format.

# Bit Field
result = r.bitfield("bit-map", "incrby", "i5", 100, 4, "get", u4, 2)
print(result) # 7

Bit Count

The final command we will look at is bitcount. This allows us to count all the number of set bits, value of 1, in a string. We can also bass a start and end value to count a range.

# Bit Count
result = r.bitcount("users-subscribed")
print(result) # 7

result2 = r.bitcount("users-subscribed", 0, 4)
print(result2) # 7