Redis offers a feature called pipeline that allows you to bulk send commands. This can drastically improved performance if you are running queries that can be batched together. The reason for this is that you only need to traverse the network to redis once, then Redis will run all the commands and return the result. In this article, we will learn how to use redis pipelining with Python.
For setting up Redis, I would recommend using a service for you in prod. Azure for example, has a great redis service that scales easily. However, you will want to learn redis and eventually how to scale it yourself. This will help with debugging cloud services or eventually, saving money and not using them.
We will start our intro to redis via using docker compose. Create a
docker-compose.yml file and add the following.
version: "3.2" services: redis: image: "redis:alpine" command: redis-server ports: - "6379:6379" volumes: - $PWD/redis-data:/var/lib/redis - $PWD/redis.conf:/usr/local/etc/redis/redis.conf environment: - REDIS_REPLICATION_MODE=master
Ensure you have docker installed and run
In python, the main used redis module is called
redis-py and can be installed using the follows.
pip install redis
Let's open up a new file,
index.py and go through many of
the common commands you will used with lists in redis.
pipeline = r.pipeline()
Next, we can append our commands to this pipeline. For example, here are a series of counter operations. Each of these commands will be appeneded to our pipeline and not sent.
pipeline.set("count-user1", 1) pipeline.del("count-user1-old-day") pipeline.get("count-user1") pipeline.incr("count-user1") pipeline.get("count-user1")
Finally, to send all commands, we can use the
exec function on the pipeline. Redis will execute all the commands and returns the results in an array.
pipeline.execute() print(results) // [ // [ null, 'OK' ], // [ null, 0 ], // [ null, '1' ], // [ null, 2 ], // [ null, '2' ] // ]
Below is the full code for context.
import redis r = redis.Redis(host='localhost', port=6379, db=0) pipeline = r.pipeline() pipeline.set("count-user1", 1) pipeline.del("count-user1-old-day") pipeline.get("count-user1") pipeline.incr("count-user1") pipeline.get("count-user1") results = pipeline.execute() print(results)