When building large scale applications, there comes a need for scaling. There are many places to start with scaling, but one place my be scaling your reads. Let's say that you have a read heavy application, like an ecommerce store or a comment system. You may want to consider caching to address this concerns. Redis is a good place to start (and to end) when solving this problems. In this article we will get started with Redis in Python.
Caching is a large topic that is outlined here: https://github.com/donnemartin/system-design-primer#cache. We will simply introduce Redis here and in later articles, we will learn to implement this practices on large scale features.
Redis is use for a lot more than caching. For example, queues are implemented in Redis using bullqueue: https://optimalbits.github.io/bull/. I highly recommend checking out this package (and the bullq UI). This is a great start to scaling out services, not just microservices. You can read about more use cases for redis here: https://redis.com/blog/5-industry-use-cases-for-redis-developers/, and I will write articles in the future about implementing this features later on.
Let's create the project as follows.
mkdir redis-example cd redis-example npm init -y touch index.py touch docker-compose.yml
For setting up Redis, I would recommend using a service for you in prod. Azure for example, has a great redis service that scales easily. However, you will want to learn redis and eventually how to scale it yourself. This will help with debugging cloud services or eventually, saving money and not using them.
We will start our intro to redis via using docker compose. Create a
docker-compose.yml file and add the following.
version: "3.2" services: redis: image: "redis:alpine" command: redis-server ports: - "6379:6379" volumes: - $PWD/redis-data:/var/lib/redis - $PWD/redis.conf:/usr/local/etc/redis/redis.conf environment: - REDIS_REPLICATION_MODE=master
Ensure you have docker installed and run
In python, the main used redis module is called
redis-py and can be installed using the follows.
pip install redis
Let's start by opening up the index.py file and importing our module. We will also connect to the redis server. By default, the module will assume we are using localhost on port 6379, which is what we setup in our docker compose file.
import redis r = redis.Redis(host='localhost', port=6379, db=0)
Next, let's run some redis commands. We will start very basic with the
As implied by the names, the set command will set a key and the get will retrieve the key.
r.set('foo', 'bar') res = r.get('foo') print(res)
Here is the full file for context.
import redis r = redis.Redis(host='localhost', port=6379, db=0) r.set('foo', 'bar') res = r.get('foo') print(res)
Often over looked in the community is the use of a UI. Many are outdated
or hard to run. One that I often use is
patrikx3/redis-ui. Although a little
clunky, it usually does what I need. I will also suggest getting use to the
redis-cli to help where GUIs cannot.
You can download the GUI here: https://github.com/patrikx3/redis-ui.
Once you have that downloaded, open up the app. Then go to Settings -> New Connection.
Type in the following to connect to local, then hit the "Add" button at the bottom.
Finally, click the bottom right, then select your localhost to connect.
Click Home and then you should see a screen like below, but with no keys on the left.