In this article, we’ll see how you a real case example of a Redis backup process, end-to-end.
We'll cover how to configure a Redis backup, which binaries you should use and the most important settings you need to be aware of, as well as how to store your backup remotely and how to restore it.
Let's get started!
Redis is an open source in-memory key-value store written in C.
Redis stands for Remote Dictionary Server and is used as a database, cache, queue system, and message broker.
Redis is fast because its data is stored in memory, meaning, unlike traditional databases Redis doesn't need to access the disk.
While writing this article I learned that Redis is often called a "Data Structure Server" because it provides data types that are similar to those in modern programming languages.
Some data structures that Redis provides are Strings, Lists, Sets, Hashes, and Sorted Sets.
Even though Redis could be used as your primary Database it's usually not what it is used for.
Here are the most common use cases for Redis:
There are a lot of different use cases for Redis, but these are the most common ones.
As stated above Redis is storing its data in memory. But depending on your use case Redis can copy the data on your disk.
This comes obviously handy when you have a large amount of data, and you need to be able to restore it and this is also why you might be needing to back it up.
Redis regularly dumps its data to an RDB file on the disk based on how snapshots are configured.
Redis configuration
The redis.config
file contains your Redis configuration.
The configuration file is located at /etc/redis/redis.config
and straighforward: it's a list of instructions.
You'll find a section called #### SNAPSHOTTING ####
on which you can define if Redis should snapshot its data to the dis and how often it should do it.
save 60 1000
This configuration will make Redis dump the dataset to the disk every 60 seconds if at least 1000 keys are changed.
Learn more about Redis Configuration →
Redis also works with AOF (Append-Only File) which is a way to store the data on the disk by logging all write operations received by the server.
AOS won't be covered in this article but it is worth knowing it exists, especially when you'll need to backup and restore your data.
Making a Redis backup is pretty easy. You'll need to make a fresh copy of the RDB file, compress it and save it somewhere safe.
Redis offers 2 methods to "force" a snapshot:
SAVE
: This will force a snapshot to be taken synchronously.BGSAVE
: This will force a snapshot to be taken asynchronously.The easiest way is to use the SAVE
method but it will block all other operations until the snapshot is done.
Using BGSAVE
will make the server continue to accept commands and will not block other operations but you'll have to figure out when the snapshot is this one is asynchronous.
So, if you want to make a backup you'll need to do the following:
You'll need to know where your snapshot file will be generated using the redis-cli command described below.
The default location of your Redis config file is /etc/redis/redis.config
.
You can also use this command to find the location of your Redis config file: redis-cli config get dir
.
You can find the configuration file here https://redis.io/topics/config.
SAVE
commandThis method will work synchronously to make a snapshot of your Redis database.
Just ssh into your server and run the following command:
Log in to the database command line interface:
redis-cli
You might have to authenticate to your database:
auth YOUR_PASSWORD_HERE
SAVE
The output will be something like this:
OK
(1.23s)
You can then exit the redis-cli by typing exit
.
At this stage, the RDB
file will be saved in /var/lib/redis/
and will be named dump.rdb
.
BGSAVE
commandUsing the asynchronous dump function, you'll need to make sure you are aware of the end of the process.
One way to do it is to use inotifywait
which will notify you when a change to the dump file is made.
As stated in Redis documentation, it's safe to copy the RDB file even if used by your running server:
Redis is very data backup friendly since you can copy RDB files while the database is running: the RDB is never modified once produced, and while it gets produced it uses a temporary name and is renamed into its final destination atomically using rename(2) only when the new snapshot is complete.
This means that copying the RDB file is completely safe while the server is running. This is what we suggest:
- Create a cron job in your server creating hourly snapshots of the RDB file in one directory, and daily snapshots in a different directory.
- Every time the cron script runs, make sure to call the find command to make sure too old snapshots are deleted: for instance you can take hourly snapshots for the latest 48 hours, and daily snapshots for one or two months. Make sure to name the snapshots with date and time information.
- At least one time every day make sure to transfer an RDB snapshot outside your data center or at least outside the physical machine running your Redis instance.
We'll create a script that will create our dump file, then upload to Amazon S3.
cd ~
mkdir scripts
cd scripts
nano redis_backup.sh
Copy and paste the script below to it:
#!/bin/bash
rdb_file="/FOLDER_TO_YOUR_REDIS_DUMP_FILE/redis/dump.rdb"
redis_cli="/usr/bin/redis-cli"
DIR=`date +%d-%m-%y`
DEST=~/redis_backups/$DIR
mkdir $DEST
echo save| $redis_cli
exit 1
Append to redis_backup.sh
the following:
BUCKET_NAME="YOUR_S3_BUCKET_NAME"
aws s3 cp $rdb_file s3://YOUR_S3_BUCKET/redis_backups/ && echo "Backup copied to S3"
First, let's CHMOD the script to make it executable:
chmod +x ~/scripts/db_sync.sh
Then create a cron job to run the script every day at midnight:
crontab -e
0 0 * * * ~/scripts/redis_backup.sh # take a backup every midnight
Now that you've made a backup, we'll see how to restore it from a .rdb
file.
We recommend you first try this on a fresh Redis server
AOF stands for Append-Only File, which will instruct Redis to log all operations in a .aof
file.
Since we're restoring a backup, we need to disable AOF before restoring the data as we don't want Redis to log all these operations.
Open your configuration file (redis.config
) and make sure appendonly
is set to no
.
appendonly no
Before being able to replace the dump.rdb
file, you'll need to stop the Redis server.
sudo service redis-server stop
Prior to restoring the database, you can rename the existing dump.rdb file in order to have a restore point in case something goes wrong.
sudo cp /home/redis/dump.rdb /home/redis/dump.rdb.bak
You can then copy the backup rdb file as follows:
sudo cp /redis_backups/20220810/dump.rdb /home/redis/dump.rdb
And finally make sure to apply the right permissions to the dump.rdb file:
sudo chmod 660 /home/redis/dump.rdb
sudo service redis-server start
We've seen how to back up your Redis database and restore it, and we've seen how to automate the process.
SimpleBackups will save you a lot of time setting up scripts, ensuring they run without problems, all without code or maintenance.
It will alert you when things go wrong, and allows you to store your backups on many cloud storage services like Google, DigitalOcean, Wasabi, Dropbox, and more…
Schedule your Redis backup without code or maintenance.
Back up your Redis database →
Free 7-day trial. No credit card required.
Have a question? Need help getting started?
Get in touch via chat or at [email protected]