Recovering Yandex Managed Service for Valkey™ from a WAL-G backup
You can restore a Yandex Managed Service for Valkey™ cluster's data on user infrastructure from a backup transferred to an S3 bucket through WAL-G.
Getting started
To complete this guide, you will need the following:
- Valkey™ and
valkey-cli: It is advisable to use the same full version as on the original server; older versions are not suitable. You can find out the version in the backup metadata. If modules were used, such as JSON, Search, or Bloom, you will need their compatible versions. - WAL-G: A binary wal-g-pg file for Valkey™/Valkey™, which you can download from the WAL-G release page
. The names of the binary file and theWALG_REDIS_*settings are also saved for Valkey™. The examples below use the WAL-G v3.0.9 UI. - Encryption key: The key exported from the bucket and used for protecting backups.
- Bucket access: Static key (ID of the static access key and the value of the secret access key) with permissions to read the bucket.
- Disk space: Free disk space big enough to store unpacked data with the following taken into account:
- Size of the database itself.
- Size of the WAL segments to recover.
The commands below run in bash as root. In the examples, the service is called valkey-server, the user and group, valkey:valkey, the configuration file, /etc/valkey/valkey.conf, and the data folder, /var/lib/valkey. If they are different in your setup, provide your values.
To decrypt the backup encryption key:
-
Download
gpg-key-<id>-0-of-1.7zfrom your bucket. -
Open the archive and enter the key you previously received via the backup request form. If you did not save this key, you will need to request a new backup export. To extract the archive, use a utility compatible with your OS:
- Windows: 7-Zip.
- macOS: Keka.
- Run the
7z x gpg-key-<id>-0-of-1.7zcommand. Enter the password when prompted by 7z.
-
Use the key obtained from the archive to decrypt your backup.
Transmitted data
| Resource | Content |
|---|---|
basebackups_005/ |
WAL-G backups: metadata and all data objects. A complete backup is required, with the folder structure saved. |
In case of a sharded cluster, backups are transferred individually for each shard. Backups of all shards from one set of backups are required, and the folders must match the original shard IDs.
Restoring a database
-
Install WAL-G. Download
wal-g-redisfor your operating system and architecture. For example, for Ubuntu 24.04 x86-64:curl -fL \ https://github.com/wal-g/wal-g/releases/latest/download/wal-g-redis-24.04-amd64 \ -o /tmp/wal-g-redis install -m 0755 /tmp/wal-g-redis /usr/local/bin/wal-g-redis wal-g-redis --version wal-g-redis backup-fetch --helpbackup-fetchmust have the-t rdb|aofand--redis-versionparameters. The WAL-G version is not associated with the Valkey™ version. -
Add and configure WAL-G:
install -d -m 0700 /etc/wal-g-restore install -m 0600 /path/to/received-private-key.pgp /etc/wal-g-restore/private.pgpCreate a
/etc/wal-g-restore/wal-g.yaml:AWS_ACCESS_KEY_ID: "<static key ID>" AWS_SECRET_ACCESS_KEY: "<static key secret>" AWS_ENDPOINT: "https://storage.yandexcloud.net" # or the endpoint of your S3 AWS_REGION: "ru-central1" AWS_S3_FORCE_PATH_STYLE: "true" WALG_S3_PREFIX: "s3://<bucket>/<folder with basebackups_005/>" WALG_PGP_KEY_PATH: "/etc/wal-g-restore/private.pgp" # WALG_PGP_KEY_PASSPHRASE: "<password>" # only if the key is protected with a password WALG_COMPRESSION_METHOD: "brotli" WALG_REDIS_DATA_PATH: "/var/lib/valkey" WALG_REDIS_APPENDONLY_PATH: "appendonlydir" WALG_REDIS_SERVER_PROCESS_NAME: "valkey-server" WALG_STREAM_RESTORE_COMMAND: "cat > /var/lib/valkey/dump.rdb"chmod 0600 /etc/wal-g-restore/wal-g.yaml export WALG_CONFIG=/etc/wal-g-restore/wal-g.yaml
WALG_S3_PREFIX points to the basebackups_005/ parent folder, not to the folder itself or a separate file. For example, for objects in s3://customer-backups/valkey-export/shard-1/basebackups_005/, specfiy s3://customer-backups/valkey-export/shard-1. The bucket name and external prefix may differ from the original ones, while the object structure inside should be saved.
For a different S3 storage, replace the endpoint, region, and, if required, AWS_S3_FORCE_PATH_STYLE.
brotli: By default, compression in MDB. If the backup was compressed using a different algorithm, specify it: this setting is required for searching the slot map with the backup-info --tag Slots command.
You do not need to import the key to GPG, as WAL-G reads it from the file. You also need the original Valkey™'s password for downloading the backup.
-
Check the access and select and a backup:
wal-g-redis --config "$WALG_CONFIG" backup-listSelect the exact name from the list and look at the metadata:
export BACKUP_NAME="aof_20261004T223158Z" wal-g-redis --config "$WALG_CONFIG" backup-info "$BACKUP_NAME"Example of shortened output:
[ { "BackupName": "aof_20261004T223158Z", "StartLocalTime": "2026-10-04T22:31:58.891678Z", "FinishLocalTime": "2026-10-04T22:34:28.238213Z", "UserData": { "backup_id": "mdbmletglr2cuu7m51t5", "shard_name": "shard1" }, "DataSize": 1257101930, "BackupSize": 209362565, "BackupType": "aof", "Version": "9.1.2", "UsedMemory": 6437938008, "UsedMemoryRss": 6463381504, "MaxDBNumber": 0 } ]From the output, you will require:
BackupName: Name for the recovery. Use it instead ofUserData.backup_id.BackupType:rdboraof.Version: Version of the original Valkey™. If there is no field, ask the party who provided the backup to also provide the version.DataSize,BackupSize: Data and backup size in bytes.UsedMemoryandUsedMemoryRssshow the original server's memory usage but do not guarantee equal usage when recovering.MaxDBNumber: The maximum number of a non-empty logical database. The number of databases in the configuration file must be greater than that number.
StartLocalTimeandFinishLocalTimestand for the backup creation time;UserDatastands for additional info on the backup and original shard. An individual field can be read throughbackup-info --tag, e.g.:wal-g-redis --config "$WALG_CONFIG" backup-info --tag Version "$BACKUP_NAME" -
Check backup integrity:
-
Save the script to the
s3_test_script.shfile.Script code
#!/usr/bin/env bash set -u if [[ $# -lt 1 || $# -gt 2 || ! -f $1 || ! -r $1 ]]; then printf 'Usage: %s /path/to/wal-g-config [storage/directory]\n' "$0" >&2 exit 2 fi config=$1 directory=${2:-} directory=${directory#/} directory=${directory%/} wal_g=${WAL_G_BIN:-wal-g} listing=$(mktemp) || exit 2 trap 'rm -f "$listing"' EXIT if ! "$wal_g" --config "$config" st ls -r >"$listing"; then printf 'ERROR: wal-g st ls -r failed\n' >&2 exit 2 fi help=$({ "$wal_g" --config "$config" st cat --help || :; } 2>&1) if [[ $help == *--decode* ]]; then decode_flag=--decode elif [[ $help == *--decrypt* ]]; then decode_flag=--decrypt else printf 'ERROR: st cat supports neither --decode nor --decrypt\n' >&2 exit 2 fi if [[ $help != *--decompress* ]]; then printf 'ERROR: st cat does not support --decompress\n' >&2 exit 2 fi red='' reset='' if [[ -t 2 ]]; then red=$'\033[31m'; reset=$'\033[0m'; fi checked=0 decoded=0 raw=0 unknown=0 failed=0 while IFS= read -r line; do path='' case $line in obj\ *) read -r _ _ _ _ _ _ path <<<"$line" ;; [0-9]*) read -r _ _ _ _ _ path <<<"$line" ;; dir\ *|type\ *|size\ *|'') continue ;; *) printf 'UNPARSED listing line: %s\n' "$line" >&2; ((unknown += 1)); continue ;; esac if [[ -z $path ]]; then printf 'UNPARSED listing line: %s\n' "$line" >&2 ((unknown += 1)) continue fi if [[ -n $directory && $path != "$directory/"* ]]; then continue fi ((checked += 1)) case $path in *.br|*.lz4|*.lzma|*.zst|*.zstd|*.lzo) ((decoded += 1)) "$wal_g" --config "$config" st cat "$path" "$decode_flag" --decompress >/dev/null ;; *.json) ((raw += 1)) "$wal_g" --config "$config" st cat "$path" >/dev/null ;; *) ((raw += 1, unknown += 1)) printf 'RAW ONLY (unknown format): %s\n' "$path" >&2 "$wal_g" --config "$config" st cat "$path" >/dev/null ;; esac if [[ $? -ne 0 ]]; then printf '%sFAIL%s %s\n' "$red" "$reset" "$path" >&2 ((failed += 1)) fi done <"$listing" printf 'Checked: %d; decoded: %d; raw: %d; failed: %d; unknown: %d\n' \ "$checked" "$decoded" "$raw" "$failed" "$unknown" if ((failed > 0)); then exit 1; fi if ((checked == 0 || unknown > 0)); then exit 2; fi
-
Run the script:
WAL_G_BIN=wal-g-redis ./s3_test_script.sh */etc/wal-g/wal-g.yaml*If required, specify a folder:
WAL_G_BIN=wal-g-redis ./s3_test_script.sh */etc/wal-g/wal-g.yaml* basebackups_005 # checks backupsHere is an example of the command output:
FAIL test/corrupted_file_2.txt.br Checked: 2; decoded: 2; raw: 0; failed: 1; unknown:
-
-
Set up Valkey™:
Install the required version from public sources
. If the original cluster used modules, install and enable them throughloadmodulebefore loading data.Check the version of the binary file run by the service and stop the server:
valkey-server --version systemctl stop valkey-server systemctl is-active valkey-serverExpected result:
inactiveand non-zero return code. Likewise, other Valkey™ processes must not use the recover folder.backup-fetchclears the AOF folder as well as the*.rdband*.aoffiles in the data folder. Use a dedicated folder or save its contents in advance if you still need them.install -d -o valkey -g valkey -m 0750 /var/lib/valkeyIn
/etc/valkey/valkey.conf, set the following:bind 127.0.0.1 port 6379 protected-mode yes requirepass "<new Valkey™ password>" dir /var/lib/valkey dbfilename dump.rdb appenddirname appendonlydir appendfilename "appendonly.aof" aof-use-rdb-preamble yes appendfsync everysec # For the AOF backup in Step 5, replace with _yes_. appendonly no # Increase it if MaxDBNumber >= 16. databases 16 cluster-enabled noStick with the
systemdand journaling parameters suitable for your setup.dirmust matchWALG_REDIS_DATA_PATH,appenddirnamemust matchWALG_REDIS_APPENDONLY_PATH, and the RDB path, that inWALG_STREAM_RESTORE_COMMAND.Users, roles, and access settings are not recovered from the data copy. Set a new password and enter it for local commands.
read -r -s -p "Password for the new Valkey™ instance: " REDISCLI_AUTH export REDISCLI_AUTHvalkey-cliuses theREDISCLI_AUTHvariable despite its name. In the new session, enter the password again. If individual ACL users are configured, add--user <name>to the commands. -
Recover the backup. Select an option based on the
BackupTypefield. Valkey™ must be stopped; run it only afterbackup-fetchcompletes successfully.
-
Option A: RDB
wal-g-redis --config "$WALG_CONFIG" backup-fetch "$BACKUP_NAME" -t rdb test -s /var/lib/valkey/dump.rdb chown -R valkey:valkey /var/lib/valkeyWhen first running it, the configuration file must have
appendonly no, so that Valkey™ loads the recovereddump.rdbinstead of the AOF.systemctl start valkey-server valkey-cli PING valkey-cli INFO persistenceWait for
PONGandloading:0. If the upload is not complete, view the log:journalctl -u valkey-server -n 100 --no-pagerIf you need AOF further on, enable it on the already uploaded server.
valkey-cli CONFIG SET appendonly yes valkey-cli INFO persistenceWait until the AOF has been created.
INFO persistenceshould have:aof_enabled:1 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_bgrewrite_status:ok aof_last_write_status:okAfter that, save the setting:
valkey-cli CONFIG REWRITEIf the command cannot write the configuration file, replace
appendonly nowithappendonly yesin it manually. Do not restart the server with enabled AOF until its files have been successfully created.To work only with RDB, leave
appendonly noand configure the snapshot schedule. -
Option B: AOF
Transfer the full version of the installed Valkey™ from
valkey-server --version. For example:export TARGET_VERSION="9.1.2" wal-g-redis --config "$WALG_CONFIG" backup-fetch "$BACKUP_NAME" \ -t aof --redis-version "$TARGET_VERSION"WAL-G will recover the multi-file AOF to
/var/lib/valkey/appendonlydir, complete withmanifestand all files listed therein. You do not need to join parts or rename them.ls -lah /var/lib/valkey/appendonlydir chown -R valkey:valkey /var/lib/valkeyIn
/etc/valkey/valkey.conf, enable AOF:appendonly yes
Start the server:
systemctl start valkey-server
valkey-cli PING
valkey-cli INFO persistence
journalctl -u valkey-server -n 100 --no-pager
Wait for PONG, loading:0, and aof_enabled:1. The log should contain no manifest read errors, upload errors, or AOF truncation errors.
-
Check the data and configure access:
valkey-cli INFO keyspaceCheck the list of logical DBs and some of the known keys: types, values, and the application's performance. For a DB different from DB 0, use
valkey-cli -n <number>. The log should show no data read errors, missing modules, insufficient memory.Keys whose TTL has expired by the time of the recovery may be missing, since recovery does not renew their TTL. Therefore, the number of keys may be different from that at the backup creation time.
Before enabling the application, configure the users and ACL,
bind, TLS, and firewall, adjusting them to your network. Use the new credentials.
Restoring a sharded cluster
Each original shard is recovered to a dedicated new master with its own original slots. First, you need to build an empty cluster with the required slot distribution, then stop it, recover the data, and start it again.
-
Select backups and get a slot map.
For each shard, prepare a WAL-G by following Steps 1-3, with its own
WALG_S3_PREFIXand an exact backup name. Select backups from one set of backups. Independent choice ofLATESTfor each shard may lead to different sets.Record the mapping of the original shards' to the new servers. For example:
Original shard ID S3 prefix BackupNameNew server <source-shard-id-1>s3://.../shard-1<backup-name-1>10.0.0.11<source-shard-id-2>s3://.../shard-2<backup-name-2>10.0.0.12<source-shard-id-3>s3://.../shard-3<backup-name-3>10.0.0.13Read each backup's
BackupTypeandVersionand extract the slot map:wal-g-redis --config "$WALG_CONFIG" backup-info --tag Slots "$BACKUP_NAME"Here is an example:
{ "source-shard-id-1": [["0", "5460"]], "source-shard-id-2": [["5461", "10922"]], "source-shard-id-3": [["10923", "16383"]] }Keys: Original shard IDs, not the new servers' node IDs and not necessarily
UserData.shard_name. Map them to their folders based on the info provided along with the backup. A regularbackup-infodoes not output the slot map.A shard can have multiple non-contiguous intervals, with both boundaries inclusive. A single slot is a pair of identical numbers. If the value equals
null, the shard has no assigned slots and the command for assigning them should be skipped.The maps of the selected backups should be in alignment, and all ranges together should cover
0–16383without gaps or overlaps.If the map cannot be read, check the prefix, backup name and integrity, and
WALG_COMPRESSION_METHOD. If there is no map or the mapping of folders to the original IDs is lost, request this info from the party who provided the backup.Warning
Do not replace the original map with uniform division and do not assign slots through automatic
valkey-cli --cluster create, as the resulting distribution may not correspond to the data in. -
Prepare empty nodes.
Install the required version of Valkey™ and modules on each server and prepare the configuration file and data folder. The folders must be new and clear of old data or an extra
nodes.conf.Replace the network and cluster mode properties in the configuration file. Example for the first server:
bind 127.0.0.1 10.0.0.11 cluster-enabled yes cluster-config-file /var/lib/valkey/nodes.conf cluster-node-timeout 15000 cluster-require-full-coverage yes cluster-announce-ip 10.0.0.11 cluster-announce-port 6379 cluster-announce-bus-port 16379 # While the empty cluster is being created, for both backup types. appendonly no databases 1On the rest of the servers, provide their own IPs. Allow the nodes to connect to each other on ports
6379and16379. SetrequirepassandREDISCLI_AUTHas in Step 4. The application should not write to the cluster until the recovery is complete.In Valkey™ 9.0+, the number of DBs in cluster mode is defined by
cluster-databases: addcluster-databases 1for a regular cluster with DB 0, and ifMaxDBNumberis greater than zero, set a number greater than that number. Valkey™ 7.2/8.x does not feature this parameter.Start the empty servers:
systemctl start valkey-server valkey-cli CLUSTER NODESRight now, each node should only have its own record without slots.
-
Assign slots and join nodes.
To join, assign each node a unique positive config epoch and assign ranges from the
Slotsmap.For the map from the example on the first server:
valkey-cli CLUSTER SET-CONFIG-EPOCH 1 valkey-cli CLUSTER ADDSLOTSRANGE 0 5460On the second server:
valkey-cli CLUSTER SET-CONFIG-EPOCH 2 valkey-cli CLUSTER ADDSLOTSRANGE 5461 10922On the third one:
valkey-cli CLUSTER SET-CONFIG-EPOCH 3 valkey-cli CLUSTER ADDSLOTSRANGE 10923 16383Use your backup's actual ranges. You can provide multiple ranges with one command, e.g.,
CLUSTER ADDSLOTSRANGE 0 100 500 700.For the first server, meet the nodes:
valkey-cli CLUSTER MEET 10.0.0.12 6379 valkey-cli CLUSTER MEET 10.0.0.13 6379Wait for the topology exchange to complete. On each server, check and save it:
valkey-cli CLUSTER NODES valkey-cli CLUSTER INFO valkey-cli CLUSTER SAVECONFIGAll nodes with the original ranges should appear,
cluster_state:ok,cluster_slots_assigned:16384, andcluster_slots_ok:16384.Stop the service on all servers:
systemctl stop valkey-server systemctl is-active valkey-serverSave the created
/var/lib/valkey/nodes.conf: they contain the new topology. Do not delete them when performing recovery and do not replace them with the original cluster's files. -
Recover the shard data.
On each stopped node, specify its
WALG_CONFIGandBACKUP_NAME. For AOF, also specifyTARGET_VERSION, which is the complete version of the installed Valkey™.Run one command based on the backup type:
# RDB: wal-g-redis --config "$WALG_CONFIG" backup-fetch "$BACKUP_NAME" -t rdb # Or AOF: wal-g-redis --config "$WALG_CONFIG" backup-fetch "$BACKUP_NAME" \ -t aof --redis-version "$TARGET_VERSION"After it completes successfully, check the recovered files as in Step 5 and correct the owner:
chown -R valkey:valkey /var/lib/valkeyIn the configuration file, set
appendonly nofor RDB orappendonly yesfor AOF. The cluster parameters andnodes.confshould be saved.After all shards' files have been recovered, run start the services:
systemctl start valkey-serverAfter RDB recovery, enable AOF in the order described in Step 5 if required, with RDB upload followed by AOF creation and saving the settings. While the rest of the nodes are being started and the topology exchange is under way, a temporary
cluster_state:failis possible. -
Check the cluster.
On each server, run the following:
valkey-cli PING valkey-cli INFO persistence valkey-cli INFO keyspace valkey-cli CLUSTER INFO valkey-cli CLUSTER NODES journalctl -u valkey-server -n 100 --no-pagerMake sure that:
- All nodes have loaded data (
loading:0) and the logs show no data or module read errors. - The state on all nodes is as follows:
cluster_state:ok,cluster_slots_assigned:16384,cluster_slots_ok:16384,cluster_slots_fail:0, andcluster_slots_pfail:0. - Each shard ends up on the required server, with its ranges matching the
Slots. There are now incomplete migrating/importing slots or nodes with thefail/fail?flags. - Known keys are read through the client with cluster support. For the CLI, use
valkey-cli -c.
- All nodes have loaded data (
After checking all that, configure access and enable the application. The client needs new credentials and network access to the addresses announced by all nodes
If recovery failed
| Symptom | What to check |
|---|---|
backup-list is empty. |
The prefix should point to the basebackups_005/ parent. Check the endpoint, bucket, and S3 permissions. |
S3 returns AccessDenied. |
Keys, list and read permissions, region, and S3 addressing parameters. |
| Decryption error. | Private PGP key and its password phrase, if it is set. |
backup-info --tag Slots does not find the object. |
Backup integrity, presence of a map in the original backup, and WALG_COMPRESSION_METHOD. |
| WAL-G states that the version is incompatible when recovering AOF. | --redis-version should have the installed Valkey™ version, not older than the original one. |
| After recovery, the RDB is empty or different data is uploaded. | Data folder, dbfilename, and the first run with appendonly no. |
| The RDB/AOF error mentions an unknown module type or command. | Whether compatible the original cluster's compatible modules have been installed and enabled through loadmodule. |
| Valkey™ cannot open files. | Owner, permissions, and matching paths in the WAL-G and Valkey™ configuration files. |
CLUSTERDOWN after launching all nodes. |
Network, announced addresses, slot map, nodes.conf, and each shard's upload. |
| There are fewer keys than expected. | Choice of selected backup, presence of all shards and DBs, expired TTLs, and upload log. |
After a download or upload error, fix the cause and repeat the recovery with the server stopped. Partially created files do not mean backup recovery has succeeded.