Siddhant Deval
Siddhant Deval
backend20 min read

Redis Persistence: RDB, AOF & Hybrid Mode — Durability Is Not Free

Redis persistence is a configurable durability SLA, not a binary toggle. RDB snapshots trade recoverability for fast restarts; AOF trades restart speed for per-second durability; Hybrid mode combines both and is the 2025 production default. Each mode has footguns that silently change your data-loss window — from BGSAVE doubling memory via fork() COW, to no-appendfsync-on-rewrite widening the loss window to minutes.

Redis Persistence: RDB, AOF & Hybrid Mode — Durability Is Not Free

Redis is not a cache you bolt onto a slow database — it is a data structure server with a precisely bounded contract: sub-millisecond latency, in-memory semantics, and optional persistence. The word "optional" is doing enormous work in that sentence. By default, a Redis instance ships with no persistence enabled. Every write is acknowledged with OK, stored in RAM, and permanently lost the moment the process exits — whether from a crash, an OOM kill, or a routine rolling deploy. The engineers who discover this in production are not reading documentation wrong; they are discovering that Redis's default configuration makes no durability guarantee at all, and that adding persistence is not a toggle but a spectrum of trade-offs between durability, throughput, and restart time. This article maps that spectrum precisely.

Architectural Note

This is Part 2 of the Redis Mastery series. It builds on Part 1's memory model concepts — specifically the BGSAVE fork mechanism, which requires understanding of how Redis manages its in-memory dataset. If you haven't read Part 1: Data Structures & Memory Model, review the memory model section before continuing.


1. The Default Configuration: A Crash Course in Data Loss

Before examining the persistence modes, establish a baseline: what happens with default Redis configuration.

BASH
# ❌ Default Redis configuration — what ships out of the box
# redis.conf defaults (Redis 7.2 / Valkey 7.2):

# No AOF
appendonly no

# RDB with lenient triggers (saves only after significant changes)
save 3600 1      # Save if at least 1 key changed in 3600 seconds
save 300 100     # Save if at least 100 keys changed in 300 seconds
save 60 10000    # Save if at least 10,000 keys changed in 60 seconds

# Result: a server under moderate write load takes up to 5 minutes
# (the 300-second window) to trigger a snapshot.
# Crash mid-window → up to 5 minutes of writes are gone.
BASH
# Verify persistence config on a running instance
redis-cli CONFIG GET appendonly
# → appendonly: no

redis-cli CONFIG GET save
# → save: 3600 1 300 100 60 10000

# Check when the last successful RDB snapshot completed
redis-cli LASTSAVE
# → 1725350400  (Unix timestamp — compare against current time for staleness)
Performance / Safety Warning

save "" (empty string) in redis.conf disables RDB entirely. A Redis instance with appendonly no and save "" has zero persistence — every key is lost on process exit. This is a legitimate configuration for pure caching use cases where Redis is a disposable L2 in front of a database, but it is catastrophic when engineers use the same instance to store session state, rate-limit counters, or idempotency keys.


2. RDB — Point-in-Time Snapshots

RDB (Redis Database) persistence writes the entire in-memory dataset to a binary snapshot file (dump.rdb) at configured intervals or on demand.

2.1 BGSAVE: Fork and Copy-on-Write

The core mechanism of RDB is BGSAVE:

Copy-on-Write (COW) is the OS mechanism that makes fork() fast: the child process initially shares the parent's memory pages. Pages are only physically copied when either process modifies them. This means:

  • At fork time: near-zero memory overhead (just page table duplication)
  • As the parent serves writes while the child snapshots: modified pages are copied to the child's address space
Performance / Safety Warning

BGSAVE can double your effective memory usage. If your parent process is 8GB and receives a sustained write burst during the snapshot (modifying many pages), the OS must create physical copies of those pages for the child. In the worst case — 100% write churn during snapshotting — RSS approaches 2 × used_memory. Provision Redis instances with at least 2× the working set for servers running RDB persistence, or ensure your memory headroom accounts for the snapshot fork overhead.

BASH
# Monitor BGSAVE status
redis-cli BGSAVE
# → Background saving started

redis-cli LASTSAVE     # Unix timestamp of last completed RDB
redis-cli INFO persistence | grep rdb
# rdb_changes_since_last_save: 14823
# rdb_bgsave_in_progress: 1           ← 1 means snapshot running now
# rdb_last_bgsave_status: ok
# rdb_last_bgsave_time_sec: 12        ← last snapshot took 12 seconds
# rdb_current_bgsave_time_sec: 8      ← current snapshot has run 8 seconds

# Force a foreground save (BLOCKS the server — never in production)
redis-cli SAVE
# → OK  (but all clients blocked for the save duration)

2.2 fork() Latency: The Silent Spike

fork() itself is not free. On Linux, fork() must duplicate the parent's page table — an operation proportional to the number of memory pages, not their content. For a Redis instance using 10GB of RAM with 4KB pages, the page table duplication copies approximately 2.5 million page table entries.

Dataset size Approximate fork() duration
1 GB ~5ms
10 GB ~50ms
50 GB ~250ms
100 GB ~500ms

During fork(), the Redis event loop is blocked — no commands are processed. This is the source of mysterious p99 latency spikes on large Redis instances that appear every few minutes (exactly when save triggers a BGSAVE).

BASH
# Detect fork() latency in Redis logs
redis-cli LATENCY HISTORY fork
# timestamp   duration(ms)
# 1725350400  47
# 1725350700  52

# Or via slow log
redis-cli SLOWLOG GET 10 | grep -A4 "BGSAVE"
Pro Tip & Optimization

On Linux, Transparent Huge Pages (THP) dramatically worsens fork() latency because huge 2MB pages result in larger page table entries. Redis documentation explicitly recommends disabling THP:

BASH
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Add to /etc/rc.local for persistence across reboots.

2.3 RDB Configuration Reference

INI
# redis.conf — RDB configuration

# Automatic save triggers (comment all out to disable automatic RDB)
save 3600 1
save 300 100
save 60 10000

# Filename for the RDB dump
dbfilename dump.rdb

# Directory where dump.rdb is written
dir /var/lib/redis

# If BGSAVE fails, stop accepting writes (fail loudly rather than silently lose data)
stop-writes-on-bgsave-error yes

# Compress the RDB file with LZ4 (slight CPU cost, significant size reduction)
rdbcompression yes

# Checksum the RDB file (CRC64 — detects corruption on load)
rdbchecksum yes

2.4 RDB Data Loss Window

Save trigger Worst-case data loss
save 3600 1 Up to 3600 seconds (1 hour)
save 300 100 Up to 300 seconds (5 minutes) under moderate write load
save 60 10000 Up to 60 seconds under heavy write load
Manual BGSAVE only Unbounded — depends entirely on operator discipline

RDB is appropriate for disaster recovery backups and fast restarts where bounded data loss is acceptable (e.g., a pure cache layer in front of a database). It is not appropriate as the sole persistence mechanism for data that cannot be reconstructed from another source.


3. AOF — Append-Only File

AOF (Append-Only File) persistence logs every write command received by the server to an append-only log file (appendonly.aof). On restart, Redis replays the AOF to reconstruct the dataset.

3.1 AOF Write Path

The critical question is: when does the AOF buffer reach physical disk? This is controlled by appendfsync.

3.2 appendfsync: The Durability Spectrum

INI
# redis.conf — appendfsync options

appendfsync always
# Every write command is fsync()'d to disk before returning OK to the client.
# Durability: Maximum — zero data loss on crash.
# Throughput: Severely limited — disk fsync() latency (~1-10ms) on every write.
# Throughput cost: typically 50-90% write throughput reduction vs no persistence.

appendfsync everysec
# fsync() is called by a background thread every 1 second.
# Durability: At most 1 second of data loss on crash.
# Throughput: Near-native — writes go to OS page cache synchronously; fsync runs async.
# Production default: this is the recommended setting for most workloads.

appendfsync no
# Redis never calls fsync(). The OS flushes the page cache at its own discretion (typically 30s).
# Durability: Up to ~30 seconds of data loss on crash (OS-dependent).
# Throughput: Maximum — equivalent to no persistence from Redis's perspective.
# Use case: when the OS's own flush schedule is acceptable, or when running a replica.
Crucial Requirement

appendfsync everysec does not guarantee exactly 1 second of data loss. It guarantees that Redis requests an fsync() every second. If the disk is under heavy I/O pressure, the fsync() call can block, and the background thread may fall behind. Under extreme disk saturation, data loss can exceed 1 second even with everysec. The guarantee is a best-effort upper bound, not a hard SLA.

3.3 The fsync Contract: OS Page Cache vs Physical Storage

Understanding what fsync() does is essential to understanding persistence guarantees:

Write path without fsync():
  Redis write() → OS page cache (volatile RAM) → [process crash here = data lost]
                                                → [power loss here = data lost]
                                                → disk (eventually, when OS flushes)

Write path with fsync():
  Redis write() → OS page cache → Redis fsync() → [confirmed on disk]
                                                  → [process crash: safe]
                                                  → [power loss with battery controller: safe]
                                                  → [power loss without battery: may still lose]

Modern enterprise-grade disk controllers have a write-back cache with battery backup (BBU). Even after fsync() returns, data sits in the controller's volatile DRAM until the controller commits it to the physical medium. Without a BBU, a power loss after fsync() but before the controller flush can still lose data. fsync() guarantees the OS has handed data to the controller; it does not guarantee the controller has committed it to non-volatile storage.

Setup Durability on power loss
appendfsync always + BBU controller Near-zero data loss
appendfsync always + no BBU Up to controller cache flush (~100ms typical)
appendfsync everysec + BBU Up to 1 second
appendfsync everysec + no BBU Up to 1 second + controller flush
appendfsync no Up to OS flush interval (~30 seconds)

3.4 AOF Rewrite: The no-appendfsync-on-rewrite Footgun

The AOF file grows indefinitely as commands are appended. A key set and deleted 1,000 times appears as 2,000 entries in the AOF — redundant on replay. Redis compacts the AOF via rewrite (BGREWRITEAOF), which produces a minimal AOF representing the current dataset state.

BASH
redis-cli BGREWRITEAOF
# → Background append only file rewriting started

redis-cli INFO persistence | grep aof
# aof_enabled: 1
# aof_rewrite_in_progress: 1
# aof_rewrite_scheduled: 0
# aof_last_rewrite_time_sec: 45         ← last rewrite took 45 seconds
# aof_current_rewrite_time_sec: 12      ← current rewrite has run 12 seconds
# aof_last_bgrewrite_status: ok

The footgun is a configuration option that appears to be a performance optimization:

INI
# redis.conf
no-appendfsync-on-rewrite yes
# ❌ DANGER: When AOF rewrite is in progress, Redis stops calling fsync()
# for new writes to the main AOF buffer.
# Rationale: rewrite child is already doing heavy disk I/O; adding fsync()
# from the parent would cause I/O contention.
# Consequence: during the rewrite period (which can last MINUTES for large datasets),
# your data-loss window silently expands from 1 second to the FULL REWRITE DURATION.
INI
# ✅ Safe default — keep fsync() running during rewrites
no-appendfsync-on-rewrite no
# Yes, this creates I/O contention during rewrites. Accept the throughput cost
# rather than silently expanding your data-loss window to match the rewrite duration.
Performance / Safety Warning

For a 20GB Redis instance, BGREWRITEAOF can take 3–8 minutes. With no-appendfsync-on-rewrite yes, every write during those 3–8 minutes is in the OS page cache only — not fsync()'d to disk. A crash or power failure during this window loses 3–8 minutes of writes, not 1 second. This is the most dangerous default configuration interaction in Redis persistence.

3.5 AOF Restart Time: The Pure AOF Trap

On restart, Redis replays the AOF file sequentially — re-executing every write command from the beginning of time to reconstruct the dataset.

Dataset size (compressed AOF equivalent) Approximate restart time
1 GB ~2 minutes
10 GB ~20 minutes
50 GB ~90 minutes
100 GB ~3–4 hours

An engineer who enables appendonly yes without also enabling Hybrid mode will, eventually, need to restart Redis — and will discover that "restart" means a 3-hour outage on a large dataset.

BASH
# The pure AOF restart sequence (slow path):
# 1. Redis starts
# 2. Detects appendonly.aof
# 3. Opens aof file and begins sequential replay
# 4. Re-executes every SET, HSET, ZADD, EXPIRE... one by one
# 5. 3 hours later: server ready to accept connections

4. Hybrid Mode — The 2025 Production Default

Hybrid mode (aof-use-rdb-preamble yes) combines RDB and AOF into a single file:

appendonly.aof (Hybrid format):
  ┌─────────────────────────────┬───────────────────────────────────┐
  │   RDB SNAPSHOT PREAMBLE     │   AOF DIFF TAIL                   │
  │   (binary, compact, fast)   │   (text commands since last snap) │
  │   Loaded in seconds         │   Replayed sequentially           │
  └─────────────────────────────┴───────────────────────────────────┘

4.1 Restart Sequence with Hybrid Mode

INI
# redis.conf — Hybrid mode (enable alongside AOF)
appendonly yes
aof-use-rdb-preamble yes     # Hybrid mode — the 2025 production default

# AOF rewrite triggers (when to compact the AOF)
auto-aof-rewrite-percentage 100   # Rewrite when AOF is 100% larger than the base
auto-aof-rewrite-min-size 64mb    # But only if the AOF is at least 64MB

4.2 Hybrid Mode Restart Time Comparison

Dataset size Pure RDB restart Pure AOF restart Hybrid restart
1 GB ~1s ~2 min ~1–2s
10 GB ~5s ~20 min ~5–10s
50 GB ~30s ~90 min ~30–45s
100 GB ~60s ~3–4 hours ~60–90s
Pro Tip & Optimization

Hybrid mode gives you near-RDB restart speed with near-AOF (everysec) durability. The AOF diff tail replayed on startup is only the commands received since the last rewrite — typically seconds or minutes of writes, not years of history. This is the correct production default for any Redis instance storing data that is not fully reconstructable from an upstream database.


5. Data Loss Windows: A Precise Comparison

Configuration Maximum data loss on crash Restart time (100GB) Write throughput
No persistence (save "", appendonly no) 100% — all data Instant (no data to load) Baseline
RDB only (save 60 10000) Up to 60 seconds ~60s ~Baseline
AOF appendfsync always ~0 (disk controller flush) ~3–4 hours 50–90% reduction
AOF appendfsync everysec Up to 1 second ~3–4 hours ~Baseline
AOF everysec + no-appendfsync-on-rewrite yes 1s normally; full rewrite duration during rewrite ~3–4 hours ~Baseline
Hybrid (aof-use-rdb-preamble yes) + everysec Up to 1 second ~60–90s ~Baseline

The table makes the decision clear: Hybrid mode with appendfsync everysec is strictly dominant over pure AOF for any workload where restart time matters.


6. Backup Strategies

6.1 RDB as the Backup Artifact

The RDB file is a self-contained, portable binary snapshot of the entire dataset at a point in time. It is the correct artifact for backups.

BASH
# Manual snapshot (background, non-blocking)
redis-cli BGSAVE

# Check last successful snapshot timestamp
redis-cli LASTSAVE
# → 1725350400

# Convert to human-readable date
date -d @1725350400
# → Mon Sep  3 04:00:00 UTC 2026

# The RDB file location
redis-cli CONFIG GET dir
# → /var/lib/redis

redis-cli CONFIG GET dbfilename
# → dump.rdb

# Full path: /var/lib/redis/dump.rdb

6.2 Off-Box Replication for Disaster Recovery

Copying dump.rdb to the same host it was generated on provides no protection against hardware failure. Production backup strategy:

BASH
# Scheduled off-box backup (cron example)
# Every hour: trigger snapshot, wait for completion, copy to S3

#!/bin/bash
redis-cli BGSAVE
# Wait for BGSAVE to complete
while [ "$(redis-cli INFO persistence | grep rdb_bgsave_in_progress | cut -d: -f2 | tr -d '\r')" = "1" ]; do
  sleep 1
done

TIMESTAMP=$(date +%Y%m%d_%H%M%S)
aws s3 cp /var/lib/redis/dump.rdb \
  s3://my-redis-backups/dump_${TIMESTAMP}.rdb \
  --storage-class STANDARD_IA
Crucial Requirement

Replicas are not a backup strategy. A replica that replicates a FLUSHALL command or a bad DEL loop has deleted that data from all replicas within milliseconds. Off-box, timestamped, immutable snapshots (S3 versioning, Azure Blob immutable storage) are the only protection against accidental data deletion at scale.

6.3 AOF Verification

Before relying on an AOF file for recovery, verify it is not corrupted:

BASH
# Verify AOF file integrity before using it for recovery
redis-check-aof /var/lib/redis/appendonly.aof
# → AOF analyzed: size=2147483648, ok_up_to=2147483600, diff=48
# → This AOF file is truncated (last 48 bytes are incomplete)

# Auto-fix truncated AOF (removes the incomplete trailing write)
redis-check-aof --fix /var/lib/redis/appendonly.aof
# → Truncated AOF repaired at offset 2147483600

# Verify RDB file integrity
redis-check-rdb /var/lib/redis/dump.rdb
# → [offset 0] Checking RDB file dump.rdb
# → [offset 26] AUX FIELD redis-ver = '7.2.1'
# → [offset 10485760] 100000 keys read so far
# → [info] 2500000 keys read
# → [info] Sanity check passed

Summary

Concept Rule
Default config No persistence — OK does not survive a restart. Verify with CONFIG GET appendonly and CONFIG GET save on every production instance.
RDB trade-off Fast restarts, bounded data loss window, but fork() doubles memory usage during snapshot. Provision 2× working set.
fork() latency Blocks the event loop proportional to dataset size (~50ms/10GB). Disable THP to reduce it.
AOF appendfsync always = zero loss, 50–90% throughput cost. everysec = 1-second loss, near-baseline throughput. no = OS-controlled, ~30s loss.
no-appendfsync-on-rewrite Keep this no. Setting it to yes silently widens your data-loss window to the full rewrite duration (minutes) during BGREWRITEAOF.
Pure AOF restart 3–4 hours for 100GB — turns a routine restart into a multi-hour incident. Never use pure AOF for large datasets.
Hybrid mode aof-use-rdb-preamble yes — the 2025 production default. Near-1-second durability with near-RDB restart speed.
Backups RDB files off-box with timestamps. Replicas are not backups. Verify with redis-check-aof and redis-check-rdb before recovery.

What's Next

In Part 3: Redis Replication & High Availability — Sentinel, Failover & Split-Brain, we confront the second half of the durability gap: even with persistence enabled, a primary that acknowledges a write and crashes before replicating it to any replica has lost that write permanently. We deconstruct the PSYNC partial resync protocol, replication backlog sizing, the replica-serve-stale-data stale-read footgun, Sentinel quorum election, and the min-replicas-to-write + min-replicas-max-lag split-brain prevention pairing.

Research & Synthesis Note

This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.

#Redis#Persistence#Durability#RDB#AOF
Siddhant Deval

Written by Siddhant Deval

Senior Full-Stack Engineer building high-scale architectures, browser performance engineering systems, and SaaS platforms.