Redis Architecture: Data Types, Persistence and Scaling

Redis can return a value in microseconds, but fast memory alone does not make a production system safe. A session store, rate limiter, leaderboard, and cache have different persistence, eviction, replication, and failure requirements. Redis architecture is the set of choices that makes those requirements explicit.

This guide follows the cache eviction policies article. It explains Redis as an in-memory data platform: its data structures, persistence choices, replication behavior, and cluster topology. For application-managed cache reads and writes, see cache-aside.

Table of Contents

What Redis Is and Is Not

Redis is an in-memory data store that exposes atomic operations on rich data structures. It is useful when the operation belongs near the data: increment a counter, add an item to a set, read a ranked range, claim a short-lived lock, or serve a cached representation. It is not automatically a replacement for a relational database. Its replication is asynchronous by default, memory is finite, and a failover can lose recently acknowledged writes depending on persistence and acknowledgement settings.

Use the data role to choose the architecture. A disposable product cache may accept eviction and reload from a database. A rate limiter needs atomic increments and explicit expiry. A session needs a bounded loss policy. A payment ledger needs an authoritative transactional store. Treating every Redis key as equally critical is how a performance optimization becomes a correctness incident.

Data Types Follow Access Patterns

Choose the smallest Redis type that directly expresses the operation:

Data type choice reduces application races. A HINCRBY or sorted-set score update is atomic at the Redis command level; reading a JSON blob, modifying it in application memory, and writing it back is not. Still define a key namespace, TTL, maximum value size, and authorization boundary for every key. A key design that omits tenant scope can leak data just as surely as a bad SQL query. Streams are persistent only to the degree that the deployment’s persistence and retention configuration preserves them; they are not an automatic durable replacement for a replicated event log.

Single Instance Architecture

A single Redis server holds an in-memory dataset and processes commands. It is appropriate for local development, an easily rebuilt cache, or a small workload where its availability is not a product dependency. The basic request path is simple:

flowchart TD
    A[Application] --> B[Redis client pool]
    B --> C[Redis primary]
    C --> D[In-memory data structures]
    C --> E[Optional persistence files]

The simplicity has a limit: one process has finite CPU, network throughput, and memory. A restart clears data unless persistence is configured. A node failure makes every key unavailable. Before scaling, measure command latency, used memory, key cardinality, slow commands, eviction rate, and connection count. Many incidents blamed on Redis scale are really unbounded values, missing TTLs, hot keys, or a client retry storm.

Persistence: RDB and AOF

Redis persistence writes an in-memory dataset to durable storage. RDB snapshots capture point-in-time state at configured intervals. They are compact and recover quickly, but a crash can lose writes since the latest snapshot. Append-only file (AOF) persistence records write commands, reducing the loss window at the cost of more write and disk activity. Redis can rewrite the AOF in the background to compact its history.

For a feature that can tolerate roughly one second of acknowledged-write loss but needs faster recovery than per-write fsync, a common AOF baseline is:

appendonly yes
appendfsync everysec

appendfsync always requests a disk sync for each write and raises write latency. appendfsync everysec is the usual throughput and durability compromise. appendfsync no leaves scheduling to the operating system and creates a wider, less predictable loss window. Test these settings on the actual storage class: an acknowledgement does not prove that a write survives every process, host, or availability-zone failure. When both AOF and RDB are enabled, Redis restores from AOF because it is normally the more complete record.

Neither setting is a magic durability checkbox. Define the recovery point objective first: how much acknowledged data can this feature lose? Then test the actual configured fsync behavior, storage durability, restart time, and replica promotion path. If loss must be near zero, a primary database or a dedicated replicated log may be a better acknowledgement boundary than a cache-oriented Redis deployment.

Replication and Failover

A primary can stream changes to one or more replicas. Replicas improve read capacity for suitable workloads and provide a candidate to promote after a primary failure. Because replication may lag, a replica can return an older value. Keep read-after-write and correctness-sensitive reads on the primary or use a documented consistency mechanism.

flowchart TD
    A[Application writes] --> P[Primary shard]
    P --> R1[Replica one]
    P --> R2[Replica two]
    A -->|Lag-tolerant reads| R1
    A -->|Read after write| P

Failover needs a failure detector, a promotion decision, client endpoint updates, and protection against two primaries accepting writes. Managed Redis services and Sentinel-style deployments automate parts of this, but application clients still need reconnect behavior, bounded timeouts, and a safe degraded mode. The same read freshness trade-off appears in database replication, but Redis roles and loss tolerances are often different.

Sentinel and Cluster Solve Different Problems

Redis Sentinel provides monitoring and automatic failover for a non-clustered primary with replicas. It does not shard the dataset. Redis Cluster partitions keys across primary shards and also coordinates shard failover. Choose Sentinel when one primary can hold and serve the entire dataset but you need high availability. Choose Cluster when the dataset or aggregate command throughput exceeds one primary, then also design for shard-aware operations. A managed service may hide operational details, but the application must still know which topology it uses and how clients discover a promoted primary.

Redis Cluster and Sharding

When one primary cannot hold the dataset or command throughput, Redis Cluster partitions keys across multiple primary shards and typically assigns replicas to each shard. A client routes a key to its owning shard. Adding a node provides potential capacity, but open-source Redis Cluster does not automatically redistribute slots just because a node joined: an operator or managed service must plan and execute resharding. This scales aggregate throughput, not one hot key. A celebrity counter or one giant sorted set can remain a bottleneck even in a large cluster.

Plan multi-key operations carefully. Keys on different shards may not be eligible for the same atomic operation. Hash tags can deliberately co-locate related keys, but overusing them creates hot shards. Add replicas for availability and read capacity, then observe resharding, replication lag, memory skew, slot balance, and the client behavior during topology changes.

Production Design Rules

• Treat Redis as disposable cache storage unless its persistence, replication, and recovery objectives have been deliberately tested.
• Set memory limits and an eviction policy only for keys that are safe to lose; isolate critical state.
• Put TTLs on ephemeral keys and add jitter where synchronized expiry would create a miss storm.
• Use atomic commands and scripts for state transitions; never rely on a client-side read-modify-write loop for a shared invariant.
• Bound command timeouts, connection pools, and retry budgets so a Redis outage cannot exhaust application resources.
• Monitor p95 command latency, memory fragmentation, evictions, replica lag, rejected writes, hot keys, connections, and persistence health.

Make Multi-Command Changes Atomic

Redis commands are atomic one at a time, but a client-side GET, change, SET sequence can still race. Use a data-structure command when it represents the operation, or a Lua script when several commands must form one transition. This fixed-window rate-limit script increments a counter and sets its expiry only on the first request in the window:

local count = redis.call("INCR", KEYS[1])

if count == 1 then
  redis.call("EXPIRE", KEYS[1], tonumber(ARGV[1]))
end

if count > tonumber(ARGV[2]) then
  return 0
end

return 1

Run it with one key so it is safe to route in Redis Cluster:

EVAL "local count = redis.call('INCR', KEYS[1]); if count == 1 then redis.call('EXPIRE', KEYS[1], tonumber(ARGV[1])); end; if count > tonumber(ARGV[2]) then return 0; end; return 1" 1 rate:{tenant-42}:2026-09-14T13:00 60 100

The script runs atomically relative to other Redis commands on its primary, so another client cannot observe the counter before the first-hit expiry is set. That atomicity does not make a workflow atomic across shards, network calls, or failover. It also uses a fixed window, which can allow a burst at two adjacent minute boundaries; use a sliding-window or token-bucket design when that behavior is unacceptable.

Treat Distributed Locks as a Correctness Boundary

A short-lived Redis lock can coordinate best-effort work, but its lease can expire while the original holder is paused or partitioned. Another worker can then acquire the lock and both can act. Do not use a Redis lock alone to protect irreversible side effects such as payments or inventory writes. Prefer an authoritative transaction or use fencing tokens that the downstream resource rejects when stale.

For fleet-level cache topology, review the distributed cache system design guide. For write acknowledgement trade-offs, use write-through versus write-back caching rather than assuming persistence makes every Redis write equivalent to a database commit.

Interview Questions

1. When is Redis the source of truth?

Only when the product deliberately accepts Redis’s configured persistence, replication, recovery, and operational guarantees as its durable record. For ordinary caching, it is not the source of truth: a miss, eviction, or restart must safely reload from another store.

2. What is the RDB versus AOF trade-off?

RDB takes periodic compact snapshots and can lose changes after the latest snapshot. AOF records write operations and can reduce that window, but increases disk and recovery considerations. I choose from the recovery point objective, not from a generic claim that one is safer.

3. Does a Redis replica provide strongly consistent reads?

Not by default. Replication can lag, so I route read-after-write and correctness-sensitive reads to the primary, then use replicas only where the endpoint has an explicit freshness budget.

4. What does Redis Cluster solve?

It partitions a dataset and command load across shards and improves availability with replicas. It does not split one hot key, remove network latency, or make cross-shard operations automatically atomic. I still need data distribution and hot-key design.

5. How should an application behave when Redis is unavailable?

For a disposable cache, I use short timeouts and bypass to a protected source of truth when it has capacity. For rate limiting or session state, I define a product-specific fail-open or fail-closed policy. I do not let unbounded retries turn a Redis outage into an application outage.

Conclusion

Redis earns its speed by keeping active data in memory and exposing atomic data-structure operations. Production quality comes from the surrounding choices: appropriate data types, explicit TTL and eviction rules, tested persistence, lag-aware replication, and shard-aware scaling. Decide what a successful Redis write means before making it part of a critical path.

References

1. Redis Data Types
2. Redis Persistence
3. Redis Replication
4. Redis: High Availability with Sentinel

YouTube Videos

1. Replication and Clustering in Redis