A cache is just a smaller, faster copy of data that lives in front of a bigger, slower source. That's the whole idea. Everything else is what happens when "smaller" and "faster" force tradeoffs.

the read path

flowchart LR
    A[Request] --> B{In cache?}
    B -->|hit| C[Return from cache]
    B -->|miss| D[Fetch from source]
    D --> E[Store in cache]
    E --> C

That's a cache-aside pattern — the most common shape. The application checks the cache first, and on a miss, falls back to the real source and populates the cache for next time. Simple, and it's where almost every system starts.

eviction: what gets thrown out when it's full

A cache is finite. Something has to leave to make room for something new. Which policy you pick is a bet on your access pattern.

write policies

Reads get most of the attention, but how writes interact with the cache decides your durability and consistency story.

the actual hard part: invalidation

there are only two hard things in computer science: cache invalidation and naming things

Eviction is about running out of space. Invalidation is about correctness — making sure a cached value doesn't outlive its truth. Two approaches, and most systems use both:

• -
  TTL-based: accept some staleness, bounded by a timer. Simple, but a write can still be wrong for up to the TTL window.
• -
  event-based: the source explicitly tells the cache "this key changed," usually via a pub/sub invalidation channel. Tighter consistency, more moving parts, and a new failure mode if the invalidation message itself gets dropped.

cache stampede

A popular key expires. A thousand requests arrive in the same 5ms window, all miss at once, and all fall through to the source simultaneously — which is usually the exact moment the source can least afford it.

sequenceDiagram
    participant R1 as Request 1
    participant R2 as Request 2
    participant R3 as Request 3
    participant Cache
    participant DB
    R1->>Cache: get(key)
    Cache-->>R1: miss
    R2->>Cache: get(key)
    Cache-->>R2: miss
    R3->>Cache: get(key)
    Cache-->>R3: miss
    R1->>DB: query
    R2->>DB: query
    R3->>DB: query
    Note over DB: all three hit at once

The standard fix is request coalescing (also called single-flight): the first miss takes a lock on that key and fetches from the source; every other concurrent request for the same key waits on that in-flight fetch instead of starting its own.

distributed caching

Once one cache node isn't enough, you need to decide which node owns which key — and do it in a way that doesn't reshuffle everything when a node joins or leaves.

flowchart LR
    App[App Server] --> Router{consistent hash}
    Router --> A[Cache Node A]
    Router --> B[Cache Node B]
    Router --> C[Cache Node C]

Consistent hashing is the standard answer: keys and nodes both hash onto the same ring, and a key belongs to the next node clockwise from it. Add or remove a node, and only the keys near it on the ring move — not the whole dataset.

smoothing stampedes without a lock

Request coalescing works, but it means every other request sits and waits on one unlucky requester's fetch. There's a lock-free alternative worth knowing: probabilistic early expiration.

Instead of treating TTL as a hard cliff, each read near expiry recomputes a small probability of refreshing early — roughly proportional to how close the key is to expiring and how long the last recompute took. A handful of requests refresh the value before it actually expires, spread out over time, so no single moment sees a thundering herd at all.

cache key design: versioning beats invalidating

A trick that eliminates a whole category of invalidation bugs: don't invalidate, rename. Bake a version into the key — user:42:v3 instead of user:42 — and bump the version whenever the underlying shape of that data changes.

Old-version keys are never explicitly deleted; they just stop being requested and age out via TTL or normal eviction. No invalidation message, no race between "data changed" and "cache cleared," no forgetting to clear a key on some code path. The cost is a slightly larger working set during the rollover window, which is almost always cheaper than a stale-read bug.

multi-tier caching

A single distributed cache still costs a network hop. For very hot keys, an in-process cache in front of it removes that hop entirely for the most-requested data.

flowchart LR
    App --> L1[L1: in-process cache]
    L1 -->|miss| L2[L2: distributed cache]
    L2 -->|miss| DB[(Database)]

L1 is small, per-instance, and inconsistent by nature — two app servers can disagree for a short window. That's fine for data that tolerates a few seconds of staleness, and it's why L1 TTLs are usually kept much shorter than L2's.

negative caching

A miss that goes all the way to the source isn't just for keys that exist. A lookup for something that doesn't exist — a deleted user, a typo'd ID — will keep hitting the source on every request unless you cache the absence too.

The catch: a negative cache is a bigger attack surface for amplification (someone hammering random non-existent keys to force source lookups) and it can mask a write that just hasn't landed yet. Keep negative-cache TTLs short, and treat them as a distinct, tunable setting from your normal positive-hit TTL.

is it even helping?

A cache with a bad hit ratio is just added latency and an extra thing that can be wrong. Watch these, not just uptime:

A low hit ratio is usually a sign the access pattern doesn't have the locality you assumed — worth checking before reaching for a bigger cache.