What Is a Cache? Hits, Misses, Freshness, and Invalidation

Key takeaways

• A cache stores reusable data or results to avoid repeating more expensive work.
• Cache keys must distinguish requests whose answers can differ.
• Expiration, invalidation, and eviction solve different lifecycle problems.
• Measure source load and failure behavior alongside hit rate and response time.

A cache stores data or computed results so a system can reuse them instead of repeating more expensive work.

Caches appear in processors, browsers, databases, and applications. They may live in memory or on disk; being a cache describes their role, not one storage technology.

Imagine a shop repeatedly displaying a product description. Reading a reusable copy can avoid another database query. Reusing a price or stock count, however, introduces questions about when the copy stops being acceptable.

Cache Hits and Misses

In a cache-aside design, the application first looks for the requested value. A usable stored value produces a hit. If no usable value exists, the application loads the source and may store the result for subsequent requests.

Microsoft’s cache-aside guide describes this pattern and its consistency limitations.

For the shop, repeated views of one popular product can benefit substantially. Requests for unique, never-revisited results may gain little while still paying for cache lookup and storage.

Caching works because some work is reusable, not simply because another storage component was added.

The Key Defines Which Requests Share an Answer

A key such as product:42 looks sufficient until the response depends on language, currency, customer pricing, or tenant.

If two requests can legitimately receive different answers, the cache design must account for those differences.

For example, a public description might use a product-and-language key, while customer-specific prices need a separate approach. Reusing one customer’s response for another can become a data exposure rather than a performance issue.

Do not treat a cache hit as authorization. The caller must still be allowed to access the returned information.

Freshness Depends on the Operation

A slightly old product description may be tolerable. A checkout decision based on an old inventory count may not be.

Define what staleness means for each use. The display page and the final purchase operation need not use identical data policies.

A time to live, or TTL, limits how long an entry remains eligible under that cache’s expiry policy. It does not prove the source is current, that a replica has caught up, or that another cache layer has refreshed.

A short TTL reduces some stale windows but can increase source traffic.

Expiration, Invalidation, and Eviction Differ

Mechanism	Reason an entry stops being usable
Expiration	Its configured lifetime has passed
Invalidation	The application knows the value should no longer be reused
Eviction	Capacity policy removes it to make room
Versioned key	New requests address a newer representation

After a product update, the application might invalidate the old entry so a subsequent request loads the new value.

Concurrent reads and writes complicate this. A reader can fetch an old value before the update and populate the cache afterward. A delete operation alone does not establish strong consistency.

Choose coordination or versioning appropriate to the required freshness guarantee.

Local and Shared Caches Have Different Costs

An in-process cache is quick to access, but each application instance has its own copy. Updating one does not automatically refresh the others.

A shared cache lets instances reuse entries, but adds a network dependency and does not eliminate stale-data races.

The choice belongs in the system design: consider data size, reuse, consistency, latency, and what happens when the cache cannot be reached.

For either design, the source should remain capable of handling the intended miss path.

HTTP Caching Has Its Own Rules

Browsers and shared HTTP caches use response metadata to decide whether and how a response can be reused.

For example, Cache-Control: no-cache permits storage but requires validation before reuse. no-store directs caches not to store the response. These directives are not synonyms.

An entity tag, or ETag, can support a conditional request. If the representation is unchanged, a 304 response allows reuse without downloading the body again.

The HTTP caching guide explains freshness, validation, and private versus shared caches. Those mechanisms should not be confused with an application’s own database-result TTL.

Prepare for a Cold Cache

If a popular entry expires, many concurrent requests can all query the source. A cache restart can produce the same effect across many keys.

Coalescing concurrent loads, staggering expiration, or carefully allowing stale responses during refresh can reduce the spike where the data permits it.

Fallback also needs limits. Sending every request directly to a database during a cache outage can overload the database and turn one failure into two.

Measure the Outcome

Track hit rate, miss latency, evictions, refresh failures, source load, and customer-visible freshness problems.

A high hit rate can conceal stale or incorrectly shared results. A lower hit rate may still be worthwhile if it avoids a few exceptionally expensive computations.

For the shop, the target is responsive browsing without incorrect prices, leaked personalization, or overloaded checkout dependencies. Caching succeeds when it removes repeated work while preserving those requirements.

A fast answer is useful only when it belongs to the request and is fresh enough for the operation.