TTL and Eviction

TTL limits the age of an entry

A cache should not keep a value forever just because it was read once. TTL — time to live — defines how long an entry may live in the cache. After that time expires, it is considered expired and is no longer used as a normal hit.

key = product:42
ttl = 5 minutes

A short TTL removes old data sooner, but creates more misses and load on the primary source. A long TTL increases the chance of a hit, but also increases the time during which the cache may hold a stale copy.

Eviction limits cache size

Even entries with a TTL may not fit into available memory. Then the cache must evict some values before they expire. The eviction policy determines which entries are removed when space is limited.

For example, the cache can prefer removing entries that have not been used for a long time or data that is accessed rarely. Real systems often use approximate versions of such policies, because perfectly tracking every access also consumes resources.

TTL and eviction solve different problems

TTL answers “is this entry too old,” while eviction answers “which entry should be removed when space runs out.” One entry may be evicted long before its TTL expires, while another may be removed specifically because its lifetime ended.

Lifetime should follow the meaning of the data

A rarely changing reference table can use one TTL, while a rapidly changing price needs another. Sometimes lifetime should not be the only freshness mechanism: an important change is better reflected in the cache immediately, with TTL left as an additional safety net.