缓存预热

概念概述

缓存预热（Cache Preloading and Warming）指的是：缓存预热在流量到来前填充条目，使用户不必在部署或淘汰后承受冷缓存延迟，但需限速以免压垮回源。

从更严格的定义看，Cache preloading is the proactive population of a cache with expected keys, either at startup or in the background, so that the first real request finds a warm entry. 该概念在工程实践中的价值，在于它能帮助团队用统一的语言描述一个具体的质量属性或设计约束，从而在评审与协作中减少歧义。

为什么需要它

It is used after deployments, cache flushes, and known traffic spikes such as scheduled events. 它与 Distributed Caching、Write-Behind Caching、Cache Partitioning via Consistent Hashing 等概念经常一起出现，用于解决相近的工程问题。理解它的适用场景，有助于在多种备选方案之间做出有依据的取舍，而不是凭直觉选择技术方案。

常见误解

• preloading guarantees every needed key will be present
• loading keys in an arbitrary order is as effective as by expected demand

需要避免的反模式

• preloading an unbounded key space at startup
• warming with synthetic requests that never release their connections

实践要点

1. Loading the most probable keys first maximises the benefit, so warming should be driven by observed traffic rather than by iterating the entire data set.
2. Aggressive preloading can overload the origin, so it should be rate limited and staggered, especially when many instances warm simultaneously after a deploy.
3. The approach works best together with a sensible time to live, because a warm entry that expires before it is used delivers no benefit.

参考来源

https://en.wikipedia.org/wiki/Cache_warming

代码示例

import { LRUCache } from 'lru-cache';

const cache = new LRUCache({
  max: 500,
  ttl: 60_000,
  staleTtl: 300_000,
  allowStale: true,
  fetchMethod: async (key) => {
    const res = await fetch('https://api.example.com/rates/' + key);
    if (!res.ok) throw new Error('upstream ' + res.status);
    return res.json();
  },
});

const inflight = new Map();

async function getRate(currency) {
  const value = await cache.fetch(currency);
  return value;
}

async function revalidate(key, loader) {
  if (inflight.has(key)) return inflight.get(key);

  const task = loader()
    .then((fresh) => { cache.set(key, fresh); return fresh; })
    .finally(() => { inflight.delete(key); });

  inflight.set(key, task);
  return task;
}

const usd = await getRate('usd');
const again = await getRate('usd');
console.log('rate:', usd.rate, 'second read:', again.rate);

import asyncio

import httpx
from httpx_cache import CacheControlTransport

client = httpx.AsyncClient(
    transport=CacheControlTransport(
        cache=httpx_cache.FileCache(cache_dir='.http_cache'),
        cacheable_methods=('GET',),
        cacheable_status_codes=(200, 301, 404),
    )
)


async def fetch_product(product_id: int) -> dict | None:
    try:
        resp = await client.get(f'https://api.example.com/products/{product_id}')
        resp.raise_for_status()
        return resp.json()
    except httpx.HTTPStatusError:
        return None


async def warm_cache(ids: list[int]) -> int:
    results = await asyncio.gather(
        *(fetch_product(pid) for pid in ids),
        return_exceptions=True,
    )
    return sum(1 for r in results if not isinstance(r, Exception))


async def main() -> None:
    await warm_cache(list(range(10)))

    # 断网场景：文件缓存仍可回放此前成功的响应
    product = await fetch_product(3)
    print('served from cache offline:', product is not None)
    await client.aclose()


asyncio.run(main())