Improve handling of cache stampeding / thundering herd

[Vectorial1024]

Currently Laravel does not really have anything effective against cache stampeding / thundering herd. When cache entry expires, we would have multiple Laravel processes trying to repopulate the cache entry. This could be a problem.

One way to deal with this is an opt-in lottery-based "read failures". By a low-probability lottery (opt-in), the Cache layer pretends that a cached value doesn't exist when the entry still has not expired yet. This allows a limited number of Laravel processes to update the cached entry before it expires. This should mitigate the above-mentioned stampede problem.

Perhaps Laravel can introduce this feature?

[trtajim]

I like that this is proposed as an opt-in feature.

One question I have is whether the behavior would be consistent across different cache drivers. Drivers like Redis can support atomic operations and distributed locks, while file or database caches have different capabilities.

Do you envision this as a driver-agnostic feature implemented entirely in Laravel, or would each cache driver be expected to optimize the behavior differently?

[Vectorial1024]

To make the proposed feature work with as much cache drivers as possible, it should be driver-agonistic. The lottery-based "pretend read fail" is exactly this. For single reads (get), we could just do the lottery and "early return pretend read fail". For multiple reads (getMany), we can do the lottery and select only a subset of the requested key for actual cache reading, thus completing "early return pretend read fail".

The multiple reads approach can be as simple as this (pseudocode):

receive keyList
shuffle keyList
do lottery (obtains float between 0 and 1)
partition keyList:
- the first n * lottery items are "early return pretend read fail" and gets null
- the remaining items are sent to the next step
use existing cache-specific getMany implementation
join the getMany results with the null results

The above pseudocode preserves existing driver-specific getMany optimizations.

The point of this request is that this is a pattern which is applicable to all drivers (as long as getMany is supported, which is essentially "all drivers"), and perhaps some may also find this pattern useful (aside from me).

[Vectorial1024]

One alternative solution would be to configure random expiry ttl per each item, but that will be very messy on the writing side, especially when wanting to use putMany. Notice how putMany has uniform/shared ttl, which is not what we want.

[jonhassall]

I frequently need a beautiful solution to cache stampedes. I did submit a pull request myself but was denied.

[vshulcz]

This landed in 11.x as Cache::flexible() (the Laracon 2024 batch, September 2024), and it targets exactly this problem:

$value = Cache::flexible('key', [300, 3600], fn () => expensiveComputation());

Two TTLs: fresh for 300 seconds, usable for an hour. Inside the fresh window it returns immediately. Past it, the caller still gets the stale value and the recompute is deferred, so nobody waits and the herd never forms. Repository::flexible():

if (($created + $this->getSeconds($ttl[0])) > Carbon::now()->getTimestamp()) {
    return $value;
}

$refresh = function () use ($key, $ttl, $callback, $lock, $created) {
    $this->store->lock(
        "illuminate:cache:flexible:lock:{$this->itemKey($key)}",
        $lock['seconds'] ?? 0,
        $lock['owner'] ?? null,
    )->get(function () use ($key, $callback, $created, $ttl) {
        if ($created !== $this->get(self::FLEXIBLE_CREATED_KEY_PREFIX.$key)) {
            return;
        }
        ...

Two guards there, not one. The lock keeps concurrent refreshers out, and the $created comparison means a process that took the lock after someone else already refreshed sees a changed creation stamp and returns without recomputing.

On the driver question upthread: it is driver-agnostic in the sense that matters, because locking is not Redis-only. ArrayStore, FileStore, DatabaseStore, MemcachedStore, DynamoDbStore and RedisStore all provide locks, mostly through the shared HasCacheLock trait, so flexible() behaves the same across them.

Compared with the probabilistic early-miss idea, the tradeoff is that a lottery still makes some unlucky request pay the full recompute cost while everyone else is served from cache, and you tune a probability to control how often that happens. flexible() removes the blocking request entirely and needs no tuning beyond the two TTLs. The case it doesn't cover is a key that goes cold past the second TTL, where the next request does recompute synchronously.

[Vectorial1024]

The issue is that this does not allow many.

[vshulcz]

Both fair, and neither is covered by flexible().

The cold path really is unlocked. It's the first branch, before any of the lock machinery:

if (in_array(null, [$value, $created], true)) {
    return tap(value($callback), fn ($value) => $this->putMany([
        $key => $value,
        self::FLEXIBLE_CREATED_KEY_PREFIX.$key => Carbon::now()->getTimestamp(),
    ], $ttl[1]));
}

So what it buys you is turning a stampede at every TTL boundary into one at first population, or after an eviction. For a key where that still hurts, priming it on deploy or taking a Cache::lock around the first fill yourself is the only route today.

And there's no bulk variant, flexible() is single key only. That's the part of the original proposal that has nothing covering it: putMany already forces a shared TTL on the write side, and on the read side you'd want the staleness check per key. Worth calling out explicitly in the request, because "add stampede protection" reads as already-done, while "add it for many" doesn't.

[jonhassall]

The stale window for Cache::flexible has a cache lock around 'expensiveComputation', but a cold/fully expired cache does not.

[wasifahmed85]

Laravel's Cache::flexible() handles stale-while-revalidate but doesn't lock, so concurrent misses can still hammer the backend. Fix: add locking inside flexible() (via Cache::lock()) so only one process regenerates while others get stale data or block briefly — making stampede protection built-in rather than requiring manual Cache::lock() wrapping.

[gogoSpace]

This discussion, particularly the missing bulk equivalent of flexible(), helped shape a package I've been working on: Laravel Bulk Cache.

It adds rememberMany() and flexibleMany() with batch loading for missing items. Its Redis driver also coordinates overlapping loads across processes; that currently requires single-primary Redis.

I'm the author. It's MIT-licensed and still in beta, so the API may change. Before making a stable release, I'd like to find out whether it's useful in other applications and what I've overlooked.

Would this fit the bulk-cache workloads you had in mind? Examples where it falls short or adds unnecessary complexity would be useful too. If anyone tries it locally or in staging, I'd appreciate hearing how it goes. Feedback on the API alone would also help.