System Design

When the Upstream Is Down: Serving Stale Data on Purpose

Your page shouldn’t die, or lie, when a third-party API does. Keep serving the last known good data, show how old it is, and recover on its own.

·
11 min read
·
AI-assisted

AI tooling disclosure

Generative AI materially contributed to this article.

Human review confirmed by the author.

When the Upstream Is Down: Serving Stale Data on Purpose

Your app shows exchange rates, shipping estimates, weather, or a partner's product catalog, and all of it comes from someone else's API. Then that API has a bad afternoon. What does your app do?

Most apps pick one of three behaviors, usually by accident:

  1. Show an error page. The upstream is down, so you are down. Your uptime is now the product of every dependency's uptime.

  2. Show nothing, or zeros. The call fails, the code catches the exception, and the page renders an empty chart, a blank list, or a total of 0.00. This is worse than an error, because it looks like an answer.

  3. Serve old data silently. The page stays up, but nobody can tell the numbers are three hours old. People make decisions on them.

The better behavior sits between those: keep serving the last data you know was good, say plainly how old it is, and recover on its own when the upstream returns. This post builds that, including the part most implementations skip: the visible freshness indicator.

The goals

  • Stay up. A dependency outage becomes a degraded page, not a dead one.

  • Be honest. Anyone looking at the data can tell it's old, and by how much.

  • Heal automatically. When the upstream recovers, fresh data replaces stale data without a deploy or a manual cache flush.

  • Know about it. You should find out about an outage from your own alerts, not from a user.

Fresh, stale, and too old

A cached value goes through three stages, controlled by two timers:

stateDiagram-v2
    [*] --> Fresh: good response stored
    Fresh --> Stale: age passes fresh_for
    Stale --> Fresh: refresh succeeds
    Stale --> TooOld: age passes max_stale
    TooOld --> Fresh: refresh succeeds
  • Fresh (fresh_for, say 5 minutes): serve it with no network call.

  • Stale (up to max_stale, say 6 hours): try to refresh. If the refresh fails, serve the old value and mark it stale.

  • Too old (beyond max_stale): the data is no longer trustworthy enough to show, because "last known good" eventually becomes "last known wrong." Return an honest error.

The mistake that defeats this whole idea is using an ordinary cache TTL. When a Redis or memcached key expires, the value is deleted, so the moment you need a fallback, it's gone. The fix is to separate how long data counts as fresh from how long you keep it around. Store the value together with fetched_at, never evict it on a timer, and work out freshness when you read it.

The request flow

sequenceDiagram
    participant U as Browser
    participant A as Your app
    participant S as Last known good store
    participant P as Upstream API
    U->>A: GET /rates
    A->>S: read value and fetched_at
    alt value is fresh
        S-->>A: value
        A-->>U: 200 with data, no banner
    else value is stale or missing
        A->>P: fetch with a 2 s timeout
        alt upstream answers
            P-->>A: 200 with data
            A->>S: store value, fetched_at = now
            A-->>U: 200 with fresh data
        else upstream fails or times out
            A-->>U: 200 with old data, a stale flag, and its age
        end
    end

If the store has nothing at all (a cold start with the upstream down), the app has nothing to fall back on and returns a 503. We'll deal with that case later.

Implementation

This version uses only the Python standard library and SQLite as the store. SQLite is a good fit because the fallback has to survive restarts, and a file on disk does that without another service. The code needs Python 3.10 or newer (for the str | None annotation) and SQLite 3.24 or newer (for the upsert).

import json
import logging
import random
import sqlite3
import time
import urllib.error
import urllib.request
from dataclasses import dataclass

log = logging.getLogger("stale")

# Persistent on purpose: a fallback that lives only in memory disappears on every restart.
db = sqlite3.connect("stale_cache.db", isolation_level=None, timeout=10)
db.row_factory = sqlite3.Row
db.execute("PRAGMA journal_mode = WAL")
db.execute("""
CREATE TABLE IF NOT EXISTS last_known_good (
    key        TEXT PRIMARY KEY,
    value      TEXT NOT NULL,              -- JSON of the last GOOD response
    fetched_at REAL NOT NULL,              -- when that response arrived; only a success changes it
    failures   INTEGER NOT NULL DEFAULT 0, -- consecutive failed refreshes
    retry_at   REAL NOT NULL DEFAULT 0,    -- leave a failing upstream alone until then
    last_error TEXT
)""")

class Unavailable(Exception):
    """No usable data: nothing stored, or what we have is older than max_stale."""

class Gone(Exception):
    """The upstream says the thing does not exist. That is an answer, not an outage."""

@dataclass
class Result:
    value: object
    fetched_at: float          # unix time of the last good response
    age: float                 # seconds since fetched_at
    stale: bool                # True when we could not refresh
    error: str | None = None   # why the refresh failed, if it did

def fetch_json(url, timeout=2.0):
    """Example fetcher. Short timeout: a fallback is useless if we wait 30 s before using it."""
    try:
        with urllib.request.urlopen(url, timeout=timeout) as resp:
            return json.load(resp)
    except urllib.error.HTTPError as e:
        if e.code in (404, 410):
            raise Gone(url) from e
        raise  # 5xx, 429, 401, 403 ... treat as an outage and fall back

def get(key, fetch, fresh_for=300, max_stale=86400, validate=None):
    now = time.time()
    row = db.execute("SELECT * FROM last_known_good WHERE key = ?", (key,)).fetchone()

    if row and now - row["fetched_at"] <= fresh_for:           # 1. fresh: no network call at all
        return Result(json.loads(row["value"]), row["fetched_at"], now - row["fetched_at"], False)

    usable = row is not None and now - row["fetched_at"] <= max_stale
    error = None
    if usable and now < row["retry_at"]:                       # 2. upstream is in back-off: don't hit it
        error = row["last_error"]
    else:                                                      # 3. try to refresh
        try:
            value = fetch()
            if validate and not validate(value):
                raise ValueError("response failed validation")
        except Gone:
            raise
        except Exception as exc:
            error = f"{type(exc).__name__}: {exc}"
            if row:
                failures = row["failures"] + 1
                delay = min(5 * 2 ** (failures - 1), 300) * random.uniform(0.5, 1.5)
                db.execute(
                    "UPDATE last_known_good SET failures = ?, retry_at = ?, last_error = ? WHERE key = ?",
                    (failures, now + delay, error, key),
                )
        else:
            db.execute(
                """INSERT INTO last_known_good (key, value, fetched_at) VALUES (?, ?, ?)
                   ON CONFLICT(key) DO UPDATE SET value = excluded.value, fetched_at = excluded.fetched_at,
                                                  failures = 0, retry_at = 0, last_error = NULL""",
                (key, json.dumps(value), now),
            )
            return Result(value, now, 0.0, False)

    # 4. the refresh failed or was skipped: serve the last known good, if it is not too old
    if usable:
        age = now - row["fetched_at"]
        log.warning("serving stale key=%s age=%ds error=%s", key, age, error)
        return Result(json.loads(row["value"]), row["fetched_at"], age, True, error)
    raise Unavailable(f"{key}: no usable data (upstream error: {error})")

The decisions that matter:

  • fetched_at only changes when a good response arrives. This is the single most important line in the design. If a failed refresh, a stale read, or a cache layer that "touches" entries when it serves them reset the timestamp, old data would look fresh and your freshness indicator would lie.

  • A response must be valid before it replaces the last known good. validate catches the case where the upstream returns 200 OK with an error page, an empty object, or a changed schema. A bad response that overwrites good data is how a small outage becomes a long one.

  • The timeout is short. If the upstream hangs and your HTTP client waits 30 seconds, every user waits 30 seconds and then gets the stale data. Pick a timeout that matches what a user will tolerate, usually 1 to 3 seconds.

  • Back-off protects both sides. After a failure the code stops calling the upstream for a growing interval (5 s, 10 s, 20 s, up to 5 minutes, with jitter). Users get the stale value instantly instead of paying for a timeout on every request, the failing service gets breathing room, and recovery doesn't arrive as a synchronized stampede.

  • Back-off only applies while there's something usable to serve. Once the stored value is older than max_stale there's nothing to fall back on, so the code keeps trying and recovers on the first success.

  • Gone is not an outage. A 404 or 410 means the upstream is healthy and says the thing doesn't exist. Serving a stale copy of something that was deleted is a bug, so that case is raised to the caller instead.

  • Not every error is the same kind of error. Timeouts, connection failures, 5xx, and 429 are outages and trigger the fallback. A 401 or 403 means your credentials are broken. Falling back is still the right thing for the user, but it should also page someone, because the fallback will quietly run out of time otherwise.

Using it takes a few lines:

import functools

fetch_rates = functools.partial(fetch_json, "https://api.example.com/v1/rates", 2.0)

result = get(
    "rates:usd",
    fetch_rates,
    fresh_for=300,           # 5 minutes counts as fresh
    max_stale=6 * 3600,      # beyond 6 hours, we'd rather show an error
    validate=lambda v: isinstance(v, dict) and "rates" in v,
)

Make the staleness visible

Returning old data is only half of graceful degradation. The other half is telling people. Three places to do it:

In the API response. Give clients the facts instead of making them guess:

from datetime import datetime, timezone

def api_rates():
    try:
        r = get("rates:usd", fetch_rates, fresh_for=300, max_stale=6 * 3600,
                validate=lambda v: isinstance(v, dict) and "rates" in v)
    except Unavailable:
        return {"error": "rates_unavailable"}, 503, {}

    headers = {"X-Data-Stale": "true"} if r.stale else {}
    body = {
        "data": r.value,
        "meta": {
            "fetched_at": datetime.fromtimestamp(r.fetched_at, timezone.utc).isoformat(timespec="seconds"),
            "age_seconds": int(r.age),
            "stale": r.stale,
        },
    }
    return body, 200, headers

In the UI. A small, calm banner beats a scary one, and it should say the age in words people understand:

def humanize_age(seconds):
    s = int(seconds)
    if s < 10:
        return "just now"
    if s < 60:
        return f"{s} seconds ago"
    for unit, size in (("day", 86400), ("hour", 3600), ("minute", 60)):
        if s >= size:
            n = s // size
            return f"{n} {unit}{'' if n == 1 else 's'} ago"
{% if result.stale %}
<div class="freshness freshness--stale" role="status">
  Showing data from {{ result.age | humanize_age }}. Live data is temporarily unavailable.
</div>
{% endif %}

This assumes you pass result to the template and register humanize_age as a template filter.

Some guidelines for the indicator:

  • Show nothing when data is fresh. A permanent "updated 2 minutes ago" label is fine for dashboards, but a banner should mean something is wrong.

  • Show the age, not just the word "stale." "Showing data from 14 minutes ago" lets the reader decide whether it still matters. "Stale data" doesn't.

  • Put the exact timestamp in a tooltip or title attribute for people who need it.

  • Escalate with age. A neutral note at 15 minutes, a warning color at 2 hours, and disabled actions past whatever threshold is dangerous.

  • Compute age on the server. Send age_seconds, and let the browser add the time elapsed since the page loaded (using something like performance.now()) to keep the label current. Never subtract fetched_at from the browser's clock, because user devices are routinely wrong by minutes or more.

Not all data degrades the same way

"Serve stale" is the right call for a lot of data and the wrong call for some. Decide per kind of data, and write the decision down:

Kind of data

Examples

When it's stale

Informational display

Weather, news lists, leaderboards, catalog browsing

Serve it with a freshness label. A long max_stale is fine.

Decision support

Stock levels shown to staff, delivery estimates, status dashboards

Serve it, label it prominently, and use a short max_stale.

Transactional

Price at checkout, account balance, inventory reservation

Don't guess. Disable the action, or ask the user to confirm the price they're seeing.

Security

Permissions, revoked tokens, kill switches

Don't serve stale beyond a very short window. Default to the safe answer (deny). Stale permissions can grant access that was already taken away.

The rule of thumb: the more irreversible the action the data triggers, the less stale it may be.

The cold-start problem

A last-known-good store only helps if it contains something. Three situations leave it empty:

  • First deploy, or a new key. There's no good data yet and the upstream is already down.

  • A fresh container with no persistent disk. The cache dies with the container.

  • A new region or replica that hasn't served the request before.

What helps:

  • Persist the store (a database table, a mounted volume) instead of keeping it in process memory.

  • Warm it on deploy. A startup step or a scheduled job that refreshes the important keys means the fallback exists before it's needed.

  • Ship a baked-in snapshot for the data that matters most: a JSON file committed to the repo, loaded when the store is empty. Label it honestly, for example "Built-in data from 1 November," so even the worst case degrades gracefully.

Know when you're serving stale

Graceful degradation has a failure mode of its own: it works so well that nobody notices the upstream has been down for four days. Treat "serving stale" as an event worth watching.

  • Log it. The code above emits one warning per stale response with the key, the age, and the error. Count these.

  • Expose the age. A health check that lists the oldest data you hold shows trouble quickly:

SELECT key,
       CAST(strftime('%s', 'now') AS INTEGER) - CAST(fetched_at AS INTEGER) AS age_seconds,
       failures,
       last_error
FROM last_known_good
ORDER BY fetched_at
LIMIT 10;
  • Alert on age, not on single failures. One failed call is noise. "Rates haven't refreshed for 30 minutes" is a signal. Set the alert threshold a little above fresh_for, and a second, louder one well before max_stale, so you hear about it before users see an error.

Test the outage on purpose

Don't wait for the upstream to fail to find out whether this works. Run a fake upstream locally and break it:

  1. Serve a small JSON endpoint and fetch it once. Confirm that the value is stored.

  2. Make it return 503. Confirm the app serves the old value, flags it stale, and that age keeps growing rather than resetting.

  3. Make it hang for longer than your timeout. Confirm responses still arrive in about one timeout, not the full hang.

  4. Call it repeatedly while it's down. Confirm the upstream is not contacted during back-off.

  5. Bring it back. Confirm the next refresh succeeds, the stale flag clears, and failures returns to zero.

  6. Return a 200 with a garbage body. Confirm the last known good value is untouched.

  7. Return 404. Confirm you get Gone, not the old data.

  8. Let the data age past max_stale. Confirm you get a clean error, not ancient data.

Those eight checks cover the real failure modes: hard failure, slow failure, flapping, bad data, and permanent absence.

Pitfalls

  • urlopen's timeout isn't a deadline. In many HTTP clients, including the standard library's, the timeout applies to each blocking socket operation. A server that trickles bytes slowly can keep you waiting well beyond it. If you need a hard cap on total time, use a client that supports an overall deadline, or enforce one yourself.

  • Caching per user without keying by user. If the data differs by user or tenant, the key must include that. Otherwise the fallback shows one customer's data to another.

  • A thundering herd when the entry expires. If 200 requests arrive while a refresh is due, 200 requests hit the upstream. Use a per-key lock within one process, or a lease row across processes, so one caller refreshes while the rest serve the stale value.

A quicker option at the HTTP layer

If your data flows through an HTTP cache or CDN, two Cache-Control extensions from RFC 5861 do part of this for you: stale-while-revalidate (serve stale while refreshing in the background) and stale-if-error (serve stale when the origin returns errors). Support varies between CDNs and proxies, so check the documentation for the one you use.

They're a good first step, but they don't replace the application-level version. They can't distinguish a 404 from an outage in the way your code can, they don't validate payloads, and they don't give your UI the age of the data. Many setups use both: headers at the edge, and a last-known-good store in the app for the data that matters.

You can also serve stale while refreshing in the background inside your own app: return the old value immediately and kick off the refresh on a worker thread or a queue. It lowers latency further, at the cost of more moving parts, so start with the synchronous version above.

Takeaways

  • Separate freshness (fresh_for) from retention (max_stale). A TTL that deletes data deletes your fallback.

  • Store the last good response with its fetched_at, and change that timestamp only on success.

  • Validate responses before they overwrite good data, and use short timeouts plus back-off with jitter.

  • Show the age, calmly, and escalate as data gets older. Compute it on the server.

  • Decide per kind of data: serve stale for display, be careful for decisions, fail closed for money and security.

  • Persist and warm the store so the fallback exists on a cold start.

  • Alert on how long you've been stale, and rehearse the outage before it happens.

Comments(0)

Join the discussion by logging into your account.

No comments yet. Be the first to comment!

Anshu Pathak

Passionate developer sharing knowledge about modern web technologies and best practices.

Subscribe to Anshu Pathak's Newsletter

Direct email dispatches when new stories are published. Zero algorithms.