
Your app shows exchange rates, shipping estimates, weather, or a partner's product catalog, and all of it comes from someone else's API. Then that API has a bad afternoon. What does your app do?
Most apps pick one of three behaviors, usually by accident:
Show an error page. The upstream is down, so you are down. Your uptime is now the product of every dependency's uptime.
Show nothing, or zeros. The call fails, the code catches the exception, and the page renders an empty chart, a blank list, or a total of
0.00. This is worse than an error, because it looks like an answer.Serve old data silently. The page stays up, but nobody can tell the numbers are three hours old. People make decisions on them.
The better behavior sits between those: keep serving the last data you know was good, say plainly how old it is, and recover on its own when the upstream returns. This post builds that, including the part most implementations skip: the visible freshness indicator.
The goals
Stay up. A dependency outage becomes a degraded page, not a dead one.
Be honest. Anyone looking at the data can tell it's old, and by how much.
Heal automatically. When the upstream recovers, fresh data replaces stale data without a deploy or a manual cache flush.
Know about it. You should find out about an outage from your own alerts, not from a user.
Fresh, stale, and too old
A cached value goes through three stages, controlled by two timers:
stateDiagram-v2
[*] --> Fresh: good response stored
Fresh --> Stale: age passes fresh_for
Stale --> Fresh: refresh succeeds
Stale --> TooOld: age passes max_stale
TooOld --> Fresh: refresh succeedsFresh (
fresh_for, say 5 minutes): serve it with no network call.Stale (up to
max_stale, say 6 hours): try to refresh. If the refresh fails, serve the old value and mark it stale.Too old (beyond
max_stale): the data is no longer trustworthy enough to show, because "last known good" eventually becomes "last known wrong." Return an honest error.
The mistake that defeats this whole idea is using an ordinary cache TTL. When a Redis or memcached key expires, the value is deleted, so the moment you need a fallback, it's gone. The fix is to separate how long data counts as fresh from how long you keep it around. Store the value together with fetched_at, never evict it on a timer, and work out freshness when you read it.
The request flow
sequenceDiagram
participant U as Browser
participant A as Your app
participant S as Last known good store
participant P as Upstream API
U->>A: GET /rates
A->>S: read value and fetched_at
alt value is fresh
S-->>A: value
A-->>U: 200 with data, no banner
else value is stale or missing
A->>P: fetch with a 2 s timeout
alt upstream answers
P-->>A: 200 with data
A->>S: store value, fetched_at = now
A-->>U: 200 with fresh data
else upstream fails or times out
A-->>U: 200 with old data, a stale flag, and its age
end
endIf the store has nothing at all (a cold start with the upstream down), the app has nothing to fall back on and returns a 503. We'll deal with that case later.
Implementation
This version uses only the Python standard library and SQLite as the store. SQLite is a good fit because the fallback has to survive restarts, and a file on disk does that without another service. The code needs Python 3.10 or newer (for the str | None annotation) and SQLite 3.24 or newer (for the upsert).
import json
import logging
import random
import sqlite3
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
log = logging.getLogger("stale")
# Persistent on purpose: a fallback that lives only in memory disappears on every restart.
db = sqlite3.connect("stale_cache.db", isolation_level=None, timeout=10)
db.row_factory = sqlite3.Row
db.execute("PRAGMA journal_mode = WAL")
db.execute("""
CREATE TABLE IF NOT EXISTS last_known_good (
key TEXT PRIMARY KEY,
value TEXT NOT NULL, -- JSON of the last GOOD response
fetched_at REAL NOT NULL, -- when that response arrived; only a success changes it
failures INTEGER NOT NULL DEFAULT 0, -- consecutive failed refreshes
retry_at REAL NOT NULL DEFAULT 0, -- leave a failing upstream alone until then
last_error TEXT
)""")
class Unavailable(Exception):
"""No usable data: nothing stored, or what we have is older than max_stale."""
class Gone(Exception):
"""The upstream says the thing does not exist. That is an answer, not an outage."""
@dataclass
class Result:
value: object
fetched_at: float # unix time of the last good response
age: float # seconds since fetched_at
stale: bool # True when we could not refresh
error: str | None = None # why the refresh failed, if it did
def fetch_json(url, timeout=2.0):
"""Example fetcher. Short timeout: a fallback is useless if we wait 30 s before using it."""
try:
with urllib.request.urlopen(url, timeout=timeout) as resp:
return json.load(resp)
except urllib.error.HTTPError as e:
if e.code in (404, 410):
raise Gone(url) from e
raise # 5xx, 429, 401, 403 ... treat as an outage and fall back
def get(key, fetch, fresh_for=300, max_stale=86400, validate=None):
now = time.time()
row = db.execute("SELECT * FROM last_known_good WHERE key = ?", (key,)).fetchone()
if row and now - row["fetched_at"] <= fresh_for: # 1. fresh: no network call at all
return Result(json.loads(row["value"]), row["fetched_at"], now - row["fetched_at"], False)
usable = row is not None and now - row["fetched_at"] <= max_stale
error = None
if usable and now < row["retry_at"]: # 2. upstream is in back-off: don't hit it
error = row["last_error"]
else: # 3. try to refresh
try:
value = fetch()
if validate and not validate(value):
raise ValueError("response failed validation")
except Gone:
raise
except Exception as exc:
error = f"{type(exc).__name__}: {exc}"
if row:
failures = row["failures"] + 1
delay = min(5 * 2 ** (failures - 1), 300) * random.uniform(0.5, 1.5)
db.execute(
"UPDATE last_known_good SET failures = ?, retry_at = ?, last_error = ? WHERE key = ?",
(failures, now + delay, error, key),
)
else:
db.execute(
"""INSERT INTO last_known_good (key, value, fetched_at) VALUES (?, ?, ?)
ON CONFLICT(key) DO UPDATE SET value = excluded.value, fetched_at = excluded.fetched_at,
failures = 0, retry_at = 0, last_error = NULL""",
(key, json.dumps(value), now),
)
return Result(value, now, 0.0, False)
# 4. the refresh failed or was skipped: serve the last known good, if it is not too old
if usable:
age = now - row["fetched_at"]
log.warning("serving stale key=%s age=%ds error=%s", key, age, error)
return Result(json.loads(row["value"]), row["fetched_at"], age, True, error)
raise Unavailable(f"{key}: no usable data (upstream error: {error})")The decisions that matter:
fetched_atonly changes when a good response arrives. This is the single most important line in the design. If a failed refresh, a stale read, or a cache layer that "touches" entries when it serves them reset the timestamp, old data would look fresh and your freshness indicator would lie.A response must be valid before it replaces the last known good.
validatecatches the case where the upstream returns200 OKwith an error page, an empty object, or a changed schema. A bad response that overwrites good data is how a small outage becomes a long one.The timeout is short. If the upstream hangs and your HTTP client waits 30 seconds, every user waits 30 seconds and then gets the stale data. Pick a timeout that matches what a user will tolerate, usually 1 to 3 seconds.
Back-off protects both sides. After a failure the code stops calling the upstream for a growing interval (5 s, 10 s, 20 s, up to 5 minutes, with jitter). Users get the stale value instantly instead of paying for a timeout on every request, the failing service gets breathing room, and recovery doesn't arrive as a synchronized stampede.
Back-off only applies while there's something usable to serve. Once the stored value is older than
max_stalethere's nothing to fall back on, so the code keeps trying and recovers on the first success.Goneis not an outage. A 404 or 410 means the upstream is healthy and says the thing doesn't exist. Serving a stale copy of something that was deleted is a bug, so that case is raised to the caller instead.Not every error is the same kind of error. Timeouts, connection failures, 5xx, and 429 are outages and trigger the fallback. A 401 or 403 means your credentials are broken. Falling back is still the right thing for the user, but it should also page someone, because the fallback will quietly run out of time otherwise.
Using it takes a few lines:
import functools
fetch_rates = functools.partial(fetch_json, "https://api.example.com/v1/rates", 2.0)
result = get(
"rates:usd",
fetch_rates,
fresh_for=300, # 5 minutes counts as fresh
max_stale=6 * 3600, # beyond 6 hours, we'd rather show an error
validate=lambda v: isinstance(v, dict) and "rates" in v,
)Make the staleness visible
Returning old data is only half of graceful degradation. The other half is telling people. Three places to do it:
In the API response. Give clients the facts instead of making them guess:
from datetime import datetime, timezone
def api_rates():
try:
r = get("rates:usd", fetch_rates, fresh_for=300, max_stale=6 * 3600,
validate=lambda v: isinstance(v, dict) and "rates" in v)
except Unavailable:
return {"error": "rates_unavailable"}, 503, {}
headers = {"X-Data-Stale": "true"} if r.stale else {}
body = {
"data": r.value,
"meta": {
"fetched_at": datetime.fromtimestamp(r.fetched_at, timezone.utc).isoformat(timespec="seconds"),
"age_seconds": int(r.age),
"stale": r.stale,
},
}
return body, 200, headersIn the UI. A small, calm banner beats a scary one, and it should say the age in words people understand:
def humanize_age(seconds):
s = int(seconds)
if s < 10:
return "just now"
if s < 60:
return f"{s} seconds ago"
for unit, size in (("day", 86400), ("hour", 3600), ("minute", 60)):
if s >= size:
n = s // size
return f"{n} {unit}{'' if n == 1 else 's'} ago"{% if result.stale %}
<div class="freshness freshness--stale" role="status">
Showing data from {{ result.age | humanize_age }}. Live data is temporarily unavailable.
</div>
{% endif %}This assumes you pass result to the template and register humanize_age as a template filter.
Some guidelines for the indicator:
Show nothing when data is fresh. A permanent "updated 2 minutes ago" label is fine for dashboards, but a banner should mean something is wrong.
Show the age, not just the word "stale." "Showing data from 14 minutes ago" lets the reader decide whether it still matters. "Stale data" doesn't.
Put the exact timestamp in a tooltip or
titleattribute for people who need it.Escalate with age. A neutral note at 15 minutes, a warning color at 2 hours, and disabled actions past whatever threshold is dangerous.
Compute age on the server. Send
age_seconds, and let the browser add the time elapsed since the page loaded (using something likeperformance.now()) to keep the label current. Never subtractfetched_atfrom the browser's clock, because user devices are routinely wrong by minutes or more.
Not all data degrades the same way
"Serve stale" is the right call for a lot of data and the wrong call for some. Decide per kind of data, and write the decision down:
Kind of data | Examples | When it's stale |
|---|---|---|
Informational display | Weather, news lists, leaderboards, catalog browsing | Serve it with a freshness label. A long |
Decision support | Stock levels shown to staff, delivery estimates, status dashboards | Serve it, label it prominently, and use a short |
Transactional | Price at checkout, account balance, inventory reservation | Don't guess. Disable the action, or ask the user to confirm the price they're seeing. |
Security | Permissions, revoked tokens, kill switches | Don't serve stale beyond a very short window. Default to the safe answer (deny). Stale permissions can grant access that was already taken away. |
The rule of thumb: the more irreversible the action the data triggers, the less stale it may be.
The cold-start problem
A last-known-good store only helps if it contains something. Three situations leave it empty:
First deploy, or a new key. There's no good data yet and the upstream is already down.
A fresh container with no persistent disk. The cache dies with the container.
A new region or replica that hasn't served the request before.
What helps:
Persist the store (a database table, a mounted volume) instead of keeping it in process memory.
Warm it on deploy. A startup step or a scheduled job that refreshes the important keys means the fallback exists before it's needed.
Ship a baked-in snapshot for the data that matters most: a JSON file committed to the repo, loaded when the store is empty. Label it honestly, for example "Built-in data from 1 November," so even the worst case degrades gracefully.
Know when you're serving stale
Graceful degradation has a failure mode of its own: it works so well that nobody notices the upstream has been down for four days. Treat "serving stale" as an event worth watching.
Log it. The code above emits one warning per stale response with the key, the age, and the error. Count these.
Expose the age. A health check that lists the oldest data you hold shows trouble quickly:
SELECT key,
CAST(strftime('%s', 'now') AS INTEGER) - CAST(fetched_at AS INTEGER) AS age_seconds,
failures,
last_error
FROM last_known_good
ORDER BY fetched_at
LIMIT 10;Alert on age, not on single failures. One failed call is noise. "Rates haven't refreshed for 30 minutes" is a signal. Set the alert threshold a little above
fresh_for, and a second, louder one well beforemax_stale, so you hear about it before users see an error.
Test the outage on purpose
Don't wait for the upstream to fail to find out whether this works. Run a fake upstream locally and break it:
Serve a small JSON endpoint and fetch it once. Confirm that the value is stored.
Make it return
503. Confirm the app serves the old value, flags it stale, and thatagekeeps growing rather than resetting.Make it hang for longer than your timeout. Confirm responses still arrive in about one timeout, not the full hang.
Call it repeatedly while it's down. Confirm the upstream is not contacted during back-off.
Bring it back. Confirm the next refresh succeeds, the stale flag clears, and
failuresreturns to zero.Return a
200with a garbage body. Confirm the last known good value is untouched.Return
404. Confirm you getGone, not the old data.Let the data age past
max_stale. Confirm you get a clean error, not ancient data.
Those eight checks cover the real failure modes: hard failure, slow failure, flapping, bad data, and permanent absence.
Pitfalls
urlopen's timeout isn't a deadline. In many HTTP clients, including the standard library's, the timeout applies to each blocking socket operation. A server that trickles bytes slowly can keep you waiting well beyond it. If you need a hard cap on total time, use a client that supports an overall deadline, or enforce one yourself.Caching per user without keying by user. If the data differs by user or tenant, the key must include that. Otherwise the fallback shows one customer's data to another.
A thundering herd when the entry expires. If 200 requests arrive while a refresh is due, 200 requests hit the upstream. Use a per-key lock within one process, or a lease row across processes, so one caller refreshes while the rest serve the stale value.
A quicker option at the HTTP layer
If your data flows through an HTTP cache or CDN, two Cache-Control extensions from RFC 5861 do part of this for you: stale-while-revalidate (serve stale while refreshing in the background) and stale-if-error (serve stale when the origin returns errors). Support varies between CDNs and proxies, so check the documentation for the one you use.
They're a good first step, but they don't replace the application-level version. They can't distinguish a 404 from an outage in the way your code can, they don't validate payloads, and they don't give your UI the age of the data. Many setups use both: headers at the edge, and a last-known-good store in the app for the data that matters.
You can also serve stale while refreshing in the background inside your own app: return the old value immediately and kick off the refresh on a worker thread or a queue. It lowers latency further, at the cost of more moving parts, so start with the synchronous version above.
Takeaways
Separate freshness (
fresh_for) from retention (max_stale). A TTL that deletes data deletes your fallback.Store the last good response with its
fetched_at, and change that timestamp only on success.Validate responses before they overwrite good data, and use short timeouts plus back-off with jitter.
Show the age, calmly, and escalate as data gets older. Compute it on the server.
Decide per kind of data: serve stale for display, be careful for decisions, fail closed for money and security.
Persist and warm the store so the fallback exists on a cold start.
Alert on how long you've been stale, and rehearse the outage before it happens.
Comments(0)
Join the discussion by logging into your account.
No comments yet. Be the first to comment!