Tutorial

Your API Key Doesn't Know When to Stop: Building a Spend Kill-Switch in an Afternoon

A tiny local proxy that holds your API key, tracks cost per script, and shuts the tap when a loop goes wrong, in under 200 lines of Python.

·
9 min read
·
AI-assisted

AI tooling disclosure

Generative AI materially contributed to this article.

Human review confirmed by the author.

Your API Key Doesn't Know When to Stop: Building a Spend Kill-Switch in an Afternoon

A script that calls a paid API keeps spending for as long as it keeps calling. A retry loop with no limit, a missing exit condition, or a job started twice by accident will bill for every attempt, and nothing in your code notices until the invoice does.

Provider-side controls help, but they tend to be coarse: monthly limits, account-wide scope, and budget alerts that notify you instead of blocking anything. None of them can tell you which script burned the money, and none will stop a loop within minutes.

So let's build the missing layer: a small local proxy that sits between your scripts and the API, tracks cost per script, and refuses calls once a cap is reached. It's standard-library Python plus SQLite, with no dependencies, in under 200 lines.

The design

Three rules do all the work.

  1. The real key lives only in the proxy. Scripts get a per-script token that is worthless anywhere else. A runaway script can't route around the limit, because it never held the key.

  2. Reserve before, settle after. Before forwarding a call, the proxy books its worst-case cost. When the response arrives, it swaps the reservation for the real cost.

  3. Three caps. A global daily cap as the backstop, a daily cap per script so one script can't eat everyone's budget, and a short burst cap per script (spend in the last five minutes), which is what catches a loop quickly.

sequenceDiagram
    participant S as Your script
    participant P as Spend proxy
    participant L as Ledger (SQLite)
    participant A as Provider API
    S->>P: request (script token, no real key)
    P->>L: reserve worst-case cost (atomic)
    alt over any cap
        L-->>P: denied
        P-->>S: 402 budget exceeded
    else within caps
        L-->>P: reserved
        P->>A: forward with real key
        A-->>P: response + usage
        P->>L: settle actual cost
        P-->>S: response
    end

The ledger

Everything hinges on one function: check the caps and book the reservation as a single atomic step.

import sqlite3
import time

DB = "ledger.db"
DAY_OFFSET_SECONDS = 0  # shifts the day boundary, e.g. 19800 for IST (UTC+5:30)


def _conn():
    c = sqlite3.connect(DB, isolation_level=None, timeout=10)  # we manage transactions
    c.execute("PRAGMA journal_mode=WAL")
    return c


def _day(ts):
    return time.strftime("%Y-%m-%d", time.gmtime(ts + DAY_OFFSET_SECONDS))


def init():
    c = _conn()
    c.execute("""CREATE TABLE IF NOT EXISTS spend(
        id INTEGER PRIMARY KEY, ts REAL, day TEXT, tag TEXT,
        reserved REAL, actual REAL)""")
    c.execute("CREATE INDEX IF NOT EXISTS spend_day ON spend(day, tag)")
    c.execute("CREATE INDEX IF NOT EXISTS spend_ts ON spend(ts)")
    c.close()


# A row counts at its settled cost if it has one, otherwise at its reservation.
SUM = "SELECT COALESCE(SUM(COALESCE(actual, reserved)), 0) FROM spend WHERE "


def reserve(tag, worst, caps):
    """Atomically check all caps and book the worst-case cost.
    Returns (entry_id, None) on success or (None, reason) when denied."""
    now = time.time()
    day = _day(now)
    c = _conn()
    c.execute("BEGIN IMMEDIATE")  # take the write lock before reading
    try:
        day_total = c.execute(SUM + "day=?", (day,)).fetchone()[0]
        tag_total = c.execute(SUM + "day=? AND tag=?", (day, tag)).fetchone()[0]
        burst = c.execute(SUM + "tag=? AND ts>?",
                          (tag, now - caps["burst_window"])).fetchone()[0]
        reason = None
        if day_total + worst > caps["daily"]:
            reason = "daily cap"
        elif tag_total + worst > caps["per_tag"]:
            reason = f"daily cap for '{tag}'"
        elif burst + worst > caps["burst"]:
            reason = f"burst cap for '{tag}'"
        if reason:
            c.execute("ROLLBACK")
            return None, reason
        cur = c.execute(
            "INSERT INTO spend(ts, day, tag, reserved) VALUES (?,?,?,?)",
            (now, day, tag, worst))
        c.execute("COMMIT")
        return cur.lastrowid, None
    except Exception:
        c.execute("ROLLBACK")
        raise
    finally:
        c.close()


def settle(entry_id, actual):
    c = _conn()
    c.execute("UPDATE spend SET actual=? WHERE id=?", (actual, entry_id))
    c.close()


def summary():
    c = _conn()
    day = _day(time.time())
    rows = c.execute(
        "SELECT tag, COALESCE(SUM(COALESCE(actual, reserved)), 0), COUNT(*) "
        "FROM spend WHERE day=? GROUP BY tag", (day,)).fetchall()
    c.close()
    return {"day": day,
            "total": round(sum(r[1] for r in rows), 4),
            "by_tag": {t: {"spent": round(s, 4), "calls": n} for t, s, n in rows}}

Three details are worth a closer look:

  • BEGIN IMMEDIATE takes SQLite's write lock before any read, so "check the totals" and "book the reservation" can't be interleaved with another caller's. Twenty threads, or two proxy processes, can't all see room for the last cent.

  • COALESCE(actual, reserved) counts a row at its real cost once settled and at its reservation until then. A call that crashes before settling keeps counting at worst case, so failures make the ledger stricter, never looser.

  • DAY_OFFSET_SECONDS exists because "today" needs a definition. Your local midnight and your provider's billing day may not line up, so pick the boundary on purpose.

The proxy

import json
import os
import urllib.error
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

import ledger

UPSTREAM = os.environ.get("UPSTREAM", "https://api.anthropic.com")
REAL_KEY = os.environ["REAL_API_KEY"]  # the only place the real key lives

# script token -> tag used for per-script accounting
TOKENS = {"tok-report-bot": "report-bot", "tok-scraper": "scraper"}

# $ per million tokens (input, output). Placeholders: copy current prices
# from your provider's pricing page and note the date you did.
PRICES = {"demo-model": (3.0, 15.0)}  # prices as of: <date>

CAPS = {"daily": 0.50, "per_tag": 0.30, "burst": 0.20, "burst_window": 300}


def worst_case(body, req, p_in, p_out):
    max_out = req["max_tokens"]
    if not isinstance(max_out, int) or max_out <= 0:
        raise ValueError("max_tokens must be a positive integer")
    est_in = len(body) / 2  # pessimistic: ~2 bytes per token
    return (est_in * p_in + max_out * p_out) / 1_000_000


def extract_usage(data):
    try:
        u = json.loads(data)["usage"]
        i = u.get("input_tokens", u.get("prompt_tokens"))
        o = u.get("output_tokens", u.get("completion_tokens"))
        return (i, o) if i is not None and o is not None else None
    except (ValueError, KeyError, TypeError, AttributeError):
        return None


def forward(path, body, headers):
    h = {"content-type": "application/json", "x-api-key": REAL_KEY}
    for name in ("anthropic-version", "anthropic-beta"):
        if name in headers:
            h[name] = headers[name]
    req = urllib.request.Request(UPSTREAM + path, data=body, method="POST", headers=h)
    try:
        with urllib.request.urlopen(req, timeout=60) as r:
            return r.status, r.read()
    except urllib.error.HTTPError as e:
        return e.code, e.read()  # upstream answered with an error


class Handler(BaseHTTPRequestHandler):
    def reply(self, status, payload):
        if not isinstance(payload, bytes):
            payload = json.dumps(payload).encode()
        self.send_response(status)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(payload)))
        self.end_headers()
        self.wfile.write(payload)

    def do_GET(self):
        if self.path == "/_spend":
            return self.reply(200, ledger.summary())
        self.reply(404, {"error": "not found"})

    def do_POST(self):
        tag = TOKENS.get(self.headers.get("x-api-key", ""))
        if tag is None:
            return self.reply(401, {"error": "unknown script token"})
        body = self.rfile.read(int(self.headers.get("content-length", 0)))
        try:
            req = json.loads(body)
            p_in, p_out = PRICES[req["model"]]
            worst = worst_case(body, req, p_in, p_out)
        except (ValueError, KeyError, TypeError):
            return self.reply(400, {"error": "bad JSON, unknown model, or missing "
                                             "max_tokens: refusing to guess a price"})

        entry, reason = ledger.reserve(tag, worst, CAPS)
        if entry is None:
            return self.reply(402, {"error": f"budget exceeded: {reason}"})

        try:
            status, data = forward(self.path, body, self.headers)
        except Exception:
            # Unknown whether the provider billed us: keep the reservation.
            return self.reply(502, {"error": "upstream unreachable or timed out"})

        if status >= 400:
            ledger.settle(entry, 0.0)  # errors aren't billed: give the money back
        else:
            usage = extract_usage(data)
            actual = (usage[0] * p_in + usage[1] * p_out) / 1_000_000 if usage else worst
            ledger.settle(entry, actual)
        self.reply(status, data)

    def log_message(self, *args):
        pass


if __name__ == "__main__":
    ledger.init()
    port = int(os.environ.get("PORT", "8787"))
    print(f"spendguard listening on 127.0.0.1:{port}")
    ThreadingHTTPServer(("127.0.0.1", port), Handler).serve_forever()

The decisions that matter:

  • The worst-case estimate is a deliberately pessimistic input size (two bytes per token) plus the full max_tokens at the output price. That's why requests with a missing or non-positive max_tokens, or with a model that isn't in the price table, are refused with a 400. A proxy that guesses a price is a proxy that can be wrong in your favor once and in the provider's favor forever.

  • Denied calls return 402, not 429. Many SDKs automatically retry 429s, and a budget refusal is not a transient hiccup.

  • Settlement follows what you actually know. An upstream error is refunded, since errors typically aren't billed. A network failure or timeout keeps the reservation, because you can't tell whether the provider billed. A success with no readable usage settles at worst case.

Running it

Put your real key in the proxy's environment and start it:

export REAL_API_KEY=sk-ant-...   # your real key, only ever here
python proxy.py

Then point a script at the proxy with its own token:

from anthropic import Anthropic

client = Anthropic(base_url="http://127.0.0.1:8787", api_key="tok-scraper")

Edit TOKENS, PRICES and CAPS at the top of proxy.py to match your scripts, your provider's current prices and your tolerance for pain. curl localhost:8787/_spend shows today's totals per script. For OpenAI-style APIs, change the token lookup and the forwarded header to use Authorization: Bearer; the usage parsing already accepts both field naming styles.

The kill test

A budget control you haven't attacked is a budget control you're hoping works. This test uses a mock provider that returns fixed usage (500 input tokens, 800 output tokens, which costs $0.0135 per call at the placeholder prices), so it spends no real money. It also only accepts the real key, so a passing run shows the proxy is injecting it.

import json
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer


class Mock(BaseHTTPRequestHandler):
    def do_POST(self):
        self.rfile.read(int(self.headers.get("content-length", 0)))
        if self.headers.get("x-api-key") != "sk-demo":  # only the real key works
            self.send_response(401)
            self.end_headers()
            return
        time.sleep(0.01)  # pretend to think
        out = json.dumps({"content": [{"type": "text", "text": "ok"}],
                          "usage": {"input_tokens": 500, "output_tokens": 800}}).encode()
        self.send_response(200)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(out)))
        self.end_headers()
        self.wfile.write(out)

    def log_message(self, *args):
        pass


ThreadingHTTPServer(("127.0.0.1", 9001), Mock).serve_forever()

The test script has two parts. The first runs a loop with no exit condition until the proxy stops it. The second pits two ways of enforcing the same $0.20 budget against 20 parallel workers: check the total, call, then record, versus reserve, call, then settle.

import json
import os
import threading
import time
import urllib.error
import urllib.request

import ledger

PROXY = "http://127.0.0.1:8787"


def call(token, max_tokens=1000):
    body = json.dumps({"model": "demo-model", "max_tokens": max_tokens,
                       "messages": [{"role": "user", "content": "hi"}]}).encode()
    req = urllib.request.Request(PROXY + "/v1/messages", data=body, method="POST",
                                 headers={"x-api-key": token,
                                          "content-type": "application/json"})
    try:
        with urllib.request.urlopen(req) as r:
            return r.status, json.loads(r.read())
    except urllib.error.HTTPError as e:
        return e.code, json.loads(e.read())


def spend():
    with urllib.request.urlopen(PROXY + "/_spend") as r:
        return json.loads(r.read())


def runaway_loop():
    print("== Test 1: a runaway loop ==")
    n = 0
    while True:
        status, resp = call("tok-scraper")
        if status != 200:
            print(f"call {n + 1}: HTTP {status} -> {resp['error']}")
            break
        n += 1
    print(f"calls that went through: {n}")
    print("ledger:", json.dumps(spend()["by_tag"]))
    status, _ = call("tok-report-bot")
    print(f"other script (report-bot) right after: HTTP {status}\n")


# --- Test 2: same budget, two ways of checking it, 20 parallel workers ---
CAP, ACTUAL, WORST, WORKERS = 0.20, 0.0135, 0.0152, 20


def race_naive():
    spent = 0.0
    lock = threading.Lock()

    def worker():
        nonlocal spent
        while True:
            with lock:
                if spent >= CAP:  # check ...
                    return
            time.sleep(0.05)  # ... the API call happens ...
            with lock:
                spent += ACTUAL  # ... and only then do we record it
    run(worker)
    return spent


def race_reserve():
    ledger.DB = "race.db"
    if os.path.exists("race.db"):
        os.remove("race.db")
    ledger.init()
    caps = {"daily": CAP, "per_tag": 99, "burst": 99, "burst_window": 300}

    def worker():
        while True:
            entry, _ = ledger.reserve("race", WORST, caps)  # reserve first
            if entry is None:
                return
            time.sleep(0.05)  # the API call
            ledger.settle(entry, ACTUAL)
    run(worker)
    c = ledger._conn()
    total = c.execute(ledger.SUM + "tag='race'").fetchone()[0]
    c.close()
    return total


def run(worker):
    threads = [threading.Thread(target=worker) for _ in range(WORKERS)]
    [t.start() for t in threads]
    [t.join() for t in threads]


if __name__ == "__main__":
    runaway_loop()
    print(f"== Test 2: {WORKERS} parallel workers, cap ${CAP:.2f} ==")
    for name, fn in [("check-then-record", race_naive), ("reserve-then-settle", race_reserve)]:
        total = fn()
        print(f"{name:<20} spent ${total:.4f}  ({total / CAP * 100:.0f}% of cap)")

Run it:

rm -f ledger.db*   # the ledger remembers earlier spend, so start each run clean
python mock_upstream.py &
UPSTREAM=http://127.0.0.1:9001 REAL_API_KEY=sk-demo python proxy.py &
python killtest.py
kill %1 %2         # stop the mock and the proxy
== Test 1: a runaway loop ==
call 15: HTTP 402 -> budget exceeded: burst cap for 'scraper'
calls that went through: 14
ledger: {"scraper": {"spent": 0.189, "calls": 14}}
other script (report-bot) right after: HTTP 200

== Test 2: 20 parallel workers, cap $0.20 ==
check-then-record    spent $0.4590  (230% of cap)
reserve-then-settle  spent $0.1890  (94% of cap)

Test 1 shows the loop getting 14 calls through before the 15th was refused: its worst-case cost of about $0.015 would have crossed the $0.20 burst cap. The spend stopped at $0.189, and a different script kept working, so one runaway doesn't take down the rest.

Test 2 is the reason for reservations. With check-then-record, all 20 workers pass the check before any of them has recorded a cent, so every worker's first call goes through, and the stragglers add more. It overshot the cap by more than double (the exact figure varies with timing on your machine, but it always overshoots). With reservations, spend can't exceed the cap as long as each call's real cost is at or below its reservation, which is exactly what the pessimistic estimate is for.

Rough edges to know about

  • Streaming. This version buffers the whole response, so streamed calls work but settle at worst case, because the proxy can't read a usage block out of an event stream. The proper fix is to pass bytes through as they arrive, read the final usage event, settle then, and settle at worst case if the client disconnects mid-stream.

  • Prices drift. The price table is a hand-copied snapshot, which is why it carries a date comment. Refusing unknown models keeps a new model from slipping through at an imagined price.

  • Stuck reservations. If the proxy dies mid-call, the unsettled row counts at worst case until the day rolls over. That's the safe direction, and a small janitor that settles stale rows is easy to add if it bothers you.

What this can't do

  • It's a guardrail for your own scripts, not a security boundary. If the real key also sits in an old .env file or your shell history, anything can still use it directly. Rotate the key when you move it into the proxy.

  • It can't see cloud bills. Billing data typically arrives with a delay, so a ledger like this can't catch a runaway VM or an egress surprise. For those, rely on quotas, instance limits and budget alerts as a second layer.

  • If the proxy is down, your scripts stop. That's fail-closed by design, and you'll notice quickly.

A good next step is an alert when a cap trips (a push notification or a Slack message), so you hear about a refused loop before you wonder why a job went quiet. Until then, run the kill test with your own caps and prices before trusting any of it.

Comments(0)

Join the discussion by logging into your account.

No comments yet. Be the first to comment!

Anshu Pathak

Passionate developer sharing knowledge about modern web technologies and best practices.

Subscribe to Anshu Pathak's Newsletter

Direct email dispatches when new stories are published. Zero algorithms.