Rate Limiting Algorithms: How to Protect Your API Without Annoying Your Users

Every API eventually meets a client that sends too much traffic. Sometimes it’s a buggy retry loop, sometimes a scraper, and sometimes a perfectly well-meaning customer whose cron job just got ten times bigger. Rate limiting is how you stay up, and stay fair, when that happens.

The idea is simple: cap how many requests a client can make in a given period. The hard part is how you count. Different algorithms make different trade-offs between accuracy, memory use, burst tolerance and how easy they are to run across many servers. This post walks through the five you’ll meet most often, then covers what changes when you go distributed.

Why rate limit at all?

• Availability. One noisy client shouldn’t be able to starve everyone else of capacity.

• Fairness. Limits make sure shared resources are divided sensibly between tenants.

• Cost control. Downstream calls (databases, third-party APIs, LLM inference) cost money per request.

• Security. Limits slow down brute-force login attempts, credential stuffing and enumeration attacks.

First, decide what you’re limiting

Before picking an algorithm, decide on the key you count against. Common choices are the API key or user ID, the IP address, the tenant or organization, or a combination such as user plus endpoint. IP address is the weakest option on its own, because many users can share one address behind a corporate NAT or mobile carrier, while one attacker can rotate through thousands.

You also need to decide what a “request” costs. A cheap read and an expensive report generation probably shouldn’t consume the same quota. Most of the algorithms below can handle this by charging a variable cost per request.

1. Fixed window counter

The simplest approach: divide time into fixed windows (say, one minute each), keep a counter per client per window, and reject requests once the counter passes the limit. When the window rolls over, the counter resets.

import time

def allow(redis, key, limit, window_seconds):
    window = int(time.time() // window_seconds)
    bucket = f'{key}:{window}'
    pipe = redis.pipeline()  # runs as one MULTI/EXEC transaction
    pipe.incr(bucket)
    pipe.expire(bucket, window_seconds)
    count, _ = pipe.execute()
    return count <= limit

Pros: Tiny memory footprint (one integer per client), trivial to implement, very fast.

Cons: It has a well-known boundary problem. With a limit of 100 requests per minute, a client can send 100 requests at 12:00:59 and another 100 at 12:01:00. That’s 200 requests in two seconds, double the intended rate, and every one of them is allowed.

Use it when: You want something cheap and the boundary spikes are tolerable, for example coarse daily quotas or internal tooling.

2. Sliding window log

To fix the boundary problem, store the timestamp of every request. When a new request arrives, discard timestamps older than the window, count what remains, and allow the request only if the count is under the limit.

import time
import uuid

def allow(redis, key, limit, window_seconds):
    now = time.time()
    member = f'{now}:{uuid.uuid4().hex}'  # unique even if two requests share a timestamp
    pipe = redis.pipeline()
    pipe.zremrangebyscore(key, 0, now - window_seconds)  # drop old entries
    pipe.zadd(key, {member: now})
    pipe.zcard(key)
    pipe.expire(key, window_seconds)
    _, _, count, _ = pipe.execute()
    if count > limit:
        redis.zrem(key, member)  # a rejected request shouldn’t count against the client
        return False
    return True

Pros: Exact. At any instant, the client has made at most limit requests in the trailing window, with no boundary loophole.

Cons: Memory grows with the limit, since you store one entry per request. At 10,000 requests per minute per client, that adds up quickly. In the sketch above, rejected requests are removed again so they don’t count against the client, but under heavy concurrency you’d want the whole check inside a Lua script to keep it atomic.

Use it when: Accuracy matters more than memory and limits are low, such as login attempts or password resets.

3. Sliding window counter

This is a clever hybrid. Keep fixed-window counters for the current and previous windows, then estimate the sliding count by weighting the previous window by how much of it still overlaps with the trailing window:

estimated = previous_count * (1 - elapsed_fraction) + current_count

If you’re 30% of the way into the current minute, you count 70% of last minute’s requests plus all of this minute’s.

Pros: Two integers per client, and it smooths out the boundary spike nearly as well as the log. Cloudflare measured this approach across 400 million requests and found that only 0.003% were wrongly allowed or rate limited.

Cons: It’s an estimate, not an exact count, and it assumes the previous window’s requests were spread evenly. If traffic in the previous window was heavily bunched at one end, the estimate can be off.

Use it when: You want sliding-window behavior at fixed-window cost. For many public APIs this is the sweet spot.

4. Token bucket

Imagine a bucket that holds up to B tokens and is refilled at a steady rate of r tokens per second. Each request removes a token (or several, if the request is expensive). If the bucket is empty, the request is rejected.

The important property: an idle client accumulates tokens, so it can send a burst of up to B requests at once, but its long-run average can never exceed r per second.

import time

class TokenBucket:
    def __init__(self, capacity, refill_rate):
        self.capacity = capacity        # maximum burst size
        self.refill_rate = refill_rate  # tokens added per second
        self.tokens = capacity
        self.last_refill = time.monotonic()

    def allow(self, cost=1):
        now = time.monotonic()
        elapsed = now - self.last_refill
        self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
        self.last_refill = now

        if self.tokens >= cost:
            self.tokens -= cost
            return True
        return False

Notice there’s no background timer. Tokens are refilled lazily, computed from the elapsed time whenever a request arrives. That makes it cheap to keep millions of buckets, since each one is just two numbers: the token count and a timestamp.

Pros: Allows natural bursts, uses constant memory, supports variable request costs, and is easy to reason about with two knobs (burst size and sustained rate). This is what Stripe, AWS API Gateway and many others use or resemble.

Cons: Bursts pass through to your backend. If your downstream system can’t absorb a burst of B requests, the bucket size needs to be tuned with that in mind.

Use it when: Clients behave in bursts (page loads that fire several API calls at once, batch uploads) and you want to allow that while still capping the average.

5. Leaky bucket

The leaky bucket flips the picture. Requests go into a queue (the bucket) of fixed size and are processed at a constant rate, as if the bucket had a small hole in the bottom. If the queue is full when a request arrives, it’s dropped.

Where a token bucket permits bursts, a queue-based leaky bucket smooths them: no matter how spiky the input, the output rate is constant.

Pros: Produces a steady, predictable load on downstream systems. Ideal when the thing behind the limiter has a hard throughput ceiling.

Cons: A burst of traffic fills the queue, and queued requests wait, so latency goes up. Old requests can also crowd out fresh ones. If you implement it as a pure “meter” with no queue, it becomes equivalent to a token bucket, with the counting run in the opposite direction.

Use it when: You’re protecting something that needs smooth input, such as a payment processor, a legacy system or an outbound call to a rate-limited third party.

Comparing them side by side

Going distributed

Everything above is easy on a single server. It gets trickier when many instances sit behind a load balancer and all need to agree on a client’s usage.

Use a shared store. Redis is the usual choice, thanks to atomic operations, expiring keys and low latency. Each app server asks Redis whether a request is allowed instead of keeping local counters.

Make the check atomic. A naive read-then-write sequence has a race condition. Two servers read a count of 99, both decide the request is allowed, and both write 100. Use atomic commands like INCR, or wrap the whole read-compute-write cycle in a Lua script so Redis executes it as a single step. Token bucket is a classic Lua-script use case.

Watch the clocks. If each app server computes timestamps with its own clock, drift between machines will skew the results. Either take the time from Redis itself or keep windows coarse enough that small drift doesn’t matter.

Decide how to fail. What happens when the rate limiter’s store is unreachable? Fail open (allow everything) keeps your API available but leaves you unprotected. Fail closed (reject everything) protects the backend but turns a limiter outage into a full outage. Many APIs fail open, perhaps with a coarse local fallback limiter as a safety net.

Consider a hybrid. Checking Redis on every request adds a network round trip. A common optimization is to keep a small local limiter in each instance for a first line of defense, and sync with the central store periodically or only for expensive endpoints. You trade a little accuracy for a lot of speed.

Being a good citizen: talking to clients

A rate limiter that silently drops requests is miserable to integrate with. A good one tells clients exactly what’s going on.

• Respond with HTTP 429 Too Many Requests, not a generic 500 or 503.

• Include a Retry-After header telling the client how many seconds to wait.

• Expose the current state in headers. The widely used convention is X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset. An IETF draft (revision 11, May 2026) is still unfinished and takes a different approach: a RateLimit-Policy header that advertises your quota and a RateLimit header that reports the current state.

• Document your limits clearly, including how they’re scoped and whether they differ by plan or endpoint.

On the client side, the right response to a 429 is exponential backoff with jitter. Waiting a random amount within a growing window stops a thousand throttled clients from all retrying at the same instant and re-creating the spike that caused the throttling.

Common pitfalls

• Limiting only by IP. This punishes legitimate users behind shared addresses and does little against distributed attackers. Prefer authenticated identities where you can.

• One limit for everything. A single global number is too tight for cheap endpoints and too loose for expensive ones. Use per-endpoint limits or cost-based tokens.

• Ignoring retry storms. If clients retry immediately on failure, a brief overload snowballs. Pair server-side limits with client-side backoff, and consider load shedding as a complement to rate limiting.

• No observability. Track how often limits trigger, for whom, and on which endpoints. A spike in 429s is either an attack, a buggy client or a limit set too low, and you want to know which.

• Surprise changes. Tightening limits without notice breaks integrations. Announce changes and give clients time to adapt.

So, which one should you pick?

A rough decision guide:

• Just getting started, or limits are coarse? Fixed window counter. Accept the boundary quirk and move on.

• Public API with diverse clients? Token bucket or sliding window counter. Both are cheap, accurate enough and friendly to bursty traffic.

• Protecting a fragile downstream with a hard throughput limit? Leaky bucket.

• Security-critical paths with small limits (logins, password resets)? Sliding window log, where exactness is worth the memory.

In practice, many production systems layer several of these: a token bucket per API key for general usage, a stricter sliding window on sensitive endpoints, and a coarse global limit as a final safety net.

Wrapping up

Rate limiting looks like a counting problem, but it’s really a question about what behavior you want to allow. Do you want to permit bursts or flatten them? Is exactness worth the memory? Can you tolerate approximation for the sake of speed? Once you’re clear on those answers, the right algorithm usually picks itself, and the rest is careful engineering: atomic operations, sensible headers and clear communication with the people calling your API.