Top Load Balancing Algorithms Explained: How to Choose the Right One

Sooner or later, a single server stops being enough. Traffic grows, a deployment needs zero downtime, or one machine failing shouldn’t take your whole product offline. The standard answer is to run several servers and put a load balancer in front of them.

The load balancer’s job sounds simple: take each incoming request and send it to a healthy server. The hard part is the rule it uses to choose. That rule is the load balancing algorithm, and it decides whether your traffic spreads evenly or whether one server melts while the others sit idle.

In this post, we’ll walk through the most important load balancing algorithms, see how each one works, and finish with a practical way to choose between them.

flowchart LR
    C1[Client A] --> LB{{"Load balancer"}}
    C2[Client B] --> LB
    C3[Client C] --> LB
    LB --> S1[Server 1]
    LB --> S2[Server 2]
    LB --> S3[Server 3]

Before the Algorithms: Layer 4 vs Layer 7

Load balancers work at two levels, and it affects which algorithms make sense:

• Layer 4 (transport): balances TCP or UDP connections using IP addresses and ports. It is fast and content-agnostic, but it can’t see inside the request.

• Layer 7 (application): understands HTTP, so it can route by URL, header, or cookie. This is what makes URL-based hashing and cookie-based stickiness possible.

Everything below applies to both, except where an algorithm needs to read request content.

Two Families: Static and Dynamic

Algorithms fall into two broad groups. Static algorithms follow a fixed rule and ignore what the servers are doing right now. Dynamic algorithms look at live information such as active connections or response times before deciding.

flowchart LR
    LB(("Load balancing<br/>algorithms")) --> ST[Static: ignore live server state]
    LB --> DY[Dynamic: use live server state]
    ST --> RR[Round robin]
    ST --> WRR[Weighted round robin]
    ST --> IPH[IP hash]
    ST --> CH[Consistent hashing]
    DY --> LC[Least connections]
    DY --> WLC[Weighted least connections]
    DY --> LRT[Least response time]
    DY --> P2C[Power of two choices]

1. Round Robin

How it works: The load balancer keeps an ordered list of servers and hands out requests in turn: first to Server 1, then Server 2, then Server 3, then back to Server 1.

flowchart LR
    C[Clients] --> LB{{"Load balancer"}}
    LB -->|"requests 1, 4, 7 ..."| S1[Server 1]
    LB -->|"requests 2, 5, 8 ..."| S2[Server 2]
    LB -->|"requests 3, 6, 9 ..."| S3[Server 3]

Strengths

• Extremely simple and cheap to run, with almost no state to track

• Perfectly even distribution when servers and requests are alike

• The default in most load balancers

Weaknesses

• Ignores server capacity and current load

• If some requests take much longer than others, slow ones can pile up on one server while its neighbors idle

Best for: identical, stateless servers handling short, uniform requests.

2. Weighted Round Robin

How it works: Same rotation, but each server gets a weight that reflects its capacity. A server with weight 3 receives three times as many requests as a server with weight 1. Many implementations use a “smooth” variant that interleaves picks instead of sending a burst to the biggest server.

flowchart LR
    LB{{"Load balancer"}} -->|"3 of every 6 requests"| A["Server A<br/>weight 3 (large)"]
    LB -->|"2 of every 6 requests"| B["Server B<br/>weight 2 (medium)"]
    LB -->|"1 of every 6 requests"| C["Server C<br/>weight 1 (small)"]

Strengths

• Handles mixed hardware, such as a fleet with older and newer machines

• Handy for gradual rollouts: give a new version a small weight and raise it over time

Weaknesses

• Weights are set by hand and don’t react to real-time load

• Still blind to how long individual requests take

Best for: heterogeneous servers with fairly uniform requests, and canary-style rollouts.

3. Least Connections

How it works: For each new request, the load balancer picks the server with the fewest active connections right now.

flowchart LR
    NR[New request] --> LB{{"Load balancer"}}
    LB -.->|skip| S1["Server 1<br/>12 active connections"]
    LB ==>|"picks fewest"| S2["Server 2<br/>4 active connections"]
    LB -.->|skip| S3["Server 3<br/>9 active connections"]

Strengths

• Adapts automatically when request durations vary widely

• Naturally steers traffic away from servers bogged down by slow requests

Weaknesses

• Needs to track connection counts for every server

• Connection count is only a proxy for load: an idle keep-alive connection counts the same as a heavy one

• A freshly added server can be flooded all at once unless the balancer supports a slow-start period

Best for: long-lived or variable-length connections such as WebSockets, streaming, database proxies, and APIs with uneven request costs.

4. Weighted Least Connections

How it works: This combines the previous two ideas. The load balancer divides each server’s active connections by its weight and picks the lowest score, so bigger servers are allowed to hold proportionally more connections.

Server A has the most connections, but it is also the biggest, and Server B is relatively the least loaded, so B wins.

Best for: mixed hardware combined with uneven request durations. This is a strong general-purpose choice for real-world fleets.

5. Least Response Time

How it works: The load balancer measures how quickly each server has been responding and prefers the fastest one, usually combined with its connection count. In NGINX, for example, the least_time directive (NGINX Plus, and open source since version 1.31.0) picks the server with the lowest average latency and the fewest active connections.

Strengths

• Optimizes what users actually feel: latency

• Automatically avoids degraded or struggling servers

• Works well when backends sit in different locations or have different performance profiles

Weaknesses

• Measurements are noisy, so good implementations smooth them with a moving average

• Can overload whichever server looks fastest, which then becomes slow, so it needs to be paired with connection awareness

• Not built into every load balancer

Best for: latency-sensitive applications and backends with uneven or changing performance.

6. IP Hash

How it works: The load balancer hashes the client’s IP address and uses the result to choose a server. The same client keeps landing on the same server, which gives you session stickiness without cookies or shared session storage.

sequenceDiagram
    participant A as Client A
    participant LB as Load balancer
    participant S2 as Server 2
    A->>LB: Request 1
    Note over LB: Hash of client IP<br/>picks Server 2
    LB->>S2: Forward
    A->>LB: Request 2
    LB->>S2: Same IP, same server

Strengths

• Simple, stateless stickiness

• Works at Layer 4 as well as Layer 7

Weaknesses

• Many users behind one corporate NAT or proxy share an IP, so they all hit one server. Some implementations hash only part of the address (NGINX’s ip_hash uses the first three octets of an IPv4 address), which makes this effect stronger

• Mobile users change IPs as they move

• With a simple modulo approach, adding or removing a server remaps a large share of clients at once

Best for: simple session affinity on a stable server pool.

7. Consistent Hashing

How it works: Servers (and requests) are placed on a conceptual hash ring. Each request is hashed to a point on the ring and handled by the next server clockwise. When a server joins or leaves, only the keys that belonged to it move. On average, roughly 1/N of keys are affected, instead of nearly everything as with a plain modulo hash. Implementations typically place each server on the ring many times (“virtual nodes”) to even out the load.

flowchart TB
    subgraph Before["Before: servers A, B, C"]
        direction LR
        G1["Keys 1 and 2"] --> A1[Server A]
        G2["Keys 3 and 4"] --> B1[Server B]
        G3["Keys 5 and 6"] --> C1[Server C]
    end
    subgraph After["After: Server B leaves"]
        direction LR
        H1["Keys 1 and 2"] --> A2[Server A]
        H2["Keys 3 and 4"] ==>|"only these move"| C2[Server C]
        H3["Keys 5 and 6"] --> C2
    end
    Before ~~~ After

Only keys 3 and 4 moved. Keys 1, 2, 5, and 6 stayed exactly where they were.

Strengths

• Minimal disruption when the server pool changes

• Ideal for caches: the same URL keeps hitting the same cache node, so hit rates stay high

Weaknesses

• More complex than the other algorithms

• Uneven without enough virtual nodes

• A single very popular key can still overload its server

Best for: cache tiers, CDNs, sharded services, and any pool that scales up and down often.

8. Power of Two Choices

How it works: Pick two servers at random, compare their current load, and send the request to the lighter one. It sounds almost too simple, but it dramatically reduces worst-case imbalance compared with purely random choice, and it does so without needing a complete, up-to-date view of every server.

flowchart TD
    R[New request] --> P["Pick two servers at random"]
    P --> C{"Compare current load"}
    C -->|"First is lighter"| A["Send to first server"]
    C -->|"Second is lighter"| B["Send to second server"]

Strengths

• Cheap: only two lookups per request

• Excellent when several load balancers share the same backends, because each one doesn’t need a perfect global picture. This is why it avoids the herding problem where every balancer rushes to the same “least loaded” server

• Scales well to very large pools

Weaknesses

• Needs a reliable load signal such as active requests

• No session affinity

Best for: large, distributed deployments with multiple load balancers or proxies.

Config Cheat Sheet: NGINX and HAProxy

You rarely implement these yourself. Here is how the algorithms map to two popular load balancers:

A weighted least-connections setup looks like this in each:

upstream backend {
    least_conn;
    server 10.0.0.11 weight=3;
    server 10.0.0.12 weight=2;
    server 10.0.0.13;
}

backend app
    balance leastconn
    server app1 10.0.0.11:8080 weight 3 check
    server app2 10.0.0.12:8080 weight 2 check
    server app3 10.0.0.13:8080 check

Directive support changes between versions and editions, so confirm against the documentation for the version you run.

Side-by-Side Comparison

Don’t Forget Health Checks

The best algorithm in the world is useless if it keeps sending traffic to a dead server. Every production load balancer should run health checks:

• Active checks probe each server on a schedule, such as an HTTP request to /health

• Passive checks watch real traffic and mark a server unhealthy after repeated errors or timeouts

sequenceDiagram
    participant S1 as Server 1
    participant LB as Load balancer
    participant S2 as Server 2
    LB->>S1: GET /health
    S1-->>LB: 200 OK
    LB->>S2: GET /health
    S2--xLB: No response
    Note over LB: Server 2 marked<br/>unhealthy
    LB->>S2: GET /health (keep probing)
    S2-->>LB: 200 OK
    Note over LB: Server 2 back<br/>in rotation

Two related features are worth enabling where available: slow start, which ramps traffic up gradually to a newly healthy server, and connection draining, which lets in-flight requests finish before a server is removed.

How to Choose the Right Algorithm

Start from your constraints rather than from a favorite algorithm:

flowchart TD
    A[Choose an algorithm] --> Q1{"Must the same client or key reach the same server?"}
    Q1 -->|"Yes, pool changes often"| CH[Consistent hashing]
    Q1 -->|"Yes, pool is stable"| IP[IP hash]
    Q1 -->|No| Q2{"Are the servers identical?"}
    Q2 -->|Yes| Q3{"Do request durations vary widely?"}
    Q2 -->|No| Q4{"Do request durations vary widely?"}
    Q3 -->|No| RR[Round robin]
    Q3 -->|Yes| LC[Least connections]
    Q4 -->|No| WRR[Weighted round robin]
    Q4 -->|Yes| WLC[Weighted least connections]

Two extra rules of thumb sit outside the flowchart:

• If you run many load balancers in front of a large pool, consider power of two choices to avoid herding.

• If latency is your top priority and your load balancer supports it, least response time is worth testing.

Common Pitfalls

• Using round robin with long-lived connections. One server can end up holding all the heavy WebSocket sessions.

• Treating connection count as load. Ten idle connections and ten busy ones look identical to least connections.

• Hashing on a shared IP. Large offices and mobile carriers can funnel thousands of users onto one server.

• Skipping failure tests. Remove a server on purpose and watch how traffic redistributes before it happens in production.

• Forgetting stickiness has a cost. Affinity can hide a stateful design. Where possible, keep sessions in a shared store so any server can serve any user.

Final Thoughts

There is no universally best load balancing algorithm. Round robin is the dependable default for uniform, stateless workloads. Weighted variants handle mixed hardware. Least connections and its weighted version shine when request durations vary. Hash-based methods give you stickiness and cache-friendliness, and power of two choices scales gracefully when many balancers share one pool.

Start simple, measure real traffic, and move to a smarter algorithm only when the data shows a problem it would solve. And whichever you choose, pair it with solid health checks. A good algorithm picks the best server, but health checks make sure that server is actually alive.

Which load balancing algorithm do you rely on in production, and why? Share your experience in the comments.