Load balancers explained — when do you actually need one?

A load balancer is a traffic “traffic cop” that sits in front of one or more servers (often called backends or upstreams) and decides where each incoming connection or request should go. Instead of users hitting `app-server-1` directly, they hit a single stable entry point (a DNS name or IP), and the load balancer distributes traffic across healthy servers.

For beginners, load balancers can sound like something you only need at huge scale. In reality, they become useful earlier than many people expect—not just for handling high traffic, but for reliability, maintenance, and clean growth.

---

What load balancers actually do

At a high level, a load balancer can provide:

One stable address for many servers

Users access `https://example.com`. Behind the scenes you might have:

• `10.0.1.10` (app1)
• `10.0.1.11` (app2)
• `10.0.1.12` (app3)

The load balancer routes each new request/connection to one of these.

Health checks and automatic failover

Load balancers can continuously check if a backend is healthy (for example, by calling `/health` or attempting a TCP connection). If a server stops responding, the load balancer stops sending it traffic automatically.

This is one of the most practical benefits: your site can keep working even when a server fails.

Scaling without changing your public endpoint

When you need more capacity, you add more servers and register them as backends. Clients don’t need to know anything changed.

Maintenance without downtime

Want to deploy a new version or reboot a machine?

• Take one backend out of rotation
• Update it
• Put it back
• Repeat

This is sometimes called rolling maintenance or draining connections.

Optional: TLS termination and HTTP features

Depending on the type of load balancer, it may also:

• Handle HTTPS certificates (“TLS termination”)
• Add headers (like `X-Forwarded-For`)
• Route by host/path (`api.example.com` vs `www.example.com`, `/images/` vs `/app/`)
• Provide rate limiting or basic WAF-like protections

These capabilities depend heavily on whether you’re using a Layer 4 or Layer 7 load balancer.

---

Layer 4 vs Layer 7 load balancing (L4 vs L7)

The “layer” refers to the OSI networking model. You don’t need to memorize it—just focus on what the load balancer can “see” and decide on.

Layer 4 (Transport) load balancers

L4 operates on TCP/UDP connections. It generally does not understand HTTP semantics (URLs, headers, cookies).

What L4 can do well:

• Forward TCP connections (e.g., HTTPS, SSH, databases)
• Forward UDP (e.g., DNS, some game traffic)
• Very fast and low overhead
• Often preserves the connection end-to-end (pass-through)

Decisions are based on:

• Source/destination IP and port
• Connection state
• Backend availability

Example use cases:

• Balancing traffic to multiple web servers without inspecting HTTP
• Load balancing a TCP service like Redis (with care) or a custom protocol
• Very high throughput, minimal features

Layer 7 (Application) load balancers

L7 understands application protocols, most commonly HTTP/HTTPS. It can make routing decisions based on content of the request.

What L7 can do well:

• Route based on hostname: `api.example.com` → API servers, `www.example.com` → frontend
• Route based on path: `/static/` → static servers, `/` → app servers
• Sticky sessions via cookies (when needed)
• TLS termination: decrypt HTTPS at the load balancer and talk HTTP to backends
• Header manipulation, redirects, rewrites, caching (depending on software)

Example use cases:

• Microservices or multiple apps behind one domain
• Blue/green deployments or canary releases (send 5% of users to new version)
• Centralized certificate management

Trade-off: L7 has more overhead and complexity, but gives you powerful routing and observability.

---

Common load balancing algorithms (and when to use them)

A load balancer needs a method to choose which backend receives traffic. Common algorithms include:

Round-robin

Each new request/connection goes to the next server in a rotating list.

Pros:

• Simple and effective when servers are similar
• Predictable distribution

Cons:

• Doesn’t account for uneven load (one server might be slower or handling heavy requests)

Good for: uniform backends with similar performance, or as a default starting point.

Least connections

Sends new traffic to the backend with the fewest active connections.

Pros:

• Adapts to uneven load better than round-robin
• Useful when requests vary in duration (some users keep connections open longer)

Cons:

• “Connections” aren’t always equal work (one connection could be CPU-heavy)
• Requires tracking active connection counts

Good for: long-lived connections (WebSockets), mixed request times, uneven client behavior.

Weighted variants (weighted round-robin / weighted least connections)

Assign weights so a bigger server gets more traffic.

Example: If you have two servers, one twice as powerful, you might set weights 2 and 1.

Good for: mixed instance sizes or gradual scaling.

IP hash / consistent hashing

Route a client consistently to the same backend based on source IP (or another key).

Pros:

• “Sticky” behavior without cookies
• Useful for caching or stateful services

Cons:

• Doesn’t rebalance well when backends change unless using consistent hashing
• Source IP may be obscured by NAT or proxies

Good for: certain caching designs or legacy stateful apps (but try to avoid server-side session state when possible).

---

When do you actually need a load balancer?

You “need” one when it meaningfully reduces risk or enables capabilities you can’t easily get otherwise. Common trigger points:

1) You need high availability (HA)

If your application running on a single server is a business dependency, that server is a single point of failure. A load balancer in front of at least two backends lets you survive:

• VM/hardware failure
• kernel panic / reboot
• a bad deploy on one node

Rule of thumb: if downtime costs more than the extra infrastructure, you’re in load balancer territory.

2) You want to scale horizontally (add more servers)

Vertical scaling (bigger server) works until it doesn’t—cost, hardware limits, or risk concentration. When you add a second app server, you need something to split traffic. A load balancer is the standard solution.

3) You need zero/low-downtime deployments and maintenance

Without a load balancer, taking a server down typically means downtime. With one, you can:

• drain connections from `app1`
• deploy
• add it back
• repeat for `app2`

4) You host multiple services behind one public endpoint

An L7 load balancer (or reverse proxy) can route:

• `example.com` → frontend
• `api.example.com` → API
• `example.com/static` → CDN origin or static service

This reduces public IP sprawl and centralizes TLS certs and routing logic.

5) You’re handling spikes or unpredictable traffic

Auto-scaling groups and cloud load balancers work well together: when traffic surges, new instances come online and the load balancer starts sending them traffic after health checks pass.

6) You need WebSockets or long-lived connections at scale

Long-lived connections can create uneven load. Least-connections or careful tuning becomes important, and a load balancer provides that control.

---

When you probably don’t need one (yet)

Load balancers add cost and operational surface area. You may not need one if:

• You have a personal project or low-stakes app where occasional downtime is acceptable.
• One server can handle your traffic comfortably and you can restore quickly.
• A simple reverse proxy for HTTPS and routing is sufficient (sometimes “reverse proxy” and “load balancer” overlap, but if you only have one backend, you aren’t really balancing).

That said, many people adopt a load balancer early specifically to standardize TLS termination and routing, then add more backends later.

---

Practical examples

Example A: Small app, growing traffic

Stage 1 (single server):

• `example.com` → 1 VM running Nginx + app

Problem: you need to deploy updates and avoid downtime.

Stage 2 (add load balancer + 2 app servers):

• `example.com` → load balancer → `app1`, `app2`

Now you can deploy to `app1` while `app2` serves users.

Example B: Path-based routing with L7

You have:

• frontend app
• API service
• admin panel

An L7 load balancer routes:

• `/` → frontend servers
• `/api/*` → API servers
• `/admin/*` → admin servers (maybe protected further)

Example C: Different server sizes with weights

You add a larger instance to handle more load:

• `app1` (small) weight 1
• `app2` (large) weight 3

The load balancer sends about 75% of traffic to `app2`.

---

Popular options: HAProxy, Nginx, and cloud load balancers

HAProxy

HAProxy is a dedicated, high-performance load balancer widely used in production.

Strengths:

• Excellent L4 and L7 capabilities
• Very strong performance and reliability
• Rich health checks and routing rules
• Great observability/stats options

Common use: serious self-managed load balancing, especially when you want fine control.

Nginx (and Nginx Open Source)

Nginx is often used as a web server and reverse proxy, and it can also load balance.

Strengths:

• Very common and well-understood
• Great at HTTP(S) reverse proxying (L7)
• Useful when you also need static file serving, caching, compression, etc.

Common use: “edge” reverse proxy that also balances between multiple application servers.

Cloud load balancers (AWS, GCP, Azure, etc.)

Each major cloud provides managed load balancers (names differ by provider: Application LB, Network LB, HTTP(S) LB, etc.).

Strengths:

• Managed scaling and high availability
• Integrated with auto-scaling, health checks, certificates, logging
• Less operational burden than self-hosting

Trade-offs:

• Cost can be higher than running your own VM-based balancer
• Some features are provider-specific
• Debugging can feel less transparent compared to self-managed tools

Common use: production workloads where you value managed HA and integration over full low-level control.

---

Key considerations before adopting a load balancer

Statelessness and sessions

If your app stores sessions in server memory, users might “lose” their session when routed to a different backend. Options:

• Move sessions to a shared store (Redis, database) — preferred
• Use sticky sessions — sometimes acceptable, but can reduce flexibility during failures

Health checks that reflect real readiness

A good `/health` endpoint should indicate whether the app is actually able to serve traffic (dependencies reachable, migrations complete, etc.), not just “process is running.”

Timeouts and connection limits

Load balancers introduce settings like idle timeouts and max connections. Defaults may not match your app (especially for streaming, uploads, WebSockets).

---

A practical rule of thumb

You should seriously consider a load balancer when:

• you need two or more servers behind one endpoint, or
• you need high availability and safe deployments, or
• you want centralized TLS + routing for multiple services.

If you’re running a single server and downtime is acceptable, focus first on backups, monitoring, and a simple reverse proxy setup. But as soon as reliability or growth becomes a goal, a load balancer is one of the most straightforward, proven building blocks you can add.