Load balancing — algorithms and health checks

A load balancer sits between clients and a pool of backend servers, and for every incoming request, decides which server handles it. This is the mechanism that makes horizontal scaling possible at all — without it, clients would need to somehow know about and choose between individual server addresses themselves, and there'd be no way to route around a server that crashed.

Round robin cycles through servers in fixed order, sending each new request to the next one in line. It's simple and works well when every server has equal capacity and every request costs roughly the same to handle — its blind spot is exactly when those assumptions break: if one server is slower, or some requests are far more expensive than others, round robin keeps sending equal shares of traffic to unequal capacity, overloading the slower server just as fast as the others.

Instead of a fixed rotation, least-connections routing sends each new request to whichever server currently has the fewest active connections — a proxy for "how busy is this server right now," which adapts automatically to servers handling requests at different speeds or requests that take wildly different amounts of time to finish. A server stuck on a few slow requests naturally receives fewer new ones, since its active-connection count stays elevated, without the load balancer needing to know why it's busy.

Both algorithms have weighted versions: weighted round robin and weighted least connections assign each server a weight reflecting its actual capacity (a server with twice the CPU gets weight 2, receiving roughly twice the traffic share), rather than assuming every server in the pool is identical. This matters in real deployments where the server pool isn't homogeneous — during a gradual hardware upgrade, or when cheaper and more powerful instance types are deliberately mixed for cost reasons.

None of these algorithms are safe to run without health checks — the load balancer periodically pings each server (an HTTP request to a dedicated /health endpoint is the common pattern) and stops routing traffic to any server that fails to respond correctly, automatically removing it from rotation:

Without health checks, a crashed or overloaded server keeps receiving its normal share of traffic indefinitely, actively making the outage worse for every user routed to it — the load balancer has no way to know it should stop, so it doesn't. This is what actually gives horizontal scaling its resilience benefit: an instance failing doesn't take the whole service down, because traffic reroutes automatically to instances that are still healthy.

Some setups need session affinity (sticky sessions): once a client's first request lands on a specific server, every subsequent request from that same client is routed to that same server, usually via a cookie the load balancer sets. This is genuinely useful for session data stored in server memory rather than a shared store — but it directly works against the stateless-server assumption that makes horizontal scaling clean in the first place, and it creates uneven load (a server holding many active "sticky" sessions can't shed that traffic to less-busy servers). Externalizing session state to a shared store (like Redis) instead of relying on sticky sessions is usually the better trade, when it's an option.