Load balancers and reverse proxies

A forward proxy sits in front of clients, making requests on their behalf (a corporate proxy every employee's traffic routes through, a VPN). A reverse proxy sits in front of servers, receiving requests on their behalf and forwarding them to whichever backend actually handles them — the client only ever sees the proxy, never the real server directly. Nginx and Caddy are common reverse proxies used exactly this way.

A load balancer is a reverse proxy whose specific purpose is spreading incoming requests across multiple backend servers — for two reasons that matter separately: scale (one server can't handle all the traffic) and availability (if one server goes down, traffic keeps flowing to the others, completely invisibly to the client).

A load balancer periodically pings each backend (often a dedicated /health endpoint) and stops routing to any server that fails enough consecutive checks — this is the actual mechanism behind "the app kept working even though a server crashed." A server that's technically running but stuck (deadlocked, out of memory, unable to reach its database) should fail its health check deliberately rather than report healthy — a /health endpoint that only checks "is the process alive" instead of "can this process actually serve requests" is a common gap that lets a load balancer keep sending traffic to a server that's alive but useless.

A Layer 4 load balancer only looks at IP and port — fast, but blind to what's actually inside the request. A Layer 7 load balancer understands HTTP itself, and can route based on the actual request — /api/* to one set of servers, /images/* to another, or route differently based on a header or cookie. Most application-level load balancing today (AWS's Application Load Balancer, Nginx configured as a reverse proxy) operates at Layer 7 specifically because content-aware routing is usually worth the extra processing cost.

As mentioned in the HTTPS lesson, a load balancer is very often exactly where TLS gets terminated — decrypting once at the edge, then forwarding plain HTTP to backend servers over a private network. This is also frequently where rate limiting, DDoS mitigation, and request logging happen centrally, rather than being duplicated in every single backend server.