Load Balancer

Multiple instances need to look like one service

When an application runs on several servers, the client should not have to choose a specific instance itself. A Load Balancer receives traffic at a shared entry point and distributes requests across available backend nodes.

             +-> app-1
client -> LB +-> app-2
             +-> app-3

This makes it possible to increase throughput horizontally and take individual instances out of service without changing the address used by the client.

How to choose a backend

The simplest algorithm is round robin: requests are sent to different instances in turn. In real systems, selection can also account for active connection count, node weight, or other characteristics.

There is no algorithm that is always best. If requests vary greatly in duration, evenly distributing the number of requests does not necessarily produce evenly distributed actual load.

Health checks exclude unavailable nodes

A balancer needs to know where requests must not be sent. It therefore checks backend instances periodically or receives readiness information through another mechanism. A node that fails the check is temporarily removed from the target set.

It is important to distinguish “the process is running” from “the instance is ready to serve requests.” An application may be alive but still loading configuration or unable to reach a required dependency. A readiness check should therefore represent the ability to accept real traffic.

L4 and L7

Load balancing can happen at different layers. An L4 balancer makes decisions using network connections and transport information, while an L7 balancer understands an application protocol such as HTTP and can inspect the host, path, or headers.

L7 provides more flexible routing but requires more work at the application-protocol level. The right choice depends on which decisions actually need to be made at the entry point.