Load Balancer

Single Point of Failure

• If this single web server goes down, the user will no longer be able to access the website.
• If many users try to access the server simultaneously and it's unable to handle the load, they may experience slow load times or may be unable to connect at all.

g)

• This single point of failure can be mitigated by introducing a load balancer and at least one additional web server on the backend.
• Typically, all of the backend servers will supply identical content so that users receive consistent content regardless of which server responds.

How's it working

• It helps to distribute load across multiple resources according to some metric
• LB also keeps track of the status of all the resources while distributing requests.
• If a server is not available to take new requests or is not responding or has elevated error rate, LB will stop sending traffic to such a server.

We can add load balancer at three places:

• Between the user and the web server
• Between web servers and an internal platform layer, like application servers or cache servers
• Between internal platform layer and database.

What kind of traffic can load balancers handle?

• HTTP — The Load Balancer sets the headers to give the backends information about the original request.
• HTTPS — Same as HTTP but with the addition of encryption.
• TCP — TCP traffic can also be balanced.
• UDP

How does the load balancer choose the backend server?

Health Checks

• LB checks regularly attempt to connect to backend servers
• If a server fails a health check, it's removed from the pool.

Load Balancing Algorithms

• Round Robin — Round Robin means servers will be selected sequentially.
• Least Connections — Least Connections means the load balancer will select the server with the least connections
• Source — based on a hash of the source IP of the request, such as the visitor's IP address. This method ensures that a particular user will consistently connect to the same server.

How to implement

Smart Clients

• A smart client will take a pool of service hosts and balances load across them.
• It also detects hosts that are not responding to avoid sending requests their way.
• Smart clients also have to detect recovered hosts, deal with adding new hosts, etc.

Hardware Load Balancers

• The most expensive–but very high performance–solution to load balancing is to buy a dedicated hardware load balancer
• Hardware solutions are very expensive, and they are not trivial to configure.

Software Load Balancers

• Load balancer can be placed between client and server or between two server side layers.
• For most systems, we should start with a software load balancer and move to smart clients or hardware load balancing as need arises.