Load Balancer
Single Point of Failure
- If this single web server goes down, the user will no longer be able to access the website.
- If many users try to access the server simultaneously and it's unable to handle the load, they may experience slow load times or may be unable to connect at all.
g)
- This single point of failure can be mitigated by introducing a load balancer and at least one additional web server on the backend.
- Typically, all of the backend servers will supply identical content so that users receive consistent content regardless of which server responds.
How's it working
- It helps to distribute load across multiple resources according to some metric
- LB also keeps track of the status of all the resources while distributing requests.
- If a server is not available to take new requests or is not responding or has elevated error rate, LB will stop sending traffic to such a server.
We can add load balancer at three places:
- Between the user and the web server
- Between web servers and an internal platform layer, like application servers or cache servers
- Between internal platform layer and database.
What kind of traffic can load balancers handle?
- HTTP — The Load Balancer sets the headers to give the backends information about the original request.
- HTTPS — Same as HTTP but with the addition of encryption.
- TCP — TCP traffic can also be balanced.
- UDP
How does the load balancer choose the backend server?
Health Checks
- LB checks regularly attempt to connect to backend servers
- If a server fails a health check, it's removed from the pool.
Load Balancing Algorithms
- Round Robin — Round Robin means servers will be selected sequentially.
- Least Connections — Least Connections means the load balancer will select the server with the least connections
- Source — based on a hash of the source IP of the request, such as the visitor's IP address. This method ensures that a particular user will consistently connect to the same server.
How to implement
Smart Clients
- A smart client will take a pool of service hosts and balances load across them.
- It also detects hosts that are not responding to avoid sending requests their way.
- Smart clients also have to detect recovered hosts, deal with adding new hosts, etc.
Hardware Load Balancers
- The most expensive–but very high performance–solution to load balancing is to buy a dedicated hardware load balancer
- Hardware solutions are very expensive, and they are not trivial to configure.
Software Load Balancers
- Load balancer can be placed between client and server or between two server side layers.
- For most systems, we should start with a software load balancer and move to smart clients or hardware load balancing as need arises.