Building a Lightweight Load Balancer in Go: A Deep Dive into Load Balancing Strategies

At Developer’s Coffee, we constantly explore innovative tools and approaches for building efficient and scalable systems. Today, we’re diving into the concept of load balancing, an essential technique for distributing network traffic across multiple servers to ensure performance, fault tolerance, and high availability. We’ll explore a lightweight load balancer written in Go, with custom strategies like Round-Robin and Hashed balancing.

The complete code is available in our GitHub repository: dynamic-load-balancer.

Understanding Load Balancers

A load balancer is a core component in any distributed system that helps distribute incoming network traffic across multiple servers. This prevents overloading a single server, ensures fault tolerance, and improves overall system availability. Whether it’s microservices, APIs, or websites, load balancers help in achieving seamless scalability.

In simple terms, load balancers act as intermediaries, forwarding incoming requests to backend servers based on predefined balancing strategies.

Why Use a Load Balancer?

• Fault Tolerance: If one server goes down, the load balancer can redirect traffic to healthy servers.
• Prevent Overloading: Distributes traffic evenly to avoid overworking any single server.
• Scalability: Easily scales as the backend server fleet grows.

Key Features of Our Load Balancer

Our Go-based load balancer comes with some essential features, making it suitable for learning and customizing as per your needs in microservices architectures or distributed systems:

• Multiple Load Balancing Strategies: Supports Round-Robin and Hashed balancing out-of-the-box.
• Health Checks: Monitors backend server health and routes traffic accordingly.
• Dynamic Server Registration: Supports adding or removing backend servers dynamically.

Designing the Load Balancer

1. Backend Server Setup

The core idea of our load balancer is to efficiently route HTTP requests to backend servers. The backends are defined as follows:

type Backend struct {
	Host        string
	Port        int
	IsHealthy   bool
	NumRequests int
}

backends := []*common.Backend{
	{Host: "localhost", Port: 8081, IsHealthy: true},
	{Host: "localhost", Port: 8082, IsHealthy: true},
	{Host: "localhost", Port: 8083, IsHealthy: true},
}

In this snippet, we have a collection of backend servers running on different ports of the same host. You can extend this by configuring multiple hosts in real-world scenarios.

2. Load Balancing Strategies: Round-Robin & Hashed

Our load balancer supports Round-Robin and Hashed balancing strategies. Each strategy is encapsulated within a strategy interface. Here’s how the Round-Robin strategy works:

type RRBalancingStrategy struct {
	Index    int
	Backends []*common.Backend
}

func (s *RRBalancingStrategy) GetNextBackend(_ common.IncomingReq) *common.Backend {
	s.Index = (s.Index + 1) % len(s.Backends)
	return s.Backends[s.Index]
}

This algorithm ensures that each server takes turns receiving traffic, which is ideal for evenly distributing loads across servers. On the other hand, Hashed balancing uses consistent hashing based on incoming requests:

type HashedBalancingStrategy struct {
	OccupiedSlots []int
	Backends      []*common.Backend
}

func hash(s string) int {
	// Hash logic
	return sum % 19
}

With hashed balancing, you can distribute requests based on a hash of the request content, such as a URL or user ID, ensuring sticky sessions—where the same request types are consistently routed to the same servers.

3. Backend Health Monitoring

Another crucial component is ensuring the load balancer routes traffic only to healthy servers. Backend servers have a IsHealthy flag, which gets updated through health checks. The balancing strategy ensures only healthy backends receive traffic:

if !backend.IsHealthy {
	// Skip this backend
	continue
}

Handling Requests Efficiently

The core functionality of the load balancer is to act as a proxy that forwards incoming requests to a selected backend. The request handling mechanism is as follows:

func (lb *LB) proxyHTTP(w http.ResponseWriter, req *http.Request) {
	backend := lb.strategy.GetNextBackend(common.IncomingReq{})
	backendURL := fmt.Sprintf("http://%s:%d%s", backend.Host, backend.Port, req.RequestURI)

	resp, err := http.Get(backendURL)
	if err != nil {
		http.Error(w, "Backend is unavailable", http.StatusServiceUnavailable)
		return
	}
	defer resp.Body.Close()

	w.WriteHeader(resp.StatusCode)
	io.Copy(w, resp.Body)
}

This function selects a backend server using the balancing strategy, forwards the client’s request to the backend, and then returns the response to the client.

Running the Load Balancer

To run the load balancer, follow these steps:

1. Clone the repository from GitHub:bashCopy codegit clone https://github.com/DevelopersCoffee/dynamic-load-balancer cd dynamic-load-balancer
2. Start the backend servers:bashCopy codego run servers/server.go -port=8081 go run servers/server.go -port=8082 go run servers/server.go -port=8083
3. Start the load balancer:bashCopy codego run main.go
4. Test the load balancer by sending HTTP requests to http://localhost:9090:bashCopy codecurl http://localhost:9090

You should see the load balancer route requests to different backend servers based on the chosen balancing strategy.

Scaling the Load Balancer

Scaling a load balancer often involves scaling the backend servers and ensuring that the load balancer itself can handle an increased number of requests. In a production environment, you might want to introduce distributed load balancing, DNS-based routing, or automated health checks.

DNS Scaling

As shown in the slide provided (see image), one popular scaling approach is using DNS-based load balancing. This involves using DNS to resolve to multiple load balancer instances:

LB Server 1  ->  10.0.0.1
LB Server 2  ->  10.0.0.2
LB Server 3  ->  10.0.0.3

This method, as depicted in the diagram, helps in balancing traffic across multiple load balancers, ensuring that no single instance is overwhelmed by requests.

Orchestration

Another interesting concept from the slide involves the orchestrator (as seen in the CoreDNS diagram). The orchestrator is responsible for maintaining the health of load balancers and backend servers, as well as scaling them up or down based on traffic metrics (like CPU or memory usage). Using Prometheus for monitoring allows us to collect real-time metrics to decide when to scale.

Conclusion

Building a custom load balancer from scratch in Go is a fantastic way to learn about traffic distribution and scaling strategies for distributed systems. The provided lightweight load balancer can be further extended to include more complex strategies such as Least Connections, Weighted Round Robin, and Dynamic Health Checks.

The flexibility offered by this project makes it an ideal starting point for understanding and implementing custom load balancers in your own microservices architecture.

For the full implementation and code details, visit the GitHub repository.

Stay tuned to Developer’s Coffee for more insightful posts on building scalable systems and exploring advanced Go-based projects!