Build a Load Balancer

Difficulty: Hard · Category: Load Balancer · Languages: Java 17, Python 3.11, Go 1.22, Node.js 20, TypeScript 5.x

Load Balancer

Background

Load balancing is one of the most fundamental patterns in distributed systems. A load balancer distributes incoming requests across multiple backend servers to ensure no single server becomes a bottleneck. Production load balancers like NGINX, HAProxy, and cloud load balancers use sophisticated algorithms to optimize throughput and minimize latency.

Building one from scratch teaches you about algorithm design, health checking, connection tracking, and concurrent state management.

Your Task

Implement the LoadBalancerContract interface. Your implementation must:

1. Persist all server state (no in-process-only state).
2. Support four routing algorithms: round-robin, weighted, least-connections, and random.
3. Be health-aware: unhealthy servers must be excluded from routing.
4. Track active connections per server for the least-connections algorithm.
5. Be thread-safe: concurrent calls must be safe and consistent.
6. Return null when no healthy servers are available.

Recommended infrastructure: redis — Redis atomic counters and sorted sets are ideal for real-time routing state.

How It Works

The platform injects your chosen infrastructure SDK(s) via constructor. For example, if you select Redis:

public class MyLoadBalancer implements LoadBalancerContract {
    private final RedisClient redis;

    public MyLoadBalancer(RedisClient redis) {
        this.redis = redis;
    }

    // ... implement methods using redis.*
}

See the SDK Docs tab for the full API reference of your chosen system.

Interface

public interface LoadBalancerContract {
    void addServer(String serverId, int weight);
    void removeServer(String serverId);
    String getNextServer(String algorithm);
    void recordConnection(String serverId);
    void releaseConnection(String serverId);
    Map<String, Object> getServerStats(String serverId);
    List<String> getActiveServers();
    void markUnhealthy(String serverId);
    void markHealthy(String serverId);
}

Method Details

Constraints

• serverId is a non-null, non-empty string
• weight ≥ 1
• algorithm is one of: "round-robin", "weighted", "least-connections", "random"
• Performance target: ≥ 10 000 ops/sec with p99 latency ≤ 5 ms

Examples

addServer("s1", 5)
addServer("s2", 3)
addServer("s3", 2)

getNextServer("round-robin") → "s1"
getNextServer("round-robin") → "s2"
getNextServer("round-robin") → "s3"
getNextServer("round-robin") → "s1"   // wraps

markUnhealthy("s1")
getNextServer("round-robin") → "s2"   // skips s1

recordConnection("s2")
recordConnection("s2")
recordConnection("s3")
getNextServer("least-connections") → "s3"   // 1 vs 2 connections

Hints

• Store server metadata in a hash/map keyed by server ID.
• Use a sorted data structure for the round-robin index.
• For least-connections, sort by active connection count.
• Use atomic increment/decrement for connection counter updates.
• For weighted selection, expand the server pool proportionally or use weighted random selection.

Infrastructure your code runs against

• Redis 7.x

How your solution is graded

Your score combines functional correctness (50%), performance (30%) and code quality (20%).

• Target throughput: 10,000 operations/second
• Target p99 latency: 5 ms
• Load test concurrency: 8 concurrent clients
• Load test duration: 10 seconds
• Maximum error rate: 0.1%

Practice this problem

• Solve Build a Load Balancer in the browser
• Read the walkthrough

Related LLD problems

• Build a Cab Booking System
• Build a Railway Reservation System
• Build a URL Shortener