{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/go/dummy-load-balancer/",
  "description": "Build a working round-robin load balancer in Go with goroutines and the standard library. No dependencies needed for this educational prototype.",
  "path": "/go/dummy-load-balancer/",
  "publishedAt": "2023-08-30T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Go",
    "TIL",
    "Networking"
  ],
  "textContent": "I was curious to see if I could prototype a simple load balancer in a single Go script. Go's\nstandard library and goroutines make this trivial. Here's what the script needs to do:\n\n- Spin up two backend servers that'll handle the incoming requests.\n- Run a reverse proxy load balancer in the foreground.\n- The load balancer will accept client connections and round-robin them to one of the\n  backend servers; balancing the inbound load.\n- Once a backend responds, the load balancer will relay the response back to the client.\n- For simplicity, we'll only handle client's GET requests.\n\nObviously, this won't have SSL termination, advanced balancing algorithms, or session\npersistence like you'd get with [Nginx] or [Caddy]. The point is to understand the basic\nworkflow and show how Go makes it easy to write this sort of stuff.\n\nArchitecture\n\nHere's an ASCII art that demonstrates the grossly simplified end-to-end workflow:\n\nThe diagram shows a load balancer receiving client requests on port 8080. It distributes the\nrequests between the backends, sending each request either to a backend running on port 8081\nor 8082. The selected backend processes the incoming request and returns a response through\nthe balancer. The balancer then routes the backend's response back to the client.\n\nTools we'll need\n\nHere are the stdlib tools we'll be using. Everything will live in the main.go script:\n\nA few global variables\n\nThe backends slice declares a list of backend server URLs that will be load-balanced\nbetween.\n\nThe currentBackend integer variable keeps track of the index of the backend server that\nhandled the most recent request. This will be used later to perform the round-robin load\nbalancing between the backends.\n\nThe backendMutex lock provides mutually exclusive access to the shared variables. We'll\nsee how it's used when we write the [load balancing algorithm].\n\nWriting the backend server\n\nThe backend is a simple server that'll just write a message to the connected client,\ndenoting which server is handling the request.\n\nThe startBackend function starts a backend HTTP server listening on a given port. It takes\nthe port number and a sync.WaitGroup. When startBackend returns, it calls Done() on\nthe wait group to signal the load balancer that the backend has finished processing a\nrequest. The function then registers a handler that responds with the port number. It starts\nlistening and serving on the provided port, printing any errors. We'll run this as\ngoroutines to spin up two backends on ports 8081 and 8082.\n\nSelecting backend servers in a round-robin fashion\n\nWhen a request from a client hits the load balancer, it'll need a way to figure out which\nbackend server to relay the request to. Here's how it does that:\n\nThe getNextBackend() function implements round-robin load balancing across the backends\nslice in a thread-safe manner. It works like this:\n\n- Acquire a lock on backendMutex to prevent concurrent access to the shared state.\n- Read the index of the current backend server from currentBackend.\n- Increment currentBackend to point to the next backend server. The modulo % operation\n  wraps around the index to the start when it reaches past the end.\n- Release the lock on backendMutex.\n- Return the URL of the backend at the index we read in step 2.\n\nThis allows each request handling goroutine to safely get the next backend server in a\nround-robin fashion. The mutex prevents race conditions where two goroutines try to\nread/write the shared currentBackend and backends state at the same time.\n\nThe mutex lock synchronizes access to the shared state across concurrent goroutines. This is\nnecessary because Go's HTTP server handles requests concurrently by default. Without the\nmutex, the goroutines could overwrite each other's changes to currentBackend, leading to\nincorrect load balancing behavior.\n\nWriting the load-balancing server\n\nThe load balancer itself is a server that sits between the backends and the clients. We can\nwrite its handler function as such:\n\nThe loadBalancerHandler() function forwards incoming requests from the clients to the\nbackend servers. First, it calls getNextBackend() to retrieve the next backend server. It\nthen makes an HTTP GET request to that backend using http.Get().\n\nIf there are any errors calling the backend, it just returns a 500 error to the client.\nOtherwise, it copies the backend's headers and response body into the response writer to\npropagate them back to the client.\n\nThis allows transparently load balancing each request across the backends in a round-robin\nfashion. The client only sees a single load balancer endpoint. Behind the scenes, requests\nare distributed to the dynamic backend servers based on round-robin ordering. Copying\nheaders and response bodies ensures clients get the proper responses from the chosen\nbackends.\n\nWiring them up together\n\nFinally, the main function here just starts the backend servers on port 8081-8082 and the\nload balancing server on port 8080:\n\nTaking it for a spin\n\nYou can find the [self-contained complete implementation] in this gist. Run the server in\none terminal with:\n\nIt'll print the port numbers of the backend and the load-balancing servers:\n\nThen from another console, make a few GET requests with curl:\n\nThis prints:\n\nNotice how the client requests are handled by different backends in an interleaving manner.\n\n\n\n\n[nginx]:\n    https://www.nginx.com/\n\n[caddy]:\n    https://caddyserver.com/\n\n\n[load balancing algorithm]:\n    /go/dummy-load-balancer/#selecting-backend-servers-in-a-round-robin-fashion\n\n\n[self-contained complete implementation]:\n    https://gist.github.com/rednafi/4f871286f42177f21a74a0ce038ce725",
  "title": "Dummy load balancer in a single Go script"
}