{
"$type": "site.standard.document",
"canonicalUrl": "https://rednafi.com/go/rate-limiting-via-nginx/",
"description": "Implement rate limiting at the infrastructure layer with Nginx reverse proxy. Protect Go services from DDoS with leaky bucket algorithm.",
"path": "/go/rate-limiting-via-nginx/",
"publishedAt": "2024-01-06T00:00:00.000Z",
"site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
"tags": [
"Go",
"Networking",
"Docker"
],
"textContent": "I needed to integrate rate limiting into a relatively small service that complements a\nmonolith I was working on. My initial thought was to apply it at the application layer, as\nit seemed to be the simplest route.\n\nPlus, I didn't want to muck around with load balancer configurations, and there's no\nshortage of libraries that allow me to do this quickly in the app. However, this turned out\nto be a bad idea. In the event of a [DDoS attack] or [thundering herd] incident, even if the\napp rejects the influx of inbound requests, the app server workers still have to do a\nminimal amount of work.\n\nAlso, ideally, rate limiting is an infrastructure concern; your app should be oblivious to\nit. Implementing rate limiting in a layer in front of your app prevents rogue requests from\neven reaching the app server in the event of an incident. So, I decided to spend some time\ninvestigating how to do it at the load balancer layer. [Nginx] makes [rate limiting\nstraightforward] and the system was already using it as a reverse proxy.\n\nFor the initial pass, I chose to go with the default Nginx settings, avoiding any additional\ncomponents like a Redis layer for centralized rate limiting.\n\nApp structure\n\nFor this demo, I'll proceed with a simple hello-world server written in Go. Here's the app\ndirectory:\n\nThe main.go file exposes the server at the /greetings endpoint on port 8080:\n\nIf you run the server with the go run main.go command and make a curl request to it,\nit'll give you the following JSON output:\n\nNow, we want to set up the rate limiter in the reverse proxy layer so that it will reject\nrequests when the inbound request rate exceeds 50 req/sec.\n\nNginx config\n\nThe Nginx config lives in the nginx directory and consists of two config files:\n\nThe nginx.conf file is the core configuration file. It's where you define the server's\nglobal settings, like how many worker processes to run, where to store log files, rate\nlimiting policies, and overarching security protocols.\n\nThen there's the default.conf file, which is typically more focused on the configuration\nof individual server blocks or virtual hosts. This is where you get into the specifics of\neach website or service you're hosting on the server. Settings like server names, SSL\ncertificates, and specific location directives are defined here. It's tailored to manage the\nnitty-gritty of how each site or application behaves under the umbrella of the global\nsettings set in nginx.conf.\n\nYou can have multiple .conf files like default.conf and all of them are included in the\nnginx.conf file.\n\nnginx.conf\n\nHere's how the nginx.conf looks:\n\nIn the nginx.conf file, you'll find two main sections: events and http. Each of these\nserves different purposes in the setup.\n\nEvents block\n\nThis section defines settings for the events block, specifically the worker_connections\ndirective. It sets the maximum number of connections that each worker process can handle\nconcurrently to 1024.\n\nHTTP block\n\nThe http block contains directives that apply to HTTP/S traffic.\n\n- Set the rate limiting policy (limit_req_zone directive)\n\n \n\n This line sets up rate limiting policy using three parameters:\n - Key ($binary_remote_addr): This is the client's IP address in a binary format. It's\n used as a key to apply the rate limit, meaning each unique IP address is subjected to\n the rate limit specified.\n\n - Zone (zone=mylimit:10m): This defines a shared memory zone named mylimit with a\n size of 10 megabytes. The zone stores the state of each IP address, including how\n often it has accessed the server. Approximately 160,000 IP addresses can be tracked\n with this size. If the zone is full, Nginx will start removing the oldest entries to\n free up space.\n\n - Rate (rate=50r/s): This parameter sets the maximum request rate to 50 requests per\n second for each IP address. If the rate is exceeded, additional requests may be\n delayed or rejected.\n\n- Include the default.conf file\n \n This directive instructs Nginx to include additional server configurations - like\n default.conf - from the /etc/nginx/conf.d/ directory. This modular approach allows\n for better organization and management of server configurations.\n\ndefault.conf\n\nThe default.conf file, included in the previously discussed nginx.conf, mainly\nconfigures a server block in Nginx. We'll use the rate limiting policy defined there in the\ndefault.conf file. Here's the content:\n\nThis file currently contains a server block where we employ the rate limiting policy and\nset up the reverse proxy.\n\nServer Block\n\nThis section defines the server block, with Nginx listening on port 80, the default port for\nHTTP traffic. The default_server parameter indicates that this server block should be used\nif no other matches are found.\n\nCustom error handling\n\nBy default, when a client experiences rate limiting, the server returns an HTTP 503 error\nwith an HTML page. But we want to return 429 (Too many requests) error code with an error\nmessage in a JSON payload. This section does that.\n\nLocation block\n\nThe location / block applies to all requests to the root URL and its subdirectories.\n\n- Apply the rate limiting policy\n\n \n\n These directives enforce the rate limiting policy set in nginx.conf. The limit_req\n directive uses the previously defined mylimit zone. The burst parameter allows a\n burst of 10 requests above the set rate before enforcing the limit. The nodelay option\n ensures that excess requests within the burst limit are processed immediately without\n delay. limit_req_status sets the HTTP status code for rate-limited requests to 429.\n\n- Configure the proxy\n\n \n\n These lines configure Nginx to act as a reverse proxy. Requests to this server are\n forwarded to an application server running on http://app:8080. The directives also\n handle HTTP headers to properly manage the connection and caching between the client,\n reverse proxy, and backend application server.\n\nContainerize everything\n\nThe Dockerfile builds the hello-world service:\n\nThen we orchestrate the app with reverse proxy in the docker-compose.yml file:\n\nThe docker-compose file defines two services: app and nginx. The app service exposes\nport 8080, meaning the app will be accessible on this port from outside the Docker\nenvironment.\n\nThe nginx service sits in front of the app and is configured to expose port 80. All the\nexternal requests will hit the default port 80 where the reverse proxy will relay the\nrequest to the backend app. The custom Nginx configuration volumes are mounted in the\nvolumes section.\n\nTake it for a spin\n\nNavigate to the app directory and start the system with the following command:\n\nNow make 200 concurrent curl requests to see the rate limiter in action:\n\nThis returns:\n\nSee the deployed service in action (might not be available later):\n\nThis will print the same output as the local service.\n\nNginx uses the [leaky bucket algorithm] to enforce the rate limiting, where requests arrive\nat the bucket at various rates and leave the bucket at fixed rate.\n\nFind the [complete implementation] on GitHub.\n\nFin!\n\n\n\n\n[ddos attack]:\n https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/\n\n\n[thundering herd]:\n https://nick.groenen.me/notes/thundering-herd/\n\n[nginx]:\n https://www.nginx.com/\n\n[rate limiting straightforward]:\n https://www.nginx.com/blog/rate-limiting-nginx/\n\n[leaky bucket algorithm]:\n https://en.wikipedia.org/wiki/Leaky_bucket\n\n[complete implementation]:\n https://github.com/rednafi/nginx-ratelimit",
"title": "Rate limiting via Nginx"
}