Rate limiting via Nginx

I needed to integrate rate limiting into a relatively small service that complements a monolith I was working on. My initial thought was to apply it at the application layer, as it seemed to be the simplest route.

Plus, I didn't want to muck around with load balancer configurations, and there's no shortage of libraries that allow me to do this quickly in the app. However, this turned out to be a bad idea. In the event of a DDoS attack or thundering herd incident, even if the app rejects the influx of inbound requests, the app server workers still have to do a minimal amount of work.

Also, ideally, rate limiting is an infrastructure concern; your app should be oblivious to it. Implementing rate limiting in a layer in front of your app prevents rogue requests from even reaching the app server in the event of an incident. So, I decided to spend some time investigating how to do it at the load balancer layer. Nginx makes rate limiting straightforward and the system was already using it as a reverse proxy.

For the initial pass, I chose to go with the default Nginx settings, avoiding any additional components like a Redis layer for centralized rate limiting.

App structure

For this demo, I'll proceed with a simple hello-world server written in Go. Here's the app directory:

The main.go file exposes the server at the /greetings endpoint on port 8080:

If you run the server with the go run main.go command and make a curl request to it, it'll give you the following JSON output:

Now, we want to set up the rate limiter in the reverse proxy layer so that it will reject requests when the inbound request rate exceeds 50 req/sec.

Nginx config

The Nginx config lives in the nginx directory and consists of two config files:

The nginx.conf file is the core configuration file. It's where you define the server's global settings, like how many worker processes to run, where to store log files, rate limiting policies, and overarching security protocols.

Then there's the default.conf file, which is typically more focused on the configuration of individual server blocks or virtual hosts. This is where you get into the specifics of each website or service you're hosting on the server. Settings like server names, SSL certificates, and specific location directives are defined here. It's tailored to manage the nitty-gritty of how each site or application behaves under the umbrella of the global settings set in nginx.conf.

You can have multiple .conf files like default.conf and all of them are included in the nginx.conf file.

nginx.conf

Here's how the nginx.conf looks:

In the nginx.conf file, you'll find two main sections: events and http. Each of these serves different purposes in the setup.

Events block

This section defines settings for the events block, specifically the worker_connections directive. It sets the maximum number of connections that each worker process can handle concurrently to 1024.

HTTP block

The http block contains directives that apply to HTTP/S traffic.

• Set the rate limiting policy (limit_req_zone directive)

  This line sets up rate limiting policy using three parameters:

  • Key ($binary_remote_addr): This is the client's IP address in a binary format. It's used as a key to apply the rate limit, meaning each unique IP address is subjected to the rate limit specified.

  • Zone (zone=mylimit:10m): This defines a shared memory zone named mylimit with a size of 10 megabytes. The zone stores the state of each IP address, including how often it has accessed the server. Approximately 160,000 IP addresses can be tracked with this size. If the zone is full, Nginx will start removing the oldest entries to free up space.

  • Rate (rate=50r/s): This parameter sets the maximum request rate to 50 requests per second for each IP address. If the rate is exceeded, additional requests may be delayed or rejected.

• Include the default.conf file

  This directive instructs Nginx to include additional server configurations - like default.conf - from the /etc/nginx/conf.d/ directory. This modular approach allows for better organization and management of server configurations.

default.conf

The default.conf file, included in the previously discussed nginx.conf, mainly configures a server block in Nginx. We'll use the rate limiting policy defined there in the default.conf file. Here's the content:

This file currently contains a server block where we employ the rate limiting policy and set up the reverse proxy.

Server Block

This section defines the server block, with Nginx listening on port 80, the default port for HTTP traffic. The default_server parameter indicates that this server block should be used if no other matches are found.

Custom error handling

By default, when a client experiences rate limiting, the server returns an HTTP 503 error with an HTML page. But we want to return 429 (Too many requests) error code with an error message in a JSON payload. This section does that.

Location block

The location / block applies to all requests to the root URL and its subdirectories.

• Apply the rate limiting policy

  These directives enforce the rate limiting policy set in nginx.conf. The limit_req directive uses the previously defined mylimit zone. The burst parameter allows a burst of 10 requests above the set rate before enforcing the limit. The nodelay option ensures that excess requests within the burst limit are processed immediately without delay. limit_req_status sets the HTTP status code for rate-limited requests to 429.

• Configure the proxy

  These lines configure Nginx to act as a reverse proxy. Requests to this server are forwarded to an application server running on http://app:8080. The directives also handle HTTP headers to properly manage the connection and caching between the client, reverse proxy, and backend application server.

Containerize everything

The Dockerfile builds the hello-world service:

Then we orchestrate the app with reverse proxy in the docker-compose.yml file:

The docker-compose file defines two services: app and nginx. The app service exposes port 8080, meaning the app will be accessible on this port from outside the Docker environment.

The nginx service sits in front of the app and is configured to expose port 80. All the external requests will hit the default port 80 where the reverse proxy will relay the request to the backend app. The custom Nginx configuration volumes are mounted in the volumes section.

Take it for a spin

Navigate to the app directory and start the system with the following command:

Now make 200 concurrent curl requests to see the rate limiter in action:

This returns:

See the deployed service in action (might not be available later):

This will print the same output as the local service.

Nginx uses the leaky bucket algorithm to enforce the rate limiting, where requests arrive at the bucket at various rates and leave the bucket at fixed rate.

Find the complete implementation on GitHub.

Fin!