{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreig2wazjmjbfyiik6labby5z5rht2in3efr3pfmqoqjghr6xulkgiu",
    "uri": "at://did:plc:25rdn5elo5izoxrmtis34zuk/app.bsky.feed.post/3mpkmuaf32b32"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreifhyibvhug2n4gp27bpxix5l6ppvrlceunarlfi26vjqrwtzn5ewa"
    },
    "mimeType": "image/webp",
    "size": 65468
  },
  "path": "/apikumo/the-token-bucket-algorithm-build-server-side-api-rate-limiting-in-40-lines-25f6",
  "publishedAt": "2026-07-01T03:18:35.000Z",
  "site": "https://dev.to",
  "tags": [
    "api",
    "webdev",
    "tutorial",
    "node",
    "APIKumo"
  ],
  "textContent": "##  The Token Bucket Algorithm: Server-Side API Rate Limiting in ~40 Lines\n\nPlenty of tutorials teach you how to _survive_ someone else's rate limit with retries and backoff. Far fewer show you how to _build_ one. If you run an API, you need rate limiting on your side too — to protect your database from a runaway client, keep one noisy tenant from starving everyone else, and give abusive traffic a polite `429` instead of a melted server.\n\nThe cleanest algorithm for the job is the **token bucket**. Let's implement it from scratch, then make it production-ready.\n\n##  How token bucket works\n\nPicture a bucket that holds up to `capacity` tokens. Every request removes one token. The bucket refills at a steady `refillRate` (tokens per second), up to its cap. If a request arrives and the bucket is empty, it's rejected.\n\nThis gives you two useful properties at once:\n\n  * A **sustained rate** — the long-run average, set by `refillRate`.\n  * A **burst allowance** — clients can spend the whole bucket at once, set by `capacity`.\n\n\n\nThat burst tolerance is why token bucket feels fair. A user who's been quiet for a minute can fire off a batch of requests without being punished for it.\n\n##  A minimal implementation\n\nHere's a self-contained bucket in JavaScript. No dependencies, no timers — we compute refill lazily based on elapsed time, which is both simpler and more accurate than a background interval.\n\n\n\n    class TokenBucket {\n      constructor(capacity, refillRatePerSec) {\n        this.capacity = capacity;\n        this.refillRate = refillRatePerSec;\n        this.tokens = capacity;\n        this.lastRefill = Date.now();\n      }\n\n      _refill() {\n        const now = Date.now();\n        const elapsedSec = (now - this.lastRefill) / 1000;\n        this.tokens = Math.min(\n          this.capacity,\n          this.tokens + elapsedSec * this.refillRate\n        );\n        this.lastRefill = now;\n      }\n\n      take(cost = 1) {\n        this._refill();\n        if (this.tokens >= cost) {\n          this.tokens -= cost;\n          return { ok: true, remaining: Math.floor(this.tokens) };\n        }\n        const deficit = cost - this.tokens;\n        const retryAfter = Math.ceil(deficit / this.refillRate);\n        return { ok: false, remaining: 0, retryAfter };\n      }\n    }\n\n\nNotice `take()` returns a `retryAfter` in seconds when it rejects — that's the value your clients need, and we'll hand it straight to them.\n\n##  Wiring it into Express\n\nGive each API key its own bucket and enforce it as middleware. Ten requests/second sustained, with a burst of up to 20:\n\n\n\n    const buckets = new Map();\n\n    function rateLimit(req, res, next) {\n      const key = req.header(\"x-api-key\") || req.ip;\n      if (!buckets.has(key)) {\n        buckets.set(key, new TokenBucket(20, 10)); // capacity 20, 10/s\n      }\n\n      const result = buckets.get(key).take();\n      res.set(\"X-RateLimit-Limit\", \"10\");\n      res.set(\"X-RateLimit-Remaining\", String(result.remaining));\n\n      if (!result.ok) {\n        res.set(\"Retry-After\", String(result.retryAfter));\n        return res.status(429).json({\n          type: \"https://example.com/errors/rate-limit\",\n          title: \"Too Many Requests\",\n          detail: `Rate limit exceeded. Retry in ${result.retryAfter}s.`,\n        });\n      }\n      next();\n    }\n\n    app.use(\"/api\", rateLimit);\n\n\nTwo details that matter: always emit the `Retry-After` header on a `429` so well-behaved clients know exactly when to come back, and return a structured error body (this one follows RFC 9457 Problem Details) instead of a bare status code.\n\n##  Making it survive production\n\nThe in-memory `Map` above works great for a single process, but it breaks the moment you scale horizontally — each instance keeps its own bucket, so a client behind a load balancer effectively gets N× the limit. Move the state to Redis and every instance shares one bucket:\n\n\n\n    // Atomic check-and-decrement via a Lua script (runs server-side in Redis)\n    const LUA = `\n    local tokens = tonumber(redis.call('get', KEYS[1]) or ARGV[1])\n    local last   = tonumber(redis.call('get', KEYS[2]) or ARGV[4])\n    local now    = tonumber(ARGV[4])\n    tokens = math.min(ARGV[1], tokens + (now - last) / 1000 * ARGV[2])\n    if tokens >= 1 then\n      redis.call('set', KEYS[1], tokens - 1)\n      redis.call('set', KEYS[2], now)\n      return 1\n    end\n    redis.call('set', KEYS[2], now)\n    return 0\n    `;\n    // KEYS = [tokensKey, timestampKey]; ARGV = [capacity, refillRate, cost, now]\n\n\nRunning the refill-and-take logic as a single Lua script keeps it **atomic** — no race between reading the token count and decrementing it, even under thousands of concurrent requests.\n\nA few more things worth doing before you ship: bound your key map (or set a Redis TTL) so idle clients get evicted, decide whether the limit is per-key, per-IP, or per-endpoint, and consider a higher `cost` for expensive routes so a single heavy query counts as several cheap ones.\n\n##  Test it before your users do\n\nThe tricky part of rate limiting isn't the happy path — it's the boundary. Does the 21st burst request actually get a `429`? Is `Retry-After` accurate? Does the bucket refill on schedule? You want to fire controlled bursts and inspect the exact headers that come back, which is exactly the kind of thing APIKumo makes easy: send repeated requests against your endpoint, watch the `X-RateLimit-Remaining` and `Retry-After` headers tick down in real time, and save the whole scenario so you can re-run it every time you touch your limiter. Build the bucket, then prove it behaves.",
  "title": "The Token Bucket Algorithm: Build Server-Side API Rate Limiting in ~40 Lines"
}