{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/python/limit-concurrency-with-semaphore/",
  "description": "Control concurrent async requests with Python asyncio.Semaphore to respect rate limits and prevent overwhelming APIs or services.",
  "path": "/python/limit-concurrency-with-semaphore/",
  "publishedAt": "2022-02-10T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Python",
    "Async",
    "Concurrency"
  ],
  "textContent": "I was working with a rate-limited API endpoint where I continuously needed to send\nshort-polling GET requests without hitting HTTP 429 error. Perusing the API doc, I found out\nthat the API endpoint only allows a maximum of 100 requests per second. So, my goal was to\nfind out a way to send the maximum amount of requests without encountering the\ntoo-many-requests error.\n\nI picked up Python's [asyncio] and the amazing [HTTPx] library by Tom Christie to make the\nrequests. This is the naive version that I wrote in the beginning; it quickly hits the HTTP\n429 error:\n\nHere, for this demonstration, I'm using the https://httpbin.org/get endpoint that's openly\naccessible. This particular endpoint doesn't impose any limit on the number of requests per\nsecond. However, in the above snippet, if you inspect the for loop in the\nmake_many_requests function, you'll see that it's sending 200 concurrent requests without\nany restrictions.\n\nAlso, the snippet will raise a ValueError if it encounters an HTTP-429-too-many-requests\nerror. Running the script produces the following output:\n\nFrom the output, it's pretty evident that the script is hammering the server without any\ndelay between the concurrent requests. While 200 requests per second may not be that high\nbut even if there weren't any restrictions, sending so many rogue requests like that isn't\ndesirable. It's easy to overwhelm any service if you're not being careful.\n\nLuckily, Python exposes a Semaphore construct that allows you to synchronize the\nconcurrent workers (threads, processes, or coroutines) regarding how they should access a\nshared resource. All concurrency primitives in Python have semaphores to help you control\nresource access. This means if you're using any of the - multiprocessing, threading, or\nasyncio module, you can take advantage of it. From the asyncio docs:\n\n> A semaphore manages an internal counter which is decremented by each acquire() call and\n> incremented by each release() call. The counter can never go below zero; when\n> acquire() finds that it is zero, it blocks, waiting until some task calls release().\n\nYou can use the semaphores in the above script as follows:\n\nHere, I only had to change the make_one_request function to take advantage of the\nsemaphore. First, I initialized an asyncio.Semaphore object with the limit 3. This means\nthe semaphore won't allow more than three concurrent workers to make HTTP GET requests at\nthe same time. The semaphore instance is then used as a context manager. Inside the\nasync with block, the line starting with if limit.locked() makes the workers wait for a\nsecond whenever the concurrency limit is reached. If you execute the script, it'll produce\nthe following output:\n\nThe output makes it clear that no more than 3 async functions are making concurrent requests\nto the server at the same time. You can tune the number of concurrent workers by changing\nthe limit in the asyncio.Semaphore object.\n\nComplete script\n\n\n\n\n[asyncio]:\n    https://docs.python.org/3/library/asyncio.html\n\n[httpx]:\n    https://www.python-httpx.org/",
  "title": "Limit concurrency with semaphore in Python asyncio"
}