Shades of testing HTTP requests in Python

Here's a Python snippet that makes an HTTP POST request:

The function make_request makes an async HTTP request with the HTTPx library. Running this with asyncio.run(make_request("https://httpbin.org/post")) gives us the following output:

We're only interested in the json field and want to assert in our test that making the HTTP call returns the expected values.

Testing the HTTP request

Now, how would you test it? One approach is by patching the httpx.AsyncClient instance to return a canned response and asserting against that. The happy path might be tested as follows:

That's quite a bit of work just to test a simple HTTP request. The mocking gets pretty hairy as the complexity of your HTTP calls increases. One way to cut down the mess is by using a library like respx that handles the patching for you.

Simplifying mocks with respx

For instance:

Much cleaner. During tests, respx intercepts HTTP requests made by httpx, allowing you to test against canned responses. The library provides a context manager that acts like an httpx client, so you can set the expected response. This removes the need to manually patch methods like post in httpx.AsyncClient.

Testing with a stub client

The previous strategy wouldn't work if you want to change your HTTP client since respx is coupled with httpx. As an alternative, you could rewrite make_request to parametrize the HTTP client, pass a stub object during the test, and assert against it. This eliminates the need to write fragile mocking sludges or depend on an external mocking library.

Here's how you'd change the code:

Now the tests would look as follows:

Much better!

Integration testing with a test server

One thing I've picked up from writing Go is that it's often just easier to perform integration tests on these I/O-bound functions. That is, you can spin up a server that returns a canned response and then test your code against it to assert if it's getting the expected output.

The test could look as follows. This assumes make_request takes in an AsyncClient instance as a parameter, as shown in the last example.

In the above test, we're using Starlette to define a simple ASGI server that returns our expected response. Then we set up the httpx.AsyncClient so it makes the request against the test server instead of making an external network call. Finally, we call the make_request function and assert the expected payload.

Sure, you could set up the server with the standard library's http module, but that code doesn't look half as pretty.