Caching connection objects in Python

Redowan Delowar March 16, 2022
Source

To avoid instantiating multiple DB connections in Python apps, a common approach is to initialize the connection objects in a module once and then import them everywhere. So, you'd do this:

However, this adds import time side effects to your module and can turn out to be expensive. In search of a better solution, my first instinct was to go for functools.lru_cache(None) to immortalize the connection objects in memory. It works like this:

This way, the connection objects returned by the functions are cached and any subsequent calls to the functions will provide the same connection objects from the cache without reinitializing them.

One problem with the above approach is - how complex the implementation of the cache decorator is. Underneath, the functools.cache decorator is an alias for functools.lru_cache(None) and it employs a Least Recently Used cache eviction policy. While this policy is quite useful when you need it but to cache simple connection objects, arguably, the complexity and the overhead of the cache decorator offset its benefits. There's a simpler way to do it and James Powell on Twitter pointed me to it. This works as follows:

Is this singleton pattern? Probably so.

Discussion in the ATmosphere

Loading comments...