{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/python/declarative-payloads-with-typedict/",
  "description": "Use Python TypedDict to declaratively define API payload structures. Get type safety for nested dictionaries and improve code maintainability.",
  "path": "/python/declarative-payloads-with-typedict/",
  "publishedAt": "2022-03-11T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Python",
    "Typing",
    "API"
  ],
  "textContent": "While working with microservices in Python, a common pattern that I see is - the usage of\ndynamically filled dictionaries as payloads of REST APIs or message queues. To understand\nwhat I mean by this, consider the following example:\n\nHere, the get_payload function constructs a payload that gets stored in a Redis DB in the\nsave_to_cache function. The get_payload function returns a dict that denotes a contrived\npayload containing the data of an imaginary zoo. To execute the above snippet, you'll need\nto spin up a Redis database first. You can use [Docker] to do so. Install and configure\nDocker on your system and run:\n\nIf you run the above snippet after instantiating the Redis server, it'll run without raising\nany error. You can inspect the content saved in Redis with the following command (assuming\nyou've got redis-cli and jq installed in your system):\n\nThis will return the following payload to your console:\n\nAlthough this workflow is functional in runtime, there's a big gotcha here! It's really\ndifficult to picture the shape of the payload from the output of the get_payload\nfunction; as it dynamically builds the dictionary. First, it declares a dictionary with two\nfields - name and animals. Here, name is a string value that denotes the name of the\nzoo. The other field animals is a list containing the names and attributes of the animals\nin the zoo. Later on, the for-loop fills up the dictionary with nested data structures. This\ncharade of operations makes it difficult to reify the final shape of the resulting payload\nin your mind.\n\nIn this case, you'll have to inspect the content of the Redis cache to fully understand the\nshape of the data. Writing code in the above manner is effortless but it makes it really\nhard for the next person working on the codebase to understand how the payload looks without\ntapping into the data storage. There's a better way to declaratively communicate the shape\nof the payload that doesn't involve writing unmaintainably large docstrings. Here's how you\ncan leverage TypedDict and Annotated to achieve the goals:\n\nNotice, how I've used TypedDict to declare the nested structure of the payload Zoo. In\nruntime, instances of typed-dict classes behave the same way as normal dicts. Here, Zoo\ncontains two fields - name and animals. The animals field is annotated as\nlist[Animal] where Animal is another typed-dict. The Animal typed-dict houses another\ntyped-dict called Attribute that defines various properties of the animal.\n\nTaking a look at the typed-dict Zoo and following along its nested structure, the final\nshape of the payload becomes clearer without us having to look for example payloads. Also,\nMypy can check whether the payload conforms to the shape of the annotated type. I used\nAnnotated[Zoo, dict] in the input parameter of save_to_cache function to communicate\nwith the reader that an instance of the class Zoo is a dict that conforms to the contract\nlaid out in the type itself. The type Annotated can be used to add any arbitrary metadata\nto a particular type.\n\nIn runtime, this snippet will exhibit the same behavior as the previous one. Mypy also\napproves this.\n\nHandling missing key-value pairs\n\nBy default, the type checker will structurally validate the shape of the dict annotated with\na TypedDict class and all the key-value pairs expected by the annotation must be present\nin the dict. It's possible to lax this behavior by specifying _totality_. This can be\nhelpful to deal with missing fields without letting go of type safety. Consider this:\n\nMypy will complain about the missing key:\n\nYou can relax this behavior like this:\n\nNow Mypy will no longer complain about the missing field in the annotated dict. However,\nthis will still disallow arbitrary keys that isn't defined in the TypedDict. For example:\n\nSweet type safety without being too strict about missing fields!\n\nFurther reading\n\n- [PEP 589 – TypedDict: Type hints for dictionaries with a fixed set of keys]\n\n\n\n\n[docker]:\n    https://www.docker.com/\n\n[pep 589 – typeddict: type hints for dictionaries with a fixed set of keys]:\n    https://peps.python.org/pep-0589/",
  "title": "Declarative payloads with TypedDict in Python"
}