{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/python/structural-subtyping/",
  "description": "Implement Go-style structural subtyping in Python with Protocol for duck typing and interface-based APIs without inheritance dependencies.",
  "path": "/python/structural-subtyping/",
  "publishedAt": "2021-12-04T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Python",
    "Typing"
  ],
  "textContent": "I love using Go's interface feature to declaratively define my public API structure.\nConsider this example:\n\nYou can [play around with the example] on Go Playground. Running it will print:\n\nEven if you don't speak Go, you can just take a look at the Geometry interface and\ninstantly know that the function measure expects a struct that implements the Geometry\ninterface where the Geometry interface is satisfied when the struct implements two\nmethods - area and perim. The function measure doesn't care whether the struct is a\nrectangle, a circle, or a square. As long as it implements the interface Geometry,\nmeasure can work on it and calculate the area and the perimeter.\n\nThis is extremely powerful as it allows you to achieve polymorphism like dynamic languages\nwithout letting go of type safety. If you try to pass a struct that doesn't fully implement\nthe interface, the compiler will throw a type error.\n\nIn the world of Python, this polymorphism is achieved dynamically. Consider this example:\n\nHere, the type of the haystack can be anything that supports the in operation. It can be\na list, tuple, set, or dict; basically, any type that has the __contains__ method.\nPython's duck typing is more flexible than any static typing as you won't have to tell the\nfunction anything about the type of the parameters and it'll work spectacularly; it's a\ndynamically typed language, duh! The only problem is the lack of type safety. Since there's\nno compilation step in Python, it won't stop you from accidentally putting a type that\nhaystack doesn't support and Python will only raise a TypeError when your try to run the\ncode.\n\nIn bigger codebases, this can often become tricky as it's difficult to tell the types of\nthese uber-dynamic function parameters without reading through all the methods that are\nbeing called on them. In this situation, we want the best of the both world; we want the\nflexibility of dynamic polymorphism and at the same time, we want some sort of type safety.\nMoreover, the Go code is self-documented to some extent and I'd love this kind of\npolymorphic | type-safe | self-documented trio in Python. Let's try to use nominal type\nhinting to statically type the following example:\n\nIn this snippet, we're declaring haystack to be a dict and then passing a set to the\nfunction parameter. If you try to run this function, it'll happily print True. However, if\nyou run [mypy] against this file, it'll complain as follows:\n\nThat's because mypy expects the type to be a dict but we're passing a set which is\nincompatible. During runtime, Python doesn't raise any error because the set that we're\npassing as the value of haystack, supports in operation. But we're not communicating\nthat with the type checker properly and mypy isn't happy about that. To fix this mypy error,\nwe can use Union type.\n\nThis will make mypy happy. However, it's still not bulletproof. If you try to pass a list\nobject as the value of haystack, mypy will complain again. So, nominal typing can get a\nbit tedious in this kind of situation, as you'd have to explicitly tell the type checker\nabout every type that a variable can expect. There's a better way!\n\nEnter [structural subtyping]. We know that the value of haystack can be anything that has\nthe __contains__ method. So, instead of explicitly defining the name of all the allowed\ntypes - we can create a class, add the __contains__ method to it, and signal mypy the fact\nthat haystack can be anything that has the __contains__ method. Python's\ntyping.Protocol class allows us to do that. Let's use that:\n\nHere, the ProtoHaystack class statically defines the structure of the type of objects that\nare allowed to be passed as the value of haystack. The instance method __contains__\naccepts an object (obj) as the second parameter and returns a boolean value based on the\nfact whether that obj exists in the self instance or not. Now if you run mypy on this\nsnippet, it'll be satisfied.\n\nThe runtime_checkable decorator on the ProtoHaystack class allows you to check whether a\ntarget object is an instance of the ProtoHaystack class in runtime via the isinstance()\nfunction. Without the decorator, you'll only be able to test the conformity of an object to\nProtoHaystack statically but not in runtime.\n\nThis pattern of strurctural duck typing is so common, that the mixins in the\ncollections.abc module are now compatible with structural type checking. So, in this case,\ninstead of creating a ProtoHaystack class, you can directly use the\ncollections.abc.Container class from the standard library and it'll do the same job.\n\nAvoid abc inheritance\n\nAbstract base classes in Python let you validate the structure of subclasses in runtime.\nPython's standard library APIs uses abc.ABC in many places. See this example:\n\nHere, the class FooInterface inherits from abc.ABC and then the methods are decorated\nwith abstract decorators. The combination of abc.ABC class and these decorators make\nsure that any class that inherits from FooInterface will have to implement the bar,\nbaz, and qux methods. Failing to do so will raise a TypeError. The Foo class\nimplements the FooInterface.\n\nThis works well in theory and practice but often time, people inadvertently start to use the\n*Interface classes to share implementation methods with the subclasses. When you pollute\nyour interface with implementation methods, theoretically, it no longer stays and\ninterface class. Go doesn't even allow you to add implementation methods to interfaces.\nAbstract base classes have their places and often time, you can't avoid them if you need a\ndependable runtime interface conformity check.\n\nHowever, more often than not, using Protocol classes with @runtime_checkable decorator\nworks really well. Here, the Protocol class implicitly (just like Go interfaces) makes\nsure that your subclasses conform to the structure that you want, and the decorator, along\nwith isinstance check can guarantee the conformity in runtime. Let's replace the abc.ABC\nand the shenanigans with the decorators with typing.Protocol:\n\nNotice that Foo is not inheriting from ProtoFoo and when you run mypy against the\nsnippet, it'll statically check whether Foo conforms to the ProtoFoo interface or not.\nVoila, we avoided inheritance. The isinstance in the run function later checks whether\nfoo is an instance of ProtoFoo or not.\n\nComplete example with tests\n\nThis example employs static duck-typing to check the type of WebhookPayload where the\nclass represents the structure of the payload that is going to be sent to an URL by the\nsend_webhook function.\n\nDisclaimer\n\nAll the code snippets here are using Python 3.10's type annotation syntax. However, if\nyou're using from __future__ import annotations, you'll be able to run all of them in\nearlier Python versions, going as far back as Python 3.7.\n\nFurther reading\n\n- [PEP 544 -- Protocols: Structural subtyping (static duck typing)]\n\n\n\n\n[play around with the example]:\n    https://go.dev/play/p/RG82v5Ubdlc\n\n[mypy]:\n    https://mypy.readthedocs.io/en/stable/\n\n[structural subtyping]:\n    https://www.python.org/dev/peps/pep-0544/#nominal-vs-structural-subtyping\n\n[pep 544 -- protocols: structural subtyping (static duck typing)]:\n    https://www.python.org/dev/peps/pep-0544/",
  "title": "Structural subtyping in Python"
}