{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/python/metaclasses/",
  "description": "Explore Python metaclasses for metaprogramming: learn how classes are created, customize class behavior, and understand type as the default metaclass.",
  "path": "/python/metaclasses/",
  "publishedAt": "2020-06-26T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Python"
  ],
  "textContent": "_Updated on 2023-09-11_: _Fix broken URLs._\n\nIn Python, metaclass is one of the few tools that enables you to inject metaprogramming\ncapabilities into your code. The term metaprogramming refers to the potential for a program\nto manipulate itself in a self referential manner. However, messing with metaclasses is\noften considered an arcane art that's beyond the grasp of the plebeians. Heck, even [Tim\nPeters] advises you to tread carefully while dealing with these.\n\n> Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder\n> whether you need them, you don't (the people who actually need them know with certainty\n> that they need them, and don't need an explanation about why).\n\nMetaclasses are an esoteric OOP concept, lurking behind virtually all Python code. Every\nPython class that you create is attached to a default metaclass and Python cleverly\nabstracts away all the meta-magics. So, you're indirectly using them all the time whether\nyou are aware of it or not. For the most part, you don't need to be aware of it. Most Python\nprogrammers rarely, if ever, have to think about metaclasses. This makes metaclasses\nexciting for me and I want to explore them in this post to formulate my own judgement.\n\nMetaclasses\n\nA metaclass is a class whose instances are classes. Like an \"ordinary\" class defines the\nbehavior of the instances of the class, a metaclass defines the behavior of classes and\ntheir instances.\n\n![Diagram showing relationship between metaclass, class, and instance in Python][image_1]\n\nMetaclasses aren't supported by every object oriented programming language. Those\nprogramming language, which support metaclasses, considerably vary in way they implement\nthem. Python provides you a way to get under the hood and define custom metaclasses.\n\nUnderstanding type and class\n\nIn Python, everything is an object. Classes are objects as well. As a result, all classes\nmust have corresponding types. You deal with built in types like int, float, list etc\nall the time. Consider this example:\n\nIn the above example, variable a is an instance of the built in class int. Type of a\nis int and the type of int is type. User defined classes also show similar behavior.\nFor example:\n\nHere, I've defined another class named Foo and created an instance a of the class.\nApplying type on instance a reveals its type as __main__.Foo and applying type on\nclass Foo reveals the type as type. So here, we can use the term class and type\ninterchangeably. This brings up the question:\n\n> What on earth this type (function? class?) thing actually is and what is the type of\n> type?\n\nLet's apply type on type:\n\nWeird. The type of any class (not instance of a class) in Python is type and the type of\ntype is also type. By now, you've probably guessed that type is a very special class\nin Python that can reveal the type of itself and of any other class or object. In fact,\ntype is a metaclass and all the classes in Python are instances of it. You can inspect\nthat easily:\n\nThe the last line of the above code snippet demonstrates that type is also an instance of\nmetaclass type. Normally, you can't write self referential classes like that in pure\nPython. However, you can circumvent this limitation by subclassing from type. This enables\nyou to write custom metaclasses that you can use to dictate and mutate the way classes are\ncreated and instantiated. From now on, I'll be referring to the instance class of a\nmetaclass as _target class_. Let's create a custom metaclass that just prints the name of\nthe target class while creating it:\n\nDespite the fact that we haven't called class A or created an instance of it, the\n__new__ method of metaclass PrintMeta was executed and printed the name of the target\nclass. In the return statement of __new__ method, super() was used to call the __new__\nmethod of the base class (type) of the metaclass PrintMeta.\n\nSpecial methods used by metaclasses\n\nType type, as the default metaclass in Python, defines a few special methods that new\nmetaclasses can override to implement unique code generation behavior. Here is a brief\noverview of these \"magic\" methods that exist on a metaclass:\n\n- __new__: This method is called on the Metaclass before an instance of a class based on\n  the metaclass is created\n- __init__: This method is called to set up values after the instance/object is created\n- __prepare__: Defines the class namespace in a mapping that stores the attributes\n- __call__: This method is called when the constructor of the new class is to be used to\n  create an object\n\nThese are the methods to override in your custom metaclass to give your classes behaviors\ndifferent from that of type. The following example shows the default behaviors of these\nspecial methods and their execution order.\n\n> Some people immediately think of __init__, and I've occasionally called it \"the\n> constructor\" myself; but in actuality, as its name indicates, it's an initializer and by\n> the time it's invoked, the object has already been created, seeing as it's passed in as\n> self. The real constructor is a far less famous function: __new__. The reason you might\n> never hear about it or use it - is that allocation doesn't mean that much in Python, which\n> manages memory for you. So if you do override __new__, it'd be just like your\n> __init__ - except you'll have to call into Python to actually create the object, and\n> then return that object afterward.\n\nPay attention to the execution order of the special methods of the custom metaclass\nExampleMeta. The __prepare__ method is called first and is followed by __new__,\n__init__ and __call__ respectively. Only after that the first method __init__ of the\ntarget class A is called. This is important since it'll help you to decide how you'll\nmutate and change the behavior of your target class.\n\nMetaclass conflicts\n\nNote that the metaclass argument is singular – you can't attach more than one metaclass to a\nclass. However, through multiple inheritance you can accidentally end up with more than one\nmetaclass, and this produces a conflict which must be resolved.\n\nClass First and Second are attached to different metaclasses and class Third inherits\nfrom both of them. Since metaclasses trickle down to subclasses, class Third is now\neffective attached to two metaclasses (FirstMeta and SecondMeta). This will raise\nTypeError. Attachment with only one metaclass is allowed here.\n\nExamples in the wild\n\nIn this section, I'll go through a few real life examples where metaclasses can provide\nviable solutions to several tricky problems that you might encounter during software\ndevelopment. The solutions might appear over-engineered in some cases and almost all of them\ncan be solved without using metaclasses. However, the purpose is to peek into the inner\nwirings of metaclasses and see how they can offer alternative solutions.\n\nSimple logging with metaclasses\n\nThe goal here is to log a few basic information about a class without directly adding any\nlogging statements to it. Instead, you can whip up a custom metaclass to perform some\nmetaprogramming and add those statements to the target class without mutating it explicitly.\n\nIn the above example, I've created a metaclass called LittleMeta and added the necessary\nlogging statements to record the information about the target class. Since the logging\nstatements are residing in the __new__ method of the metaclass, these informations will be\nlogged before the creation of the target class. In the target class Point, LittleMeta\nreplaces the default type metaclass and produces the desired result by mutating the class.\n\nReturning class attributes in a custom list\n\nIn this case, I want to dynamically attach a new attribute to the target class called\n__attrs_ordered__. Accessing this attribute from the target class (or instance) will give\nout an alphabetically sorted list of the attribute names. Here, the __prepare__ method\ninside the metaclass AttrsListMeta returns an OrderDict instead of a simple dict - so\nall attributes gathered before the __new__ method call will be ordered. Just like the\nprevious example, here, the __new__ method inside the metaclass implements the logic\nrequired to get the sorted list of the attribute names.\n\nYou can access the __attrs_ordered__ attribute from both class A and an instance of\nclass A. Try removing the sorted() function inside the __new__ method of the metaclass\nand see what happens!\n\nCreating a singleton class\n\nIn OOP term, a singleton class is a class that can have only one object (an instance of the\nclass) at a time.\n\nAfter the first time, if you try to instantiate a Singleton class, it will basically return\nthe same instance of the class that was created before. So any modifications done to this\napparently new instance will mutate the original instance since they're basically the same\ninstance.\n\nIn the above example, at first, I've created a singleton class A by attaching the\nSingleton metaclass to it. Secondly, I've instantiated class A and assigned the instance\nof the class to a variable a. Thirdly, I've instantiated the class again and assigned\nvariable a b to this seemingly new instance. Checking the identity of the two variables\na and b reveals that both of them are actually the same object.\n\nImplementing a class that can't be subclassed\n\nSuppose you want to create a base class where the users of your class won't be able to\ncreate any subclasses from the base class. In that case, you can write a metaclass and\nattach that your base class. The base class will raise RuntimeError if someone tries to\ncreate a subclass from it.\n\nDisallowing multiple inheritance\n\nMultiple inheritance can be fragile and error prone. So, if you don't want to allow the\nusers to use a base class with any other base classes to form multiple inheritance, you can\ndo so by attaching a metaclass to that target base class.\n\nTiming classes with metaclasses\n\nSuppose you want to measure the execution time of different methods of a class. One way of\ndoing that is to define a timer decorator and decorating all the methods to measure and show\nthe execution time. However, by using a metaclass, you can avoid decorating the methods in\nthe class individually and the metaclass will dynamically apply the timer decorator to all\nof the methods of your target class. This can reduce code repetition and improve code\nreadability.\n\nRegistering plugins with metaclasses\n\nSuppose a specific single class represents a plugin in your code. You can write a metaclass\nto keep track of all of the plugins so than you don't have to count them manually.\n\nDebugging methods with metaclasses\n\nDebugging a class often involves inspecting the individual methods and adding extra\ndebugging logic to those. However, this can get tedious if you've do this over an over\nagain. Instead, you can write an inspection decorator and use a metaclass to dynamically\napply the decorator to all of the methods of your target class. Later on, you can simply\ndetach the metaclass once you're done with debugging and don't want the extra logic in your\ntarget class.\n\nException handling with metaclasses\n\nSometimes you need to handle exceptions in multiple methods of a class in a generic manner.\nThat means all the methods of the class have the same exception handling, logging logic etc.\nMetaclasses can help you avoid adding repetitive exception handling and logging logics to\nyour methods.\n\nAbstract base classes\n\nAn abstract class can be regarded as a blueprint for other classes. It allows you to provide\na set of methods that must be implemented within any child classes built from the abstract\nclass. Abstract classes usually house multiple abstract methods. An abstract method is a\nmethod that has a declaration but does not have an implementation.\n\nWhen you want to provide a common interface for different implementations of a component,\nabstract classes are the way to go. You can't directly initialize or use an abstract class.\nRather, you've to subclass the abstract base class and provide concrete implementations of\nall the abstract methods. Python has a dedicated abc module to help you create abstract\nclasses. Let's see how you can define a simple abstract class that provides four abstract\nmethods:\n\nAlthough it seems like interface ICalc is simply inheriting from the class ABC, in fact,\nABC is attaching a metaclass ABCMeta to ICalc. This metaclass transforms the ICalc\nclass into an abstract class. You can see that the class ICalc gives TypeError when you\ntake an attempt to initialize it. The only way to use this interface is via creating\nsubclasses from ICalc base class and implementing all the abstract methods there. The\nsnippet below shows that:\n\nMetaclasses & dataclasses\n\nData classes were introduced to python in version 3.7. Basically they can be regarded as\ncode generators that reduce the amount of boilerplate you need to write while generating\ngeneric classes. Dataclasses automatically create __init__, __repr__, __eq__,\n__gt__, __lt__ etc methods without you having to add them explicitly. This can be very\nhandy when you need to create custom collections for your data. You can create dataclasses\nin the following manner:\n\nCreating multiple dataclasses\n\nAvoiding dataclass decorator with metaclasses\n\nNow, one thing that I find cumbersome while creating multiple dataclasses is having to\nattach the @dataclasses.dataclass decorator to each of the dataclasses. Also, the\ndecorator takes multiple arguments to customize the dataclass behavior and it can quickly\nget cumbersome when you've to create multiple dataclasses with custom behavior. Moreover,\nthis goes against the DRY (Don't Repeat Yourself) principle in software engineering.\n\nTo avoid this, you can write a metaclass that will automatically apply the customized\ndataclass decorator to all of the target classes implicitly. All you have to do is to attach\nthe metaclass to a base dataclass and inherit from it in the later dataclasses that need to\nbe created.\n\nShould you use it?\n\nAlmost all of the problems you've encountered above can be solved without using metaclasses.\nDecorators can also be exclusively used to perform metaprogramming in a more manageable and\nsubjectively cleaner way. One case where you absolutely have to use metaclasses is to avoid\napplying decorators to multiple classes or methods repetitively.\n\nAlso, metaclasses can easily veer into the realm of being a _\"solution in search of a\nproblem\"._ If the problem at hand can be solved in a simpler way, it probably should be.\nHowever, I still think that you should at least try to understand how metaclasses work to\nhave a better grasp on how Python classes work in general and can recognize when a metaclass\nreally is the appropriate tool to use.\n\nRemarks\n\nWrapping your mind around metaclasses can be tricky. So, to avoid any unnecessary confusion,\nI've entirely evaded any discussion regarding the behavioral difference between _old style\nclasses_ and _new style classes_ in Python. Also, I've intentionally excluded mentioning the\ndifferences between type in Python 2 and type in Python 3 entirely. Python 2.x has\nreached its EOL. Save yourself some trouble and switch to Python 3.x if you already haven't\ndone so.\n\nThis article assumes familiarity with decorators, dataclasses etc. If your knowledge on them\nis rusty, checkout these posts on [decorators] and [dataclasses].\n\nFurther reading\n\n- [Understanding Python's metaclasses]\n- [What are metaclasses in Python - Stackoverflow]\n- [Python metaclasses - Real Python]\n- [Metaclasses - Python course EU]\n- [When to use metaclasses in Python]\n- [A primer on Python metaclasses]\n\n\n\n\n[tim peters]:\n    https://en.wikipedia.org/wiki/Tim_Peters_(software_engineer)\n\n[decorators]:\n    /python/decorators\n\n[dataclasses]:\n    /python/dataclasses\n\n[understanding python's metaclasses]:\n    https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/\n\n[what are metaclasses in python - stackoverflow]:\n    https://stackoverflow.com/questions/100003/what-are-metaclasses-in-python\n\n[python metaclasses - real python]:\n    https://realpython.com/python-metaclasses/\n\n[metaclasses - python course eu]:\n    https://www.python-course.eu/python3_metaclasses.php\n\n[when to use metaclasses in python]:\n    https://breadcrumbscollector.tech/when-to-use-metaclasses-in-python-5-interesting-use-cases/\n\n[a primer on python metaclasses]:\n    https://jakevdp.github.io/blog/2012/12/01/a-primer-on-python-metaclasses/\n\n[image_1]:\n    https://blob.rednafi.com/static/images/metaclasses/img_1.jpeg",
  "title": "Deciphering Python's metaclasses"
}