blog banner alt

Introspection: Reflection in Python

Last updated Feb 17, 2018


If you are from another programming language background, you might have used the Reflection feature with in that language.

For example, the ReflectionClass in PHP or ‘object.getMethods()’ in Java.

In Python, the terms Introspection and Reflection seems to have been used interchangeably. However they might seem similar, they are actually very different fundamentally!

If you had followed my previous posts, Dynamic Import in Python 3 and Factory Pattern in Python 3, I have used both introspection and reflection capabilities of Python in the examples.

 

At this point, you may ask,

“How are introspection and reflection different?”

“What even are they anyways?”

Before we go any further, remember one simple thing, the purpose of introspection is to examine, whereas reflection can both example and modify.

In other words, introspection is passive, and reflection is active.

And remember, everything in Python is an object.

 

Introspection

Introspection is also called ‘type introspection’, it’s the ability to examine something, typically an object, at runtime. With introspection, everything about an object is exposed. Such as, what it is, what attributes it has, and etc.

If this doesn’t make any sense, let me show you some examples.

 

Example 1: dir()
class dummy:
    def __init__(self, foo):
        self.foo = foo

    def get_foo(self):
        return self.foo

my_dummy = dummy('blah')
dir(my_dummy)

I can’t talk about introspection in python without talking about dir(). This builtin function will get us all the attributes names of an object as a list of strings.

Here is the output to the code:

>>> ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__',
    '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__',
    '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', 
    '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'foo', 'get_foo']

You may wonder, “why am I seeing all the dunder attributes in the list, besides ‘foo’ and ‘get_foo’ that I have defined?”

This is because everything in python is an object. This means all objects are inherited from the builtin object class, it doesn’t matter if it’s the ones you defined or not.

 

Example 2: Type checking

Type checking can vary in different ways, like so:

# Check the type/class of an object
type(my_object)
my_object.__class__

# check is the object is a specific type or class
isinstance(my_str, str)

# check if an object's class is a subclass of a parent class
issubclass(my_object.__class__, ParentClass)

Important: I would really be careful with isinstance(), it does not work as expected all the time. If the first argument (the object passed in)’s class  is an extended class of the second argument (the class passed in), this would always evaluate to True.

I have encountered this while making a function for logging to ensure no duplicated handlers. However, since logging.FileHandler is a child class of logging.StreamHandler, so isninstance() will return True:

isinstance(my_filehandler, logging.StreamHandler)
# Outputs: True

# However, the opposite is not True
isinstance(my_streamhandler, logging.FileHandler)
# Outputs: False

Other than the examples above, there are quite a lot more you can do with introspection in Python. I would highly suggest looking into the inspect module. This module provides functionalities to check if an object is a class, a method, a module, or a function.

inspect.isclass(), inspect.ismethod(), inspect.isfunction() are very useful for decorators, which I won’t cover in this article. 

Here is a rather silly example to detect whether the decorated object is a class or a function:

import inspect
from functools import wraps


def what_am_i(decorated):

    @wraps(decorated)
    def wrapper(*args, **kwargs):

        if inspect.isclass(decorated):
            print(f"{decorated.__name__} is a class")
            return decorated(*args, **kwargs)

        if inspect.isfunction(decorated):
            print(f"{decorated.__name__} is a function!")
            return decorated(*args, **kwargs)

    return wrapper


@what_am_i
def my_function():
    print('hello')


@what_am_i
class MyClass:
    def __init__(self, name):
        self.name = name

    def age(self, age):
        return age

Don’t worry if you don’t exactly understand how this works for now, since this article isn’t about decorators. If you do have questions, feel free to message me or leave a comment.

Moving on.

 

Reflection

In comparison to introspection, reflection is much more powerful. As I had mentioned earlier: Introspection is passive, the purpose is to examine; Reflection is active, and it’s not only able examine, but also to modify.

Concretely, reflection means the ability of a program to examine and modify its own structure and behavior at runtime. For example, setting an attribute of a module at runtime, as I have briefly mentioned in Dynamic Import in Python 3, I will go in more depth in the first example.

 

Example 1: Setting Attributes Dynamically

Supposedly, I have an empty python file, and I want to have all the environment variables as attributes of this module. let’s call this file envs.py.

Here is how it would look:

import os
from importlib import import_module

for k, v in os.environ.items():
    setattr(import_module(__name__), k, v)

import_module(__name__) gets the current module as an object, and setattr() assign the attribute to the module with the key value of each environment variable.

In fact, import_module() method itself utilizes the reflective capability of Python, as it resolves to module objects based on the string passed.

What if I now want to use this envs.py, to get a mapping of environment variables as dict, and envs.py also contains other objects that I’m not interested in?

This leads to our next example.

 

Example 2: Using getattr() to get attribute values dynamically

In another file outputs.py in the same directory. In this file we want to import env.py, and have a function to get all the environment variables as dict.

import envs


def get_env_vars():
    env_dict = {}
    for name in dir(envs):
        attr = getattr(envs, name)

        if isinstance(attr, str):
            env_dict[name] = attr

    return env_dict

As mentioned previously in the introspection section, dir() gets us all the attribute names of an object.

What’s important here is the getattr() builtin function. This function allows you to get the attribute value by passing in an object and attribute name (as string value).

 

Example 3: Dynamic Method

What I mean by dynamic method is, when calling a method that does not exist within the defined class of the object, it will still be invoked successful if certain conditions are met.

Let’s look at a simple example.

class GreetMe:
    def __init__(self, name):
        self.name = name

    def __getattr__(self, attr):

        allowed = ['hello', 'bye', 'nice_to_meet_you', 'good_bye', 'goodnight']

        def call_(name=None):
            if attr in allowed:
                greeting = attr.replace('_', ' ')
                target = name if name else self.name

                return f"{target}, {greeting.capitalize()}"
            else:
                raise ValueError(f"Invalid name or greeting. name: {name}, greeting: {attr}")

        return call_


greet = GreetMe('Luna')

greet.hello()
# Outputs: 'Luna, Hello'

greet.bye(name='John')
# Outputs: 'John, Bye'

greet.nice_to_meet_you(name='Jane')
# Outputs: 'Jane, Nice to meet you'

As you can see, here we create an object of GreetMe, and called to the methods ‘hello(), bye() and nice_to_meet_you()’, and none of these methods are defined with in the GreetMe class.

We are able to call these methods is because of the __getattr__() method implemented in the class. Python calls __getattr__() whenever a undefined method is being accessed. In our example, only the methods names within the allowed list can be called.

 

Closing Thoughts

Both introspection and reflection are very powerful concepts, and Python has strong support for them. You can certainly gain a great amount of flexibility and control once you get a hang of them. One thing to keep in mind is that reflection can fall into the category of implicity very quickly.


Tags