Introduction to Python Decorators

This post was originally written in August 2009. Since then, I have come to believe that there is a much better way to explain Python decorators, which I describe in another post.


Writing introductions to decorators is a popular pastime in the Python community. Here, for example, are some useful links on the subject:

But when it comes to technical topics, everyone has his or her own style of learning and one size of explanation does not fit all.

So I thought I’d try my hand at writing an introduction to Python decorators. My goal is not to explain everything about decorators. Instead I want to try to explain just the basics, just enough to give you a workable mental model of what decorators are and how to use them. Just enough to get started on doing useful work with decorators.

As Aristotle said, “Let us begin at the beginning”, which is to say, we begin by looking at functions.

Functions

When the Python interpreter encounters this code:

def hello():
    print ("Hello, world!")

it:

  • compiles the code to create a function object
  • binds the name "hello" to that function object.

Then, to run the function object, you can code

hello()

which causes this to be printed:

Hello, world!

If you code:

print (hello)

you will get something like:

<function hello at 0x02D021E0>

which is the string representation of the hello function object.
 

Annotations

Many discussions of decorators use the word "decorator" rather loosely, to refer to different decorator-related concepts. This kind of ambiguity is disconcerting at best, and confusing at worst.

To help avoid this ambiguity I will use the term "annotation" in this discussion to refer to lines of code that begin with "@".

Here is a snippet of Python code that begins with two annotations:

@helloGalaxy
@helloSolarSystem
def hello():
    print ("Hello, world!")

We can say that the definition of the hello function is "decorated" with these two annotations. Since there are multiple annotations, we say that the annotations are "stacked".

When the interpreter sees these lines of code, here is what it does.

  • It pushes helloGalaxy onto the annotation stack.
  • It pushes helloSolarSystem onto the annotation stack.

then it does the standard processing for a function definition ...

  • It compiles the code for hello into a function object (lets call it functionObject1)
  • It binds the name "hello" to functionObject1.

then...

  • It pops helloSolarSystem off of the annotation stack,
  • passes functionObject1 to helloSolarSystem ...
  • helloSolarSystem returns a new function object (lets call it functionObject2), and...
  • the interpreter binds the original name "hello" to functionObject2

then...

  • It pops helloGalaxy off of the annotation stack,
  • passes functionObject2 to helloGalaxy ...
  • helloGalaxy returns a new function object (lets call it functionObject3), and...
  • the interpreter binds the original name "hello" to functionObject3

As you can see, this process could be repeated for indefinitely many annotations.

I've been vague about what kind of thing that helloSolarSystem and helloGalaxy are. For now, think of them as a special kind of function -- a kind of function that takes one function object as an argument, and returns another function object as a result. The annotations:

@helloGalaxy
@helloSolarSystem

are calls to these functions. So this snippet of Python code:

@helloGalaxy
@helloSolarSystem
def hello():
    print ("Hello, world!")

is functionally equivalent to this:

def hello():
    print ("Hello, world!")
hello = helloSolarSystem(hello)
hello = helloGalaxy(hello)

Decorators

Now we are ready to define "decorator".

A decorator is a function that is called by an annotation.

Where where do decorators come from?

You write them, just the way that you write other function definitions.

So let's write some decorators. Here is helloSolarSystem.

def helloSolarSystem(original_function):
    def new_function():
        original_function()  # the () after "original_function" causes original_function to be called
        print("Hello, solar system!")
    return new_function

And let's write helloGalaxy.

def helloGalaxy(original_function):
    def new_function():
        original_function()  # the () after "original_function" causes original_function to be called
        print("Hello, galaxy!")
    return new_function

As you can see, both of these decorators add a bit of functionality to the function object — original_function — that they receive as input. They wrap a call to original_function in a new function, new_function, put some additional functionality in new_function, and then they return the new_function object. (They return the new_function object to the annotation, which binds the original name to the new function object.)

So now let's run our whole program and see what we get. Here's the program.

def helloSolarSystem(original_function):
    def new_function():
        original_function()  # the () after "original_function" causes original_function to be called
        print("Hello, solar system!")
    return new_function
	
def helloGalaxy(original_function):
    def new_function():
        original_function()  # the parentheses after "original_function" cause original_function to be called
        print("Hello, galaxy!")
    return new_function

@helloGalaxy
@helloSolarSystem
def hello():
    print ("Hello, world!")

# Here is where we actually *do* something!
hello()

And here is what we get:

Hello, world!
Hello, solar system!
Hello, galaxy!

Arguments to functions

Now lets look at decorating functions that take arguments. Let's modify the hello function so it accepts an argument, like this:

def hello(targetName=None):
    if targetName:
        print("Hello, " +  targetName +"!")
    else:
        print("Hello, world!")

If we were to run an undecorated version of the hello function, we'd get a nice greeting, like this:

>>> hello("Earth")
Hello, Earth!

But if we run the decorated version of the hello function, we get this:

TypeError: new_function() takes no arguments (1 given)

What's the problem?

Remember that we wrapped functionObject1 (created from hello) in functionObject2 (created from helloSolarSystem) and then in functionObject3 (created from Galaxy), and then bound the name "hello" to functionObject3. So when we use the "hello" function, we are calling functionObject3.

FunctionObject3 was created by the code for new_function in helloGalaxy, and it accepts no arguments. Which is why we get the error message:

TypeError: new_function() takes no arguments (1 given)

The solution is to add support for arguments to the function objects that our decorators create. We need to add code to new_function so that it will accept arguments, and we need to add code to original_function so that it will accept the arguments that its wrapper (new_function) makes available to it.

def helloSolarSystem(original_function):
    def new_function(*args, **kwargs):
        original_function(*args, **kwargs)
        print("Hello, solar system!")
    return new_function

def helloGalaxy(original_function):
    def new_function(*args, **kwargs):
        original_function(*args, **kwargs)
        print("Hello, galaxy!")
    return new_function

And now:

>>>hello("Earth")
Hello, Earth!
Hello, solar system!
Hello, galaxy!
About these ads

3 thoughts on “Introduction to Python Decorators

  1. Thanks for the post. Got an idea of Decorators.
    I feel using functions is much more easy and readable than these.

    Can you please shower some light on Pychecker, pylint/pyflakes. No blogs/documentations found regarding the usage of these.

    • Re: pychecker, pylint, pyflakes…

      Here are a couple of comparison/reviews from 2008

      http://www.doughellmann.com/articles/pythonmagazine/completely-different/2008-03-linters/index.html

      http://amix.dk/blog/post/19361

      My own general impressions as of September 2010:

      Pychecker is stagnant. Hasn’t been updated since august 2008, has broken links on its home page.

      Pylint is active. Last updated in August 2010.

      Pyflakes seems to be moderately active: last updated: November 2009. Unlike the other tools, it checks only for logical errors in programs; it does not perform any checks on style. It is fast so appropriate for large projects. A number of supporting tools seem to have been written to help integrate pyflakes into ides and editors.

      Pychecker imports each module. Pylint and PyFlakes do not import it, so they are preferable in situations where importing the code might have undesirable side effects.

  2. Pingback: Django Make Html/Form Helpers | ProgramWith.Us

Comments are closed.