Python Packages

I’ve avoided putting my Python modules into packages.  That’s because (like a lot of other folks) I don’t really understand how to create a package.  And that’s because (probably like a lot of other folks) I’m too time-constrained and over-worked to be able to grok anything except a very short, clear, and simple explanation of how packages work… and I never found one.

That’s all over now. In a post on comp.lang.python Steven D’Aprano has made it all beautifully clear. Here is an excerpt from his post. This is the part that really turned on the lights for me.

A package is a special arrangement of folder + modules.

To be a package, there must be a file called in the folder, e.g.:

        +-- feeding/

This defines a package called parrot which includes a sub-package feeding and modules fighting, flying, sleeping and talking. You can use it by any variant of the following:

import parrot  # loads parrot/
import parrot.talking  # loads parrot/
from parrot import sleeping
import parrot.feeding
from parrot.feeding.eating import eat_cracker
#... and so on ...

Common (but not compulsory) behaviour is for parrot/ to import all the modules in the package, so that the caller can do this:

import parrot

without needing to manually import sub-packages. The os module behaves similarly: having imported os, you can immediately use functions in os.path without an additional import.

Just dumping a bunch of modules into a folder doesn’t make it a package, it just makes it a bunch of modules in a folder. Unless that folder is in the PYTHONPATH, you won’t be able to import the modules because Python doesn’t look inside folders. The one exception is that it will look inside a folder for a file, and if it finds one, it will treat that folder and its contents as a package.

And to distribute a collection of modules as a package….
  • Put your modules in a package,
  • tell the user to just place the entire package directory where they normally install Python code (usually site-packages — Steve Ferg), and
  • importing will just work

Thank you, Steven! You have no idea how much we needed that!
Updated 2011-11-06 to improve formatting

This entry was posted in Python features. Bookmark the permalink.

6 Responses to Python Packages

  1. Actually the os module isn’t a package – it’s just an ordinary module that *cheats* to look like a package. When you import os it decides which is the right version of os.path for your system (ntpath on windows and posixpath just about everywhere else these days) and then *sticks* that as the path attribute on the os module and in sys.modules as os.path.

  2. Anonymous says:

    I found this information in a lot of places, and was still struggling to make “import parrot” work for example. The only info that is never clear is that the folder that you have to put in your PYTHONPATH is the folder CONTAINING parrot and not parrot itself!!!

  3. Anonymous says:

    Thanks a lot, Anonymous! It really says nowhere, that you have to add the parent directory of the package base directory -.-!! Killed several frustrating hours of my life -.-

  4. Sven says:

    This was really helpful.

    I was wondering what the contents of the files should be. I read in the pyDocs that this can be empty. However, in the case of the parrot package does the module need to explicitly define the attribute feeding? How is this done, with another import call or through use of __import__()?

    • meowsqueak says:

      Just with ‘import flying’ in parrot/ Then you don’t need to explicitly import parrot.flying after importing parrot. It’s really simple.

      • varun_bisht says:

        should we really have to create the folders corresponding to the package names that we have used ??

Comments are closed.