Python Conquers The Universe

2011/03/28

Why import star is a bad idea

Filed under: Python features — Steve Ferg @ 1:55 pm

When I was learning Python, I of course read the usual warnings. They told me: You can do

from something_or_other import *

but don’t do it. Importing star (asterisk, everything) is a Python Worst Practice.

Don’t “import star”!

But I was young and foolish, and my scripts were short. “How bad can it be?” I thought. So I did it anyway, and everything seemed to work out OK.

Then, like they always do, the quick-and-dirty scripts grew into programs, and then grew into a full-blown system. Before long I had a monster on my hands, and I needed a tool that would look through all of the scripts and programs in the system and do (at least) some basic error checking.

I’d heard good things about pyflakes, so I thought I’d give it a try.

It worked very nicely. It found the basic kinds of errors that I wanted it to find. And it was fast, so I could run it through a directory containing a lot of .py files and it would come out alive and grinning on the other side.

During the process, I learned that pyflakes is designed to be a bit on the quick and dirty side itself, with the quick making up for the dirty. As part of this process, it basically ignores star imports.  Oh, it warns you about the star imports.  What I means is — it doesn’t try to figure out what is imported by the star import.

And that has interesting consequences.

Normally, if your file contains an undefined name — say TARGET_LANGAGE — pyflakes will report it as an error.

But if your file includes any star imports, and your script contains an undefined name like TARGET_LANGAGE, pyflakes won’t report the undefined name as an error.

My hypothesis is that pyflakes doesn’t report TARGET_LANGAGE as undefined because it can’t tell whether TARGET_LANGAGE is truly undefined, or was pulled in by some star import.

This is perfectly understandable. There is no way that pyflakes is going to go out, try to find the something_or_other module, and analyze it to see if it contains TARGET_LANGAGE. And if it doesn’t, but contains star imports, go out and look for all of the modules that something_or_other star imports, and then analyze them. And so on, and so on, and so on. No way!

So, since pyflakes can’t tell whether TARGET_LANGAGE is (a) an undefined name or (b) pulled in via some star import, it does not report TARGET_LANGAGE as an undefined name. Basically, pyflakes ignores it.

And that seems to me to be a perfectly reasonable way to do business, not just for pyflakes but for anything short of the Super Deluxe Hyperdrive model static code analyzer.

The takeaway lesson for me was that using star imports will cripple a static code analyzer. Or at least, cripple the feature that I most want a code analyser for… to find and report undefined names.

So now I don’t use star imports anymore.

There are a variety of alternatives.  The one that I prefer is the “import x as y” feature. So I can write an import statement this way:

import   some_module_with_a_big_long_hairy_name   as    bx

 and rather than coding

x = some_module_with_a_big_long_hairy_name.vector

I can code

x = bx.vector

Works for me.

6 Comments »

  1. I use some “standard” (jokingly, of course) abbreviations:

    import itertools as it, operator as op, functools as ft, unicodedata as ud

    I don’t think I abbreviate other stdlib modules.

    Comment by ΤΖΩΤΖΙΟΥ — 2011/03/28 @ 4:54 pm | Reply

  2. I always try to avoid star imports myself. I usually try to import only what I need (including functions and classes from stdlib modules and packages).

    Comment by James Mills — 2011/03/28 @ 7:31 pm | Reply

  3. I recently fixed some stuff in CVS of pychecker to deal with star imports and following what they import exactly.

    I´d be interested in having you try it out and see if it was able to catch the problems you saw that pyflakes couldn´t ?

    Feel free to drop me a line!

    T

    Comment by Thomas Vander Stichele — 2011/04/05 @ 6:12 pm | Reply

    • I hope to have some time soon. If I do, I’ll let you know what happens.

      Thanks for this update!

      Comment by Ferg, Stephen - BLS — 2011/04/06 @ 8:24 am | Reply

  4. I’m under the impression that pylint will actually load all the modules needed for star imports. You might want to give that a try!

    Comment by Anonymous — 2011/11/01 @ 12:31 pm | Reply

  5. Yeah, I would take away from it the opposite – don’t use pyflakes. It sounds “flaky.”

    Comment by Shiggity — 2012/01/07 @ 2:41 am | Reply


RSS feed for comments on this post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 48 other followers