Back to index

python3.2  3.2.2
Classes | Functions | Variables
pkgutil Namespace Reference

Classes

class  ImpImporter
class  ImpLoader

Functions

def read_code
def simplegeneric
def walk_packages
def iter_modules
def iter_importer_modules
def get_importer
def iter_importers
def get_loader
def find_loader
def extend_path
def get_data

Variables

list __all__
tuple iter_importer_modules = simplegeneric(iter_importer_modules)

Detailed Description

Utilities to support packages.

Function Documentation

def pkgutil.extend_path (   path,
  name 
)
Extend a package's path.

Intended use is to place the following code in a package's __init__.py:

    from pkgutil import extend_path
    __path__ = extend_path(__path__, __name__)

This will add to the package's __path__ all subdirectories of
directories on sys.path named after the package.  This is useful
if one wants to distribute different parts of a single logical
package as multiple directories.

It also looks for *.pkg files beginning where * matches the name
argument.  This feature is similar to *.pth files (see site.py),
except that it doesn't special-case lines starting with 'import'.
A *.pkg file is trusted at face value: apart from checking for
duplicates, all entries found in a *.pkg file are added to the
path, regardless of whether they are exist the filesystem.  (This
is a feature.)

If the input path is not a list (as is the case for frozen
packages) it is returned unchanged.  The input path is not
modified; an extended copy is returned.  Items are only appended
to the copy at the end.

It is assumed that sys.path is a sequence.  Items of sys.path that
are not (unicode or 8-bit) strings referring to existing
directories are ignored.  Unicode items of sys.path that cause
errors when used as filenames may cause this function to raise an
exception (in line with os.path.isdir() behavior).

Definition at line 471 of file pkgutil.py.

00471 
00472 def extend_path(path, name):
00473     """Extend a package's path.
00474 
00475     Intended use is to place the following code in a package's __init__.py:
00476 
00477         from pkgutil import extend_path
00478         __path__ = extend_path(__path__, __name__)
00479 
00480     This will add to the package's __path__ all subdirectories of
00481     directories on sys.path named after the package.  This is useful
00482     if one wants to distribute different parts of a single logical
00483     package as multiple directories.
00484 
00485     It also looks for *.pkg files beginning where * matches the name
00486     argument.  This feature is similar to *.pth files (see site.py),
00487     except that it doesn't special-case lines starting with 'import'.
00488     A *.pkg file is trusted at face value: apart from checking for
00489     duplicates, all entries found in a *.pkg file are added to the
00490     path, regardless of whether they are exist the filesystem.  (This
00491     is a feature.)
00492 
00493     If the input path is not a list (as is the case for frozen
00494     packages) it is returned unchanged.  The input path is not
00495     modified; an extended copy is returned.  Items are only appended
00496     to the copy at the end.
00497 
00498     It is assumed that sys.path is a sequence.  Items of sys.path that
00499     are not (unicode or 8-bit) strings referring to existing
00500     directories are ignored.  Unicode items of sys.path that cause
00501     errors when used as filenames may cause this function to raise an
00502     exception (in line with os.path.isdir() behavior).
00503     """
00504 
00505     if not isinstance(path, list):
00506         # This could happen e.g. when this is called from inside a
00507         # frozen package.  Return the path unchanged in that case.
00508         return path
00509 
00510     pname = os.path.join(*name.split('.')) # Reconstitute as relative path
00511     sname_pkg = name + ".pkg"
00512     init_py = "__init__.py"
00513 
00514     path = path[:] # Start with a copy of the existing path
00515 
00516     for dir in sys.path:
00517         if not isinstance(dir, str) or not os.path.isdir(dir):
00518             continue
00519         subdir = os.path.join(dir, pname)
00520         # XXX This may still add duplicate entries to path on
00521         # case-insensitive filesystems
00522         initfile = os.path.join(subdir, init_py)
00523         if subdir not in path and os.path.isfile(initfile):
00524             path.append(subdir)
00525         # XXX Is this the right thing for subpackages like zope.app?
00526         # It looks for a file named "zope.app.pkg"
00527         pkgfile = os.path.join(dir, sname_pkg)
00528         if os.path.isfile(pkgfile):
00529             try:
00530                 f = open(pkgfile)
00531             except IOError as msg:
00532                 sys.stderr.write("Can't open %s: %s\n" %
00533                                  (pkgfile, msg))
00534             else:
00535                 for line in f:
00536                     line = line.rstrip('\n')
00537                     if not line or line.startswith('#'):
00538                         continue
00539                     path.append(line) # Don't check for existence!
00540                 f.close()
00541 
00542     return path

def pkgutil.find_loader (   fullname)
Find a PEP 302 "loader" object for fullname

If fullname contains dots, path must be the containing package's __path__.
Returns None if the module cannot be found or imported. This function uses
iter_importers(), and is thus subject to the same limitations regarding
platform-specific special import locations such as the Windows registry.

Definition at line 455 of file pkgutil.py.

00455 
00456 def find_loader(fullname):
00457     """Find a PEP 302 "loader" object for fullname
00458 
00459     If fullname contains dots, path must be the containing package's __path__.
00460     Returns None if the module cannot be found or imported. This function uses
00461     iter_importers(), and is thus subject to the same limitations regarding
00462     platform-specific special import locations such as the Windows registry.
00463     """
00464     for importer in iter_importers(fullname):
00465         loader = importer.find_module(fullname)
00466         if loader is not None:
00467             return loader
00468 
00469     return None
00470 

Here is the call graph for this function:

Here is the caller graph for this function:

def pkgutil.get_data (   package,
  resource 
)
Get a resource from a package.

This is a wrapper round the PEP 302 loader get_data API. The package
argument should be the name of a package, in standard module format
(foo.bar). The resource argument should be in the form of a relative
filename, using '/' as the path separator. The parent directory name '..'
is not allowed, and nor is a rooted name (starting with a '/').

The function returns a binary string, which is the contents of the
specified resource.

For packages located in the filesystem, which have already been imported,
this is the rough equivalent of

    d = os.path.dirname(sys.modules[package].__file__)
    data = open(os.path.join(d, resource), 'rb').read()

If the package cannot be located or loaded, or it uses a PEP 302 loader
which does not support get_data(), then None is returned.

Definition at line 543 of file pkgutil.py.

00543 
00544 def get_data(package, resource):
00545     """Get a resource from a package.
00546 
00547     This is a wrapper round the PEP 302 loader get_data API. The package
00548     argument should be the name of a package, in standard module format
00549     (foo.bar). The resource argument should be in the form of a relative
00550     filename, using '/' as the path separator. The parent directory name '..'
00551     is not allowed, and nor is a rooted name (starting with a '/').
00552 
00553     The function returns a binary string, which is the contents of the
00554     specified resource.
00555 
00556     For packages located in the filesystem, which have already been imported,
00557     this is the rough equivalent of
00558 
00559         d = os.path.dirname(sys.modules[package].__file__)
00560         data = open(os.path.join(d, resource), 'rb').read()
00561 
00562     If the package cannot be located or loaded, or it uses a PEP 302 loader
00563     which does not support get_data(), then None is returned.
00564     """
00565 
00566     loader = get_loader(package)
00567     if loader is None or not hasattr(loader, 'get_data'):
00568         return None
00569     mod = sys.modules.get(package) or loader.load_module(package)
00570     if mod is None or not hasattr(mod, '__file__'):
00571         return None
00572 
00573     # Modify the resource name to be compatible with the loader.get_data
00574     # signature - an os.path format "filename" starting with the dirname of
00575     # the package's __file__
00576     parts = resource.split('/')
00577     parts.insert(0, os.path.dirname(mod.__file__))
00578     resource_name = os.path.join(*parts)
00579     return loader.get_data(resource_name)

Here is the call graph for this function:

Here is the caller graph for this function:

def pkgutil.get_importer (   path_item)
Retrieve a PEP 302 importer for the given path item

The returned importer is cached in sys.path_importer_cache
if it was newly created by a path hook.

If there is no importer, a wrapper around the basic import
machinery is returned. This wrapper is never inserted into
the importer cache (None is inserted instead).

The cache (or part of it) can be cleared manually if a
rescan of sys.path_hooks is necessary.

Definition at line 355 of file pkgutil.py.

00355 
00356 def get_importer(path_item):
00357     """Retrieve a PEP 302 importer for the given path item
00358 
00359     The returned importer is cached in sys.path_importer_cache
00360     if it was newly created by a path hook.
00361 
00362     If there is no importer, a wrapper around the basic import
00363     machinery is returned. This wrapper is never inserted into
00364     the importer cache (None is inserted instead).
00365 
00366     The cache (or part of it) can be cleared manually if a
00367     rescan of sys.path_hooks is necessary.
00368     """
00369     try:
00370         importer = sys.path_importer_cache[path_item]
00371     except KeyError:
00372         for path_hook in sys.path_hooks:
00373             try:
00374                 importer = path_hook(path_item)
00375                 break
00376             except ImportError:
00377                 pass
00378         else:
00379             importer = None
00380         sys.path_importer_cache.setdefault(path_item, importer)
00381 
00382     if importer is None:
00383         try:
00384             importer = ImpImporter(path_item)
00385         except ImportError:
00386             importer = None
00387     return importer
00388 

Here is the caller graph for this function:

def pkgutil.get_loader (   module_or_name)
Get a PEP 302 "loader" object for module_or_name

If the module or package is accessible via the normal import
mechanism, a wrapper around the relevant part of that machinery
is returned.  Returns None if the module cannot be found or imported.
If the named module is not already imported, its containing package
(if any) is imported, in order to establish the package __path__.

This function uses iter_importers(), and is thus subject to the same
limitations regarding platform-specific special import locations such
as the Windows registry.

Definition at line 430 of file pkgutil.py.

00430 
00431 def get_loader(module_or_name):
00432     """Get a PEP 302 "loader" object for module_or_name
00433 
00434     If the module or package is accessible via the normal import
00435     mechanism, a wrapper around the relevant part of that machinery
00436     is returned.  Returns None if the module cannot be found or imported.
00437     If the named module is not already imported, its containing package
00438     (if any) is imported, in order to establish the package __path__.
00439 
00440     This function uses iter_importers(), and is thus subject to the same
00441     limitations regarding platform-specific special import locations such
00442     as the Windows registry.
00443     """
00444     if module_or_name in sys.modules:
00445         module_or_name = sys.modules[module_or_name]
00446     if isinstance(module_or_name, ModuleType):
00447         module = module_or_name
00448         loader = getattr(module, '__loader__', None)
00449         if loader is not None:
00450             return loader
00451         fullname = module.__name__
00452     else:
00453         fullname = module_or_name
00454     return find_loader(fullname)

Here is the call graph for this function:

Here is the caller graph for this function:

def pkgutil.iter_importer_modules (   importer,
  prefix = '' 
)

Definition at line 151 of file pkgutil.py.

00151 
00152 def iter_importer_modules(importer, prefix=''):
00153     if not hasattr(importer, 'iter_modules'):
00154         return []
00155     return importer.iter_modules(prefix)

def pkgutil.iter_importers (   fullname = "")
Yield PEP 302 importers for the given module name

If fullname contains a '.', the importers will be for the package
containing fullname, otherwise they will be importers for sys.meta_path,
sys.path, and Python's "classic" import machinery, in that order.  If
the named module is in a package, that package is imported as a side
effect of invoking this function.

Non PEP 302 mechanisms (e.g. the Windows registry) used by the
standard import machinery to find files in alternative locations
are partially supported, but are searched AFTER sys.path. Normally,
these locations are searched BEFORE sys.path, preventing sys.path
entries from shadowing them.

For this to cause a visible difference in behaviour, there must
be a module or package name that is accessible via both sys.path
and one of the non PEP 302 file system mechanisms. In this case,
the emulation will find the former version, while the builtin
import mechanism will find the latter.

Items of the following types can be affected by this discrepancy:
    imp.C_EXTENSION, imp.PY_SOURCE, imp.PY_COMPILED, imp.PKG_DIRECTORY

Definition at line 389 of file pkgutil.py.

00389 
00390 def iter_importers(fullname=""):
00391     """Yield PEP 302 importers for the given module name
00392 
00393     If fullname contains a '.', the importers will be for the package
00394     containing fullname, otherwise they will be importers for sys.meta_path,
00395     sys.path, and Python's "classic" import machinery, in that order.  If
00396     the named module is in a package, that package is imported as a side
00397     effect of invoking this function.
00398 
00399     Non PEP 302 mechanisms (e.g. the Windows registry) used by the
00400     standard import machinery to find files in alternative locations
00401     are partially supported, but are searched AFTER sys.path. Normally,
00402     these locations are searched BEFORE sys.path, preventing sys.path
00403     entries from shadowing them.
00404 
00405     For this to cause a visible difference in behaviour, there must
00406     be a module or package name that is accessible via both sys.path
00407     and one of the non PEP 302 file system mechanisms. In this case,
00408     the emulation will find the former version, while the builtin
00409     import mechanism will find the latter.
00410 
00411     Items of the following types can be affected by this discrepancy:
00412         imp.C_EXTENSION, imp.PY_SOURCE, imp.PY_COMPILED, imp.PKG_DIRECTORY
00413     """
00414     if fullname.startswith('.'):
00415         raise ImportError("Relative module names not supported")
00416     if '.' in fullname:
00417         # Get the containing package's __path__
00418         pkg = '.'.join(fullname.split('.')[:-1])
00419         if pkg not in sys.modules:
00420             __import__(pkg)
00421         path = getattr(sys.modules[pkg], '__path__', None) or []
00422     else:
00423         for importer in sys.meta_path:
00424             yield importer
00425         path = sys.path
00426     for item in path:
00427         yield get_importer(item)
00428     if '.' not in fullname:
00429         yield ImpImporter()

Here is the call graph for this function:

Here is the caller graph for this function:

def pkgutil.iter_modules (   path = None,
  prefix = '' 
)
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.

Definition at line 126 of file pkgutil.py.

00126 
00127 def iter_modules(path=None, prefix=''):
00128     """Yields (module_loader, name, ispkg) for all submodules on path,
00129     or, if path is None, all top-level modules on sys.path.
00130 
00131     'path' should be either None or a list of paths to look for
00132     modules in.
00133 
00134     'prefix' is a string to output on the front of every module name
00135     on output.
00136     """
00137 
00138     if path is None:
00139         importers = iter_importers()
00140     else:
00141         importers = map(get_importer, path)
00142 
00143     yielded = {}
00144     for i in importers:
00145         for name, ispkg in iter_importer_modules(i, prefix):
00146             if name not in yielded:
00147                 yielded[name] = 1
00148                 yield i, name, ispkg
00149 
00150 
#@simplegeneric

Here is the call graph for this function:

Here is the caller graph for this function:

def pkgutil.read_code (   stream)

Definition at line 15 of file pkgutil.py.

00015 
00016 def read_code(stream):
00017     # This helper is needed in order for the PEP 302 emulation to
00018     # correctly handle compiled files
00019     import marshal
00020 
00021     magic = stream.read(4)
00022     if magic != imp.get_magic():
00023         return None
00024 
00025     stream.read(4) # Skip timestamp
00026     return marshal.load(stream)
00027 

Here is the caller graph for this function:

def pkgutil.simplegeneric (   func)
Make a trivial single-dispatch generic function

Definition at line 28 of file pkgutil.py.

00028 
00029 def simplegeneric(func):
00030     """Make a trivial single-dispatch generic function"""
00031     registry = {}
00032     def wrapper(*args, **kw):
00033         ob = args[0]
00034         try:
00035             cls = ob.__class__
00036         except AttributeError:
00037             cls = type(ob)
00038         try:
00039             mro = cls.__mro__
00040         except AttributeError:
00041             try:
00042                 class cls(cls, object):
00043                     pass
00044                 mro = cls.__mro__[1:]
00045             except TypeError:
00046                 mro = object,   # must be an ExtensionClass or some such  :(
00047         for t in mro:
00048             if t in registry:
00049                 return registry[t](*args, **kw)
00050         else:
00051             return func(*args, **kw)
00052     try:
00053         wrapper.__name__ = func.__name__
00054     except (TypeError, AttributeError):
00055         pass    # Python 2.3 doesn't allow functions to be renamed
00056 
00057     def register(typ, func=None):
00058         if func is None:
00059             return lambda f: register(typ, f)
00060         registry[typ] = func
00061         return func
00062 
00063     wrapper.__dict__ = func.__dict__
00064     wrapper.__doc__ = func.__doc__
00065     wrapper.register = register
00066     return wrapper
00067 

Here is the call graph for this function:

def pkgutil.walk_packages (   path = None,
  prefix = '',
  onerror = None 
)
Yields (module_loader, name, ispkg) for all modules recursively
on path, or, if path is None, all accessible modules.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.

Note that this function must import all *packages* (NOT all
modules!) on the given path, in order to access the __path__
attribute to find submodules.

'onerror' is a function which gets called with one argument (the
name of the package which was being imported) if any exception
occurs while trying to import a package.  If no onerror function is
supplied, ImportErrors are caught and ignored, while all other
exceptions are propagated, terminating the search.

Examples:

# list all modules python can access
walk_packages()

# list all submodules of ctypes
walk_packages(ctypes.__path__, ctypes.__name__+'.')

Definition at line 68 of file pkgutil.py.

00068 
00069 def walk_packages(path=None, prefix='', onerror=None):
00070     """Yields (module_loader, name, ispkg) for all modules recursively
00071     on path, or, if path is None, all accessible modules.
00072 
00073     'path' should be either None or a list of paths to look for
00074     modules in.
00075 
00076     'prefix' is a string to output on the front of every module name
00077     on output.
00078 
00079     Note that this function must import all *packages* (NOT all
00080     modules!) on the given path, in order to access the __path__
00081     attribute to find submodules.
00082 
00083     'onerror' is a function which gets called with one argument (the
00084     name of the package which was being imported) if any exception
00085     occurs while trying to import a package.  If no onerror function is
00086     supplied, ImportErrors are caught and ignored, while all other
00087     exceptions are propagated, terminating the search.
00088 
00089     Examples:
00090 
00091     # list all modules python can access
00092     walk_packages()
00093 
00094     # list all submodules of ctypes
00095     walk_packages(ctypes.__path__, ctypes.__name__+'.')
00096     """
00097 
00098     def seen(p, m={}):
00099         if p in m:
00100             return True
00101         m[p] = True
00102 
00103     for importer, name, ispkg in iter_modules(path, prefix):
00104         yield importer, name, ispkg
00105 
00106         if ispkg:
00107             try:
00108                 __import__(name)
00109             except ImportError:
00110                 if onerror is not None:
00111                     onerror(name)
00112             except Exception:
00113                 if onerror is not None:
00114                     onerror(name)
00115                 else:
00116                     raise
00117             else:
00118                 path = getattr(sys.modules[name], '__path__', None) or []
00119 
00120                 # don't traverse path items we've seen before
00121                 path = [p for p in path if not seen(p)]
00122 
00123                 for item in walk_packages(path, name+'.', onerror):
00124                     yield item
00125 

Here is the call graph for this function:

Here is the caller graph for this function:


Variable Documentation

Initial value:
00001 [
00002     'get_importer', 'iter_importers', 'get_loader', 'find_loader',
00003     'walk_packages', 'iter_modules', 'get_data',
00004     'ImpImporter', 'ImpLoader', 'read_code', 'extend_path',
00005 ]

Definition at line 9 of file pkgutil.py.

Definition at line 156 of file pkgutil.py.