diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -433,6 +433,16 @@ :data:`sys.modules`, but it must remove **only** the failing module, and only if the loader itself has loaded it explicitly. +Module loaders may opt in to creating the module object during loading +by implementing a :meth:`create_module()` method. It takes one argument, +the module spec, and returns the new module object to use during loading. +create_module() does not need to set any attributes on the module object. +If the loader does not define create_module(), the import machinery will +create the new module itself. + +.. versionadded:: 3.4 + The create_module() method of loaders. + .. versionchanged:: 3.4 The load_module() method was replaced by exec_module() and the import machinery assumed all the boilerplate responsibilities of loading. @@ -452,54 +462,68 @@ whereas without a module spec the loader had that responsibility. See :class:`~importlib.machinery.ModuleSpec` for more specifics on what -information a module's spec holds. +information a module's spec may hold. .. versionadded:: 3.4 - Import-related module attributes -------------------------------- +.. attribute:: __name__ + + The import machinery may set the ``__name__`` attribute of the + module. While not required, setting this attribute is highly + recommended so that the :meth:`repr()` of the module is more + informative. + +.. attribute:: __loader__ + + The ``__loader__`` attribute must be set to the loader object that + loaded the module. This is mostly for introspection and reloading, + but can be used for additional loader-specific functionality, for + example getting data associated with a loader. If the attribute is + missing or set to ``None`` then the import machinery will + automatically set it **after** the module has been imported. + +.. attribute:: __package__ + + The module's ``__package__`` attribute must be set. Its value must + be a string, but it can be the same value as its ``__name__``. If + the attribute is set to ``None`` or is missing, the import system + will fill it in with a more appropriate value **after** the module + has been imported. When the module is a package, its ``__package__`` + value should be set to its ``__name__``. When the module is not a + package, ``__package__`` should be set to the empty string for + top-level modules, or for submodules, to the parent package's name. + See :pep:`366` for further details. + + This attribute is used instead of ``__name__`` to calculate explicit + relative imports for main modules, as defined in :pep:`366`. + +.. attribute:: __spec__ + + The module spec that was used when importing the module. + +.. attribute:: __path__ + + If the module is a package (either regular or namespace), the loader must + set the module object's ``__path__`` attribute. The value must be + iterable, but may be empty if ``__path__`` has no further significance + to the loader. If ``__path__`` is not empty, it must produce strings + when iterated over. More details on the semantics of ``__path__`` are + given :ref:`below `. + .. attribute:: __file__ +.. attribute:: __cached__ ``__file__`` is optional. If set, this attribute's value must be a - string. The loader may opt to leave - ``__file__`` unset if it has no semantic meaning (e.g. a module loaded from - a database). If ``__file__`` is set, it may also be appropriate to set the - ``__cached__`` attribute which is the path to any compiled version of the - code (e.g. byte-compiled file). The file does not need to exist to set this - attribute; the path can simply point to whether the compiled file would - exist (see :pep:`3147`). - - * The loader may set the ``__name__`` attribute of the module. While not - required, setting this attribute is highly recommended so that the - :meth:`repr()` of the module is more informative. - - * If the module is a package (either regular or namespace), the loader must - set the module object's ``__path__`` attribute. The value must be - iterable, but may be empty if ``__path__`` has no further significance - to the loader. If ``__path__`` is not empty, it must produce strings - when iterated over. More details on the semantics of ``__path__`` are - given :ref:`below `. - - * The ``__loader__`` attribute must be set to the loader object that loaded - the module. This is mostly for introspection and reloading, but can be - used for additional loader-specific functionality, for example getting - data associated with a loader. If the attribute is missing or set to ``None`` - then the import machinery will automatically set it **after** the module has - been imported. - - * The module's ``__package__`` attribute must be set. Its value must be a - string, but it can be the same value as its ``__name__``. If the attribute - is set to ``None`` or is missing, the import system will fill it in with a - more appropriate value **after** the module has been imported. - When the module is a package, its ``__package__`` value should be set to its - ``__name__``. When the module is not a package, ``__package__`` should be - set to the empty string for top-level modules, or for submodules, to the - parent package's name. See :pep:`366` for further details. - - This attribute is used instead of ``__name__`` to calculate explicit - relative imports for main modules, as defined in :pep:`366`. + string. The import system may opt to leave ``__file__`` unset if it + has no semantic meaning (e.g. a module loaded from a database). If + ``__file__`` is set, it may also be appropriate to set the + ``__cached__`` attribute which is the path to any compiled version of + the code (e.g. byte-compiled file). The file does not need to exist + to set this attribute; the path can simply point to whether the + compiled file would exist (see :pep:`3147`). .. _package-path-rules: @@ -524,28 +548,27 @@ attribute, and this was typically the way namespace packages were implemented prior to :pep:`420`. With the adoption of :pep:`420`, namespace packages no longer need to supply ``__init__.py`` files containing only ``__path__`` -manipulation code; the namespace loader automatically sets ``__path__`` +manipulation code; the import machinery automatically sets ``__path__`` correctly for the namespace package. Module reprs ------------ By default, all modules have a usable repr, however depending on the -attributes set above, and hooks in the loader, you can more explicitly control -the repr of module objects. +attributes set above, and in the module's spec, you can more explicitly +control the repr of module objects. -Loaders may implement a :meth:`module_repr()` method which takes a single -argument, the module object. When ``repr(module)`` is called for a module -with a loader supporting this protocol, whatever is returned from -``module.__loader__.module_repr(module)`` is returned as the module's repr -without further processing. This return value must be a string. +If the module has a spec (``__spec__``), the import machinery will try +to generate a repr from it. If that fails or there is no spec, the import +system will craft a default repr using whatever information is available +on the module. It will try to use the ``module.__name__``, +``module.__file__``, and ``module.__loader__`` as input into the repr, +with defaults for whatever information is missing. -If the module has no ``__loader__`` attribute, or the loader has no -:meth:`module_repr()` method, then the module object implementation itself -will craft a default repr using whatever information is available. It will -try to use the ``module.__name__``, ``module.__file__``, and -``module.__loader__`` as input into the repr, with defaults for whatever -information is missing. +For backward compatibility with Python 3.3, the module repr will be +generated by calling the loader's :meth:`module_repr()` method, if +defined, before trying either approach described above. However, the +method is deprecated. Here are the exact rules used: @@ -557,6 +580,10 @@ and discarded, and the calculation of the module's repr continues as if :meth:`module_repr()` did not exist. + * If the module has a ``__spec__`` attribute, the information in the spec + is used to generate the repr. The "name", "loader", "origin", and + "has_location" attributes are consulted. + * If the module has a ``__file__`` attribute, this is used as part of the module's repr. @@ -565,13 +592,9 @@ * Otherwise, just use the module's ``__name__`` in the repr. -This example, from :pep:`420` shows how a loader can craft its own module -repr:: - - class NamespaceLoader: - @classmethod - def module_repr(cls, module): - return "".format(module.__name__) +.. versionchanged:: 3.4 + Use of loader.module_repr() has been deprecated and the module spec + is now used by the import machinery to generate a module repr. The Path Based Finder @@ -637,7 +660,7 @@ not be limited to this. As a meta path finder, the :term:`path based finder` implements the -:meth:`find_module()` protocol previously described, however it exposes +:meth:`find_spec()` protocol previously described, however it exposes additional hooks that can be used to customize how modules are found and loaded from the :term:`import path`. @@ -659,8 +682,8 @@ The :term:`path based finder` is a :term:`meta path finder`, so the import machinery begins the :term:`import path` search by calling the path -based finder's :meth:`find_module()` method as described previously. When -the ``path`` argument to :meth:`find_module()` is given, it will be a +based finder's :meth:`find_spec()` method as described previously. When +the ``path`` argument to :meth:`find_spec()` is given, it will be a list of string paths to traverse - typically a package's ``__path__`` attribute for an import within that package. If the ``path`` argument is ``None``, this indicates a top level import and :data:`sys.path` is used. @@ -691,22 +714,41 @@ argument, it should raise :exc:`ImportError`. If :data:`sys.path_hooks` iteration ends with no :term:`path entry finder` -being returned, then the path based finder's :meth:`find_module()` method +being returned, then the path based finder's :meth:`find_spec()` method will store ``None`` in :data:`sys.path_importer_cache` (to indicate that there is no finder for this path entry) and return ``None``, indicating that this :term:`meta path finder` could not find the module. If a :term:`path entry finder` *is* returned by one of the :term:`path entry hook` callables on :data:`sys.path_hooks`, then the following protocol is used -to ask the finder for a module loader, which is then used to load the module. - +to ask the finder for a module spec, which is then used when loading the +module. Path entry finder protocol -------------------------- In order to support imports of modules and initialized packages and also to contribute portions to namespace packages, path entry finders must implement -the :meth:`find_loader()` method. +the :meth:`find_spec()` method. + +:meth:`find_spec()` takes one argument, the fully qualified name of the +module being imported. :meth:`find_spec()` returns a fully populated +spec for the module. This spec will always have "loader" set (with one +exception). + +To indicate to the import machinery that the spec represents a namespace +:term:`portion`. the path entry finder sets "loader" on the spec to +``None`` and "submodule_search_locations" to a list containing the +portion. + +Legacy path entry finder protocols +---------------------------------- + +Older path entry finders may implement one of two deprecated methods +instead of :meth:`find_spec()`: :meth:`find_loader()` and +:meth:`find_module()`. The methods are still respected for the sake of +backward compatibility. Howevever, if find_spec() is implemented on the +path entry finder, the legacy methods are ignored. :meth:`find_loader()` takes one argument, the fully qualified name of the module being imported. :meth:`find_loader()` returns a 2-tuple where the @@ -732,10 +774,9 @@ The :meth:`find_module()` method on path entry finders is deprecated, as it does not allow the path entry finder to contribute portions to -namespace packages. Instead path entry finders should implement the -:meth:`find_loader()` method as described above. If it exists on the path -entry finder, the import system will always call :meth:`find_loader()` -in preference to :meth:`find_module()`. +namespace packages. If both :meth:`find_loader()` and :meth:`find_module()` +exist on a path entry finder, the import system will always call +:meth:`find_loader()` in preference to :meth:`find_module()`. Replacing the standard import system @@ -754,7 +795,7 @@ To selectively prevent import of some modules from a hook early on the meta path (rather than disabling the standard import system entirely), it is sufficient to raise :exc:`ImportError` directly from -:meth:`find_module` instead of returning ``None``. The latter indicates +:meth:`find_spec` instead of returning ``None``. The latter indicates that the meta path search should continue. while raising an exception terminates it immediately. @@ -796,6 +837,11 @@ :pep:`338` defines executing modules as scripts. +:pep:`451` adds the encapsulation of per-module import state in spec +objects. At also off-loads most of the boilerplate responsibilities of +loaders back onto the import machinery. These changes allow the +deprecation of several APIs in the import system and also addition of new +methods to finders and loaders. .. rubric:: Footnotes