Required Attributes

These are attributes in sys.implementation on which the standard library and language definition will rely, meaning implementers must define them:

name

A lower-case identifier representing the implementation. Examples include ‘pypy’, ‘jython’, ‘ironpython’, and ‘cpython’.

version

The version of the implementation, as opposed to the version of the language it implements. This value conforms to the format described in Version Format.

hexversion

The version of the implementation in the same hexadecimal format as sys.hexversion.

cache_tag

A string used for the PEP 3147 cache tag 12. It would normally be a composite of the name and version (e.g. ‘cpython-33’ for CPython 3.3). However, an implementation may explicitly use a different cache tag. If cache_tag is set to None, it indicates that module caching should be disabled.

Adding New Required Attributes

In time more required attributes will be added to sys.implementation. However, each must have a meaningful use case across all Python implementations in order to be considered. This is made most clear by a use case in the standard library or language specification.

All proposals for new required attributes will go through the normal PEP process. Such a PEP need not be long, just long enough. It will need to sufficiently spell out the rationale for the new attribute, its use cases, and the impact it will have on the various Python implementations.

Version Format

A main point of sys.implementation is to contain information that will be used internally in the standard library. In order to facilitate the usefulness of the version attribute, its value should be in a consistent format across implementations.

As such, the format of sys.implementation.version will follow that of sys.version_info, which is effectively a named tuple. It is a familiar format and generally consistent with normal version format conventions.

Type Considerations

It’s very easy to get bogged down in discussions about the type of sys.implementation. However, its purpose is to support the standard library and language definition. As such, there isn’t much that really matters regarding its type, as opposed to a feature that would be more generally used. Thus characteristics like immutability and sequence-ness have been disregarded.

The only real choice has been between an object with attribute access and a mapping with item access. This PEP espouses dotted access to reflect the relatively fixed nature of the namespace.

Non-Required Attributes

Earlier versions of this PEP included a required attribute called metadata that held any non-required, per-implementation data 17. However, this proved to be an unnecessary addition considering the purpose of sys.implementation.

Ultimately, non-required attributes are virtually ignored in this PEP. They have no impact other than that careless use may collide with future required attributes. That, however, is but a marginal concern for sys.implementation.

Why a Part of sys?

The sys module holds the new namespace because sys is the depot for interpreter-centric variables and functions. Many implementation-specific attributes are already found in sys.

Why Strict Constraints on Any of the Values?

As already noted in Version Format, values in sys.implementation are intended for use by the standard library. Constraining those values, essentially specifying an API for them, allows them to be used consistently, regardless of how they are otherwise implemented. However, care should be take to not over-specify the constraints.

platform.python_implementation()

“explicit is better than implicit”

The platform module determines the python implementation by looking for clues in a couple different sys variables 11. However, this approach is fragile, requiring changes to the standard library each time an implementation changes. Beyond that, support in platform is limited to those implementations that core developers have blessed by special-casing them in the platform module.

With sys.implementation the various implementations would explicitly set the values in their own version of the sys module.

Another concern is that the platform module is part of the stdlib, which ideally should minimize implementation details such as would be moved to sys.implementation.

Any overlap between sys.implementation and the platform module would simply defer to sys.implementation (with the same interface in platform wrapping it).

Cache Tag Generation in Frozen Importlib

PEP 3147 defined the use of a module cache and cache tags for file names. The importlib bootstrap code, frozen into the Python binary as of 3.3, uses the cache tags during the import process. Part of the project to bootstrap importlib has been to clean code out of Python/import.c 21 that did not need to be there any longer.

The cache tag defined in Python/import.c was hard-coded to "cpython" MAJOR MINOR 12. For importlib the options are either hard-coding it in the same way, or guessing the implementation in the same way as does platform.python_implementation().

As long as the hard-coded tag is limited to CPython-specific code, it is livable. However, inasmuch as other Python implementations use the importlib code to work with the module cache, a hard-coded tag would become a problem.

Directly using the platform module in this case is a non-starter. Any module used in the importlib bootstrap must be built-in or frozen, neither of which apply to the platform module. This is the point that led to the recent interest in sys.implementation.

Regardless of the outcome for the implementation name used, another problem relates to the version used in the cache tag. That version is likely to be the implementation version rather than the language version. However, the implementation version is not readily identified anywhere in the standard library.

Implementation-Specific Tests

Currently there are a number of implementation-specific tests in the test suite under Lib/test. The test support module (Lib/test/support.py 20) provides some functionality for dealing with these tests. However, like the platform module, test.support must do some guessing that sys.implementation would render unnecessary.

Jython’s os.name Hack

In Jython, os.name is set to ‘java’ to accommodate special treatment of the java environment in the standard library 15 16. Unfortunately it masks the os name that would otherwise go there. sys.implementation would help obviate the need for this special case. Currently Jython sets os._name for the normal os.name value.

The Problem With sys.(version|version_info|hexversion)

Earlier versions of this PEP made the mistake of calling sys.version_info (and friends) the version of the Python language, in contrast to the implementation. However, this is not the case. Instead, it is the version of the CPython implementation. Incidentally, the first two components of sys.version_info (major and minor) also reflect the version of the language definition.

As Barry Warsaw noted, the “semantics of sys.version_info have been sufficiently squishy in the past” 18. With sys.implementation we have the opportunity to improve this situation by first establishing an explicit location for the version of the implementation.

This PEP makes no other effort to directly clarify the semantics of sys.version_info. Regardless, having an explicit version for the implementation will definitely help to clarify the distinction from the language version.

IronPython

Jeff Hardy responded to a request for feedback 4. He said, “I’ll probably add it the day after it’s approved” 6. He also gave useful feedback on both the type of sys.implementation and on the metadata attribute (which has since been removed from the PEP).

Jython

In 2009 Frank Wierzbicki said this (relative to Jython implementing the required attributes) 8:

Speaking for Jython, so far it looks like something we would adopt
soonish after it was accepted (it looks pretty useful to me).

PyPy

Some of the PyPy developers have responded to a request for feedback 9. Armin Rigo said the following 10:

For myself, I can only say that it looks like a good idea, which we
will happily adhere to when we migrate to Python 3.3.

He also expressed support for keeping the required list small. Both Armin and Laura Creighton indicated that an effort to better catalog Python’s implementation would be welcome. Such an effort, for which this PEP is a small start, will be considered separately.

PEP 3139

PEP 3139, from 2008, recommended a clean-up of the sys module in part by extracting implementation-specific variables and functions into a separate module. PEP 421 is less ambitious version of that idea. While PEP 3139 was rejected, its goals are reflected in PEP 421 to a large extent, though with a much lighter approach.

PEP 399

PEP 399 dictates policy regarding the standard library, helping to make it friendlier to alternate implementations. PEP 421 is proposed in that same spirit.