Renaming to “Recording installed projects”

The standard is renamed from Database of Installed Python Distributions to Recording installed projects.

While putting files in known locations on disk may be thought of as a “database”, it’s not what most people think about when they hear the term. The PyPA links to PEP 376 under the heading Recording installed distributions.

The PyPA glossary defines “Distribution” (or, “Distribution Package” to prevent confusion with e.g. Linux distributions) as “A versioned archive file […]”. Since there may be other ways to install Python code than from archive files, the document uses “installed project” rather than “installed distribution”.

Removal of Implementation Details

All tool- and library-specific details are removed. The mechanisms of how a project is installed are also left out: the document focuses on the end state. One exception is a sketch of an uninstallation algorithm, which is given to better explain the purpose of the RECORD file.

References to .egg-info and .egg, formats specific to setuptools and distutils, are left out.

Explicitly Allowing Additional Files

The .dist-info directory is allowed to contain files not specified in the spec. The current tools already do this.

A note in the specification mentions files in the .dist-info directory of wheels. Current tools copy these files to the installed .dist-info—something to keep in mind for further standardization efforts.

Clarifications in the RECORD File

The CSV dialect is specified to be the default of Python’s csv module. This resolves edge cases around handling double-quotes and line terminators in file names.

The “base” of relative paths in RECORD is specified relative to the .dist-info directory, rather than tool-specific --install-lib and --prefix options.

Both hash and size fields are now optional (for any file, not just .pyc, .pyo and RECORD). Leavng them out is discouraged, except for *.pyc and RECORD itself. (Note that PEP 376 is unclear on what was optional; when taken literally, its text and examples contradict. Despite that, “both fields are optional“ is a reasonable interpretation of PEP 376. The alternative would be to mandate—rather than recommend—which files can be recorded without hash and size, and to update that list over time as new use cases come up.)

The new spec explicitly says that the RECORD file must now include all files of the installed project (the exception for .pyc files remains). Since tools use RECORD for uninstallation, incomplete file lists could introduce orphaned files to users’ environments. On the other hand, this means that there is no way to record hashes of some any files if the full list of files is unknown.

A sketch of an uninstallation algorithm is included to clarify the file’s primary purpose and contents.

Tools must not uninstall/remove projects that lack a RECORD file (unless they have external information, such as in system package managers of Linux distros).

On Windows, files in RECORD may be separated by either / or \. PEP 376 was unclear on this: it mandates forward slashes in one place, but shows backslackes in a Windows-specific example.

Optional RECORD File

The RECORD file is made optional. Not all tools can easily generate a list of installed files in a Python-specific format.

Specifically, the RECORD file is unnecessary when projects are installed by a Linux system packaging system, which has its own ways to keep track of files, uninstall them or check their integrity. Having to keep a RECORD file in sync with the disk and the system package database would be unreasonably fragile, and no RECORD file is better than one that does not correspond to reality.

(Full disclosure: The author of this PEP is an RPM packager active in the Fedora Linux distro.)

Optional INSTALLER File

The INSTALLER file is also made optional, and specified to be used for informational purposes only. It is still a single-line text file containing the name of the installer.

This file was originally added to distinguish projects installed by the Python installer (pip) from ones installed by other package managers (e.g. dnf). There were attempts to use this file to prevent pip from updating or uninstalling packages it didn’t install.

Our goal is supporting interoperating tools, and basing any action on which tool happened to install a package runs counter to that goal.

Instead of relying on the installer name, tools should use feature detection. The current document offers a crude way of making a project untouchable by Python tooling: omitting RECORD file.

On the other hand, the installer name may be useful in hints to the user.

To align with this new purpose of the file, the new specification allows any ASCII string in INSTALLER, rather than a lowercase identifier. It also suggests using the command-line command, if available.

The REQUESTED File: Removed from Spec

The REQUESTED file is now considered a tool-specific extension.

Per PEP 376, REQUESTED was to be written when a project was installed by direct user request, as opposed to automatically to satisfy dependencies of another project. Projects without this marker file could be uninstalled when no longer needed.

Despite the standard, many existing installers (including older versions of pip) never write this file. There is no distinction between projects that are “OK to remove when no longer needed” and ones simply installed by a tool that ignores REQUESTED. So, the file is currently not usable for its intended purpose (unless a tool can use additional, non-standard information).

Clarifications

When possible, terms (such as name and version) are qualified by references to existing specs.