PEP 486 – Make the Python Launcher aware of virtual environments

PEP: 486
Title: Make the Python Launcher aware of virtual environments
Author: Paul Moore <p.f.moore at gmail.com>
Status: Final
Type: Standards Track
Created: 12-Feb-2015
Python-Version: 3.5
Post-History: 12-Feb-2015
Resolution: https://mail.python.org/pipermail/python-dev/2015-February/138579.html

Abstract
Rationale
Implementation
Impact on Script Launching
Exclusions
Reference Implementation
References
Copyright

Abstract

The Windows installers for Python include a launcher that locates the correct Python interpreter to run (see PEP 397). However, the launcher is not aware of virtual environments (virtualenv 1 or PEP 405 based), and so cannot be used to run commands from the active virtualenv.

This PEP proposes making the launcher “virtualenv aware”. This means that when run without specifying an explicit Python interpreter to use, the launcher will use the currently active virtualenv, if any, before falling back to the configured default Python.

Rationale

Windows users with multiple copies of Python installed need a means of selecting which one to use. The Python launcher provides this facility by means of a py command that can be used to run either a configured “default” Python or a specific interpreter, by means of command line arguments. So typical usage would be:

# Run the Python interactive interpreter
py

# Execute an installed module
py -m pip install pytest
py -m pytest

When using virtual environments, the py launcher is unaware that a virtualenv is active, and will continue to use the system Python. So different command invocations are needed to run the same commands in a virtualenv:

# Run the Python interactive interpreter
python

# Execute an installed module (these could use python -m,
# which is longer to type but is a little more similar to the
# launcher approach)
pip install pytest
py.test

Having to use different commands is error-prone, and in many cases the error is difficult to spot immediately. The PEP proposes making the py command usable with virtual environments, so that the first form of command can be used in all cases.

Implementation

Both virtualenv and the core venv module set an environment variable VIRTUAL_ENV when activating a virtualenv. This PEP proposes that the launcher checks for the VIRTUAL_ENV environment variable whenever it would run the “default” Python interpreter for the system (i.e., when no specific version flags such as py -2.7 are used) and if present, run the Python interpreter for the virtualenv rather than the default system Python.

The “default” Python interpreter referred to above is (as per PEP 397) either the latest version of Python installed on the system, or a version configured via the py.ini configuration file. When the user specifies an explicit Python version on the command line, this will always be used (as at present).

Impact on Script Launching

As well as interactive use, the launcher is used as the Windows file association for Python scripts. In that case, a “shebang” (#!) line at the start of the script is used to identify the interpreter to run. A fully-qualified path can be used, or a version-specific Python (python3 or python2, or even python3.5), or the generic python, which means to use the default interpreter.

The launcher also looks for the specific shebang line #!/usr/bin/env python. On Unix, the env program searches for a command on $PATH and runs the command so located. Similarly, with this shebang line, the launcher will look for a copy of python.exe on the user’s current %PATH% and will run that copy.

As activating a virtualenv means that it is added to PATH, no special handling is needed to run scripts with the active virtualenv - they just need to use the #!/usr/bin/env python shebang line, exactly as on Unix. (If there is no activated virtualenv, and no python.exe on PATH, the launcher will look for a default Python exactly as if the shebang line had said #!python).

Exclusions

The PEP makes no attempt to promote the use of the launcher for running Python on Windows. Most existing documentation assumes the user of python as the command to run Python, and (for example) pip to run an installed Python command. This documentation is not expected to change, and users who choose to manage their PATH environment variable can continue to use this form. The focus of this PEP is purely on allowing users who prefer to use the launcher when dealing with their system Python installations, to be able to continue to do so when using virtual environments.