Alternatives

In the absence of clear upstream guidance and recommendations, commercial redistributors will still make their own design decisions in the interests of their customers. The main approaches available are:

• Continuing to rebase on new Python 2.7.x releases, while providing no additional assistance beyond the mechanisms defined in PEP 476 in migrating from unchecked to checked hostnames in standard library HTTPS clients
• Gating availability of the changes in default handling of HTTPS connections on upgrading from Python 2 to Python 3
• For Linux distribution vendors, gating availability of the changes in default handling of HTTPS connections on upgrading to a new operating system version
• Implementing one or both of the backport suggestions described in this PEP, regardless of the formal status of the PEP

Feature detection

The marker attribute on the ssl module related to this feature is the ssl._https_verify_certificates function itself.

Specification

The ssl._https_verify_certificates function will work as follows:

def _https_verify_certificates(enable=True):
    """Verify server HTTPS certificates by default?"""
    global _create_default_https_context
    if enable:
        _create_default_https_context = create_default_context
    else:
        _create_default_https_context = _create_unverified_context

If called without arguments, or with enable set to a true value, then standard library client modules will subsequently verify HTTPS certificates by default, otherwise they will skip verification.

If called with enable set to a false value, then standard library client modules will subsequently skip verifying HTTPS certificates by default.

Security Considerations

The inclusion of this feature will allow security sensitive applications to include the following forward-compatible snippet in their code:

if hasattr(ssl, "_https_verify_certificates"):
    ssl._https_verify_certificates()

Some developers may also choose to opt out of certificate checking using ssl._https_verify_certificates(enable=False). This doesn’t introduce any major new security concerns, as monkeypatching the affected internal APIs was already possible.

Feature detection

The marker attribute on the ssl module related to this feature is:

• the ssl._https_verify_envvar attribute, giving the name of environment variable affecting the default behaviour

This not only makes it straightforward to detect the presence (or absence) of the capability, it also makes it possible to programmatically determine the relevant environment variable name.

Specification

Rather than always defaulting to the use of ssl.create_default_context, the ssl module will be modified to:

• read the PYTHONHTTPSVERIFY environment variable when the module is first imported into a Python process
• set the ssl._create_default_https_context function to be an alias for ssl._create_unverified_context if this environment variable is present and set to '0'
• otherwise, set the ssl._create_default_https_context function to be an alias for ssl.create_default_context as usual

Example implementation

_https_verify_envvar = 'PYTHONHTTPSVERIFY'

def _get_https_context_factory():
    if not sys.flags.ignore_environment:
        config_setting = os.environ.get(_https_verify_envvar)
        if config_setting == '0':
            return _create_unverified_context
    return create_default_context

_create_default_https_context = _get_https_context_factory()

Security Considerations

Relative to the behaviour in Python 3.4.3+ and Python 2.7.9->2.7.11, this approach does introduce a new downgrade attack against the default security settings that potentially allows a sufficiently determined attacker to revert Python to the default behaviour used in CPython 2.7.8 and earlier releases.

This slight increase in the available attack surface is a key reason why:

• security sensitive applications should still define their own SSL context
• the migration features described in this PEP are not being added to Python 3

However, it’s also worth keeping in mind that carrying out such an attack requires the ability to modify the execution environment of a Python process prior to the import of the ssl module. In combination with the ability to write to any part of the filesystem (such as /tmp), any attacker with such access would already be able to modify the behaviour of the underlying OpenSSL implementation, the dynamic library loader, and other potentially security sensitive components.

Interaction with Python virtual environments

The default setting is read directly from the process environment, and hence works the same way regardless of whether or not the interpreter is being run inside an activated Python virtual environment.

Feature detection

There’s no specific attribute indicating that this situation applies. Rather, it is indicated by the ssl._https_verify_certificates and ssl._https_verify_envvar attributes being present in a Python version that is nominally older than Python 2.7.12.

Specification

Implementing this backport involves backporting the changes in PEP 466, 476 and this PEP, with the following change to the handling of the PYTHONHTTPSVERIFY environment variable in the ssl module:

• read the PYTHONHTTPSVERIFY environment variable when the module is first imported into a Python process
• set the ssl._create_default_https_context function to be an alias for ssl.create_default_context if this environment variable is present and set to any value other than '0'
• otherwise, set the ssl._create_default_https_context function to be an alias for ssl._create_unverified_context

Example implementation

_https_verify_envvar = 'PYTHONHTTPSVERIFY'

def _get_https_context_factory():
    if not sys.flags.ignore_environment:
        config_setting = os.environ.get(_https_verify_envvar)
        if config_setting != '0':
            return create_default_context
    return _create_unverified_context

_create_default_https_context = _get_https_context_factory()

def _disable_https_default_verification():
    """Skip verification of HTTPS certificates by default"""
    global _create_default_https_context
    _create_default_https_context = _create_unverified_context

Security Considerations

This change would be a strict security upgrade for any Python version that currently defaults to skipping certificate validation in standard library HTTPS clients. The technical trade-offs to be taken into account relate largely to the magnitude of the PEP 466 backport also required rather than to anything security related.

Interaction with Python virtual environments

The default setting is read directly from the process environment, and hence works the same way regardless of whether or not the interpreter is being run inside an activated Python virtual environment.

Feature detection

The marker attribute on the ssl module related to this feature is:

_cert_verification_config = '<path to configuration file>'

This not only makes it straightforward to detect the presence (or absence) of the capability, it also makes it possible to programmatically determine the relevant configuration file name.

Recommended modifications to the Python standard library

The recommended approach to backporting the PEP 476 modifications to an earlier point release is to implement the following changes relative to the default PEP 476 behaviour implemented in Python 2.7.9+:

• modify the ssl module to read a system wide configuration file when the module is first imported into a Python process
• define a platform default behaviour (either verifying or not verifying HTTPS certificates) to be used if this configuration file is not present
• support selection between the following three modes of operation:
  • ensure HTTPS certificate verification is enabled
  • ensure HTTPS certificate verification is disabled
  • delegate the decision to the redistributor providing this Python version
• set the ssl._create_default_https_context function to be an alias for either ssl.create_default_context or ssl._create_unverified_context based on the given configuration setting.

Recommended file location

As the PEP authors are not aware of any vendors providing long-term support releases targeting Windows, Mac OS X or *BSD systems, this approach is currently only specifically defined for Linux system Python installations.

The recommended configuration file name on Linux systems is /etc/python/cert-verification.cfg.

The .cfg filename extension is recommended for consistency with the pyvenv.cfg used by the venv module in Python 3’s standard library.

Recommended file format

The configuration file should use a ConfigParser ini-style format with a single section named [https] containing one required setting verify.

The suggested section name is taken from the “https” URL schema passed to affected client APIs.

Permitted values for verify are:

• enable: ensure HTTPS certificate verification is enabled by default
• disable: ensure HTTPS certificate verification is disabled by default
• platform_default: delegate the decision to the redistributor providing this particular Python version

If the [https] section or the verify setting are missing, or if the verify setting is set to an unknown value, it should be treated as if the configuration file is not present.

Example implementation

_cert_verification_config = '/etc/python/cert-verification.cfg'

def _get_https_context_factory():
    # Check for a system-wide override of the default behaviour
    context_factories = {
        'enable': create_default_context,
        'disable': _create_unverified_context,
        'platform_default': _create_unverified_context, # For now :)
    }
    import ConfigParser
    config = ConfigParser.RawConfigParser()
    config.read(_cert_verification_config)
    try:
        verify_mode = config.get('https', 'verify')
    except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
        verify_mode = 'platform_default'
    default_factory = context_factories.get('platform_default')
    return context_factories.get(verify_mode, default_factory)

_create_default_https_context = _get_https_context_factory()

Security Considerations

The specific recommendations for this backporting case are designed to work for privileged, security sensitive processes, even those being run in the following locked down configuration:

• run from a locked down administrator controlled directory rather than a normal user directory (preventing sys.path[0] based privilege escalation attacks)
• run using the -E switch (preventing PYTHON* environment variable based privilege escalation attacks)
• run using the -s switch (preventing user site directory based privilege escalation attacks)
• run using the -S switch (preventing sitecustomize based privilege escalation attacks)

The intent is that the only reason HTTPS verification should be getting turned off installation wide when using this approach is because:

• an end user is running a redistributor provided version of CPython rather than running upstream CPython directly
• that redistributor has decided to provide a smoother migration path to verifying HTTPS certificates by default than that being provided by the upstream project
• either the redistributor or the local infrastructure administrator has determined that it is appropriate to retain the default pre-2.7.9 behaviour (at least for the time being)

Using an administrator controlled configuration file rather than an environment variable has the essential feature of providing a smoother migration path, even for applications being run with the -E switch.

Interaction with Python virtual environments

This setting is scoped by the interpreter installation and affects all Python processes using that interpreter, regardless of whether or not the interpreter is being run inside an activated Python virtual environment.

Origins of this recommendation

This recommendation is based on the backporting approach adopted for Red Hat Enterprise Linux 7.2, as published in the original July 2015 draft of this PEP and described in detail in this KnowledgeBase article. Red Hat’s patches implementing this backport for Python 2.7.5 can be found in the CentOS git repository.

Example implementation

_https_verify_envvar = 'PYTHONHTTPSVERIFY'
_cert_verification_config = '/etc/python/cert-verification.cfg'

def _get_https_context_factory():
    # Check for an environmental override of the default behaviour
    if not sys.flags.ignore_environment:
        config_setting = os.environ.get(_https_verify_envvar)
        if config_setting is not None:
            if config_setting == '0':
                return _create_unverified_context
            return create_default_context

    # Check for a system-wide override of the default behaviour
    context_factories = {
        'enable': create_default_context,
        'disable': _create_unverified_context,
        'platform_default': _create_unverified_context, # For now :)
    }
    import ConfigParser
    config = ConfigParser.RawConfigParser()
    config.read(_cert_verification_config)
    try:
        verify_mode = config.get('https', 'verify')
    except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
        verify_mode = 'platform_default'
    default_factory = context_factories.get('platform_default')
    return context_factories.get(verify_mode, default_factory)

_create_default_https_context = _get_https_context_factory()