Python Enhancement Proposals

PEP 605 – A rolling feature release stream for CPython

PEP
605
Title
A rolling feature release stream for CPython
Author
Steve Dower <steve.dower at python.org>, Nick Coghlan <ncoghlan at gmail.com>
Discussions-To
https://discuss.python.org/t/pep-605-a-rolling-feature-release-stream-for-cpython/2418
Status
Rejected
Type
Informational
Created
20-Sep-2019
Python-Version
3.9
Post-History
1-Oct-2019, 6-Oct-2019, 20-Oct-2019

Contents

Rejection Notice

This PEP was rejected in favour of PEP 602. The potential alpha/beta alternation was deemed too confusing and the two year cadence between releases deemed too long.

Abstract

For a long time, CPython’s nominal feature release cadence has been “every 18-24 months”, and in recent years, has been pretty consistently on the “18 month” end of that window. PEP 607 provides some common background on the problems that arise due to that choice of cadence, as well as some of the risks that need to be accounted for when proposing to change it.

The proposal in this PEP aims to allow CPython’s user base to self-select into two distinct but overlapping groups:

  • users of stable feature releases (and their associated maintenance release streams) that are published every 24 months; and
  • early adopters of a new rolling release stream that takes the place of the previous CPython pre-release process

As part of this proposal, the usage guidance given for beta releases would become “suitable for production use only in environments with sufficiently robust compatibility testing and operational monitoring capabilities”, rather than the currently unqualified “not for production use”.

Similarly, the guidance given for alpha releases would be amended to state “intended for library compatibility testing and the creation of ABI compatible binary artifacts”, rather than simply saying “not for production use”.

The PEP authors believe these outcomes can be achieved by amending CPython’s pre-release management process as described in the Proposal section below.

This PEP also proposes that the frequency of X.Y.0 releases be adjusted to begin each new release series in August every two years (starting in 2021, around two years after the release of Python 3.8.0).

Example Future Release Schedules

Under this proposal, Python 3.9.0a1 would be released in December 2019, two months after the Python 3.8.0 baseline feature release in October 2019.

Assuming no further breaking changes were made to the full CPython ABI, the 3.9.0b2 release would then follow 2 months later in February 2020, continuing through to 3.9.0b9 in April 2021.

Any time a breaking change to the full CPython ABI was introduced, the first pre-release that included it would be marked as an alpha release.

3.9.0rc1 would be published in June 2021, 3.9.0rc2 in July 2021, and then the full release published as 3.9.0 in August 2021.

The cycle would start over again in October 2021, with the publication of 3.10.0a1 (4 months after the creation of the 3.9.x maintenance branch).

The exact schedule of maintenance releases would be up to the release team, but assuming maintenance releases of 3.9.x were also to occur every other month (offset from the 3.10.0 beta releases), the overall release timeline would look like:

  • 2019-12: 3.9.0a1
  • 2020-02: 3.9.0b2
  • … beta (or alpha) releases every other month
  • 2021-04: 3.9.0b9
  • 2021-06: 3.9.0rc1 (feature freeze, ABI freeze, pyc format freeze)
  • 2021-07: 3.9.0rc2
  • 2021-08: 3.9.0
  • 2021-09: 3.9.1, 3.8.x (final 3.8.x binary maintenance release)
  • 2021-10: 3.10.0a1
  • 2021-11: 3.9.2
  • 2021-12: 3.10.0b2
  • … beta (or alpha) and maintenance releases continue in alternate months
  • 2023-04: 3.10.0b10
  • 2023-05: 3.9.11
  • 2023-06: 3.10.0rc1 (feature freeze, ABI freeze, pyc format freeze)
  • 2023-07: 3.10.0rc2, 3.9.12
  • 2023-08: 3.10.0
  • 2023-09: 3.10.1, 3.9.13 (final 3.9.x binary maintenance release)
  • 2023-10: 3.11.0a1
  • 2023-12: 3.11.0b2
  • … etc

If we assume two additional pre-releases were made that introduced breaking changes to the full CPython ABI in the 3.9.0a5 and 3.9.0a7 releases, then the overall calendar would look like:

../_images/pep-0605-example-release-calendar.png

Figure 1. Impact of the pre-release process changes on the calendar.

There are always two or three active maintenance branches in this model, which preserves the status quo in that respect. The major difference is that we would start encouraging publishers to provide pre-built binaries for the pre-freeze rolling releases in addition to providing them for the stable maintenance branches.

../_images/pep-0605-overlapping-support-matrix.png

Figure 2. Testing matrix in the 18-month cadence vs. the 24-month

Package publishers targeting the full CPython ABI that choose to provide pre-built binaries for the rolling pre-freeze releases would at least need to build new wheel archives following the 3.9.0a1 release. Whether they needed to publish updated binaries after subsequent alpha releases (e.g. 3.9.0a5 or 3.9.0a7 releases in the example timeline) would depend on whether or not they were actually affected by the ABI breaks in those later releases.

As with the status quo, all package publishers wishing to provide pre-built binaries for the final release will need to build new wheel archives following the ABI freeze date. Unlike the status quo, this date will be clearly marked by the publication of the first release candidate, and it will occur early enough to give publishers a couple of months to get ready for the final release.

Example Future Release Announcements

If this PEP is accepted, the primary channels used to communicate the updated pre-release management process to end users would be the Python 3.9 What’s New document, and the release announcements themselves.

This section provides initial drafts of text that could be used for those purposes.

Suggested “What’s New in Python 3.9” Entry

The following subsection would be added to the Python 3.9 What’s New document, and then linked from each of the Python 3.9 alpha and beta announcements.

PEP 605: Changes to the pre-release management process

As detailed in PEP 605, the pre-release management process has been updated to produce a rolling series of beta releases that are considered suitable for production use in environments with sufficiently robust integration testing and operational monitoring capabilities.

Under this new rolling model, the alpha and beta releases are intermingled as part of a combined “pre-freeze” period, with alpha releases indicating breaks in the full CPython ABI that may require recompilation of extension modules or embedding applications, and beta releases indicating full binary compatibility with the immediately preceding pre-release.

Unlike previous releases, publishing pre-built binaries for 3.9.0 alpha and beta releases is actively encouraged, as a new pre-release ABI flag (“p”) is now set when building and loading extension modules prior to the full CPython ABI freeze, ensuring that all such pre-freeze extension module builds will be ignored by post-freeze interpreter builds.

The full CPython ABI will be frozen, and the pre-release flag dropped from the ABI flags, in 3.9.0rc1, which is expected to occur 2 months prior to the final 3.9.0 release (refer to the release schedule in PEP 596 for exact target dates).

For application developers, migrating to the rolling release stream provides the opportunity to be actively involved in the design and development of enhancements to the standard library and reference interpreter prior to the next stable release. It also provides the opportunity to benefit from interpreter performance enhancements up to a year or more before they become available in a stable release.

For library developers that publish pre-built wheel archives, opting in to supporting the 3.9.x rolling release stream in addition to the 3.8 stable release series requires no specific action if the project is already publishing either pure Python wheels (tagged as py3-none-any), or builds against the stable C ABI (tagged as cp38-abi3-<platform>, or the equivalent from an earlier CPython 3.x release). These same wheel archives will also be usable on the subsequent 3.9 stable release series.

For library developers that publish pre-built wheel archives that are built against the full CPython ABI, the binaries for the 3.9 stable release series will need to be built after the full CPython ABI freeze (i.e. using 3.9.0rc1 or later).

Developers of these libraries may also opt in to supporting the rolling release stream by building against the 3.9.0a1 release (or a subsequent beta release) and publishing the result as normal.

In the ideal case, binaries built this way will continue working all the way through to the last pre-freeze release. However, if the project is affected by a change in the full CPython C ABI during the pre-freeze period, then it will be necessary to publish a maintenance update that rebuilds the affected binaries against the alpha release that changed the relevant interface. In these cases, a corresponding Python-Requires entry should be added to the project metadata. For example, if a project is affected by an ABI change introduced in 3.9.0a5, then the Python-Requires entry to add would be:

Python-Requires: >= "3.9.0b6"; python_version == "3.9" and full_python_version != "3.9.0a5"

(This additional metadata ensures that the updated version won’t be installed on earlier pre-releases in the 3.9 series that offer an older variant of the ABI)

As for application developers, library developers that choose to support the rolling release stream will have the opportunity to provide feedback on new and updated API designs before they’re locked down for multiple years in a stable release (or before they’re included as a provisional API in a stable release series).

Example announcement text for the 3.9.0a1 release

This is the first preview release of Python 3.9. As an alpha release, it is intended for library and application compatibility testing and the creation of ABI compatible binary artifacts. It is not recommended for use in production environments.

Changes to the pre-release management process

CPython has switched to a new pre-release management process that is designed to produce a rolling series of beta releases that are considered suitable for production use in environments with sufficiently robust integration testing and operational monitoring capabilities. Refer to the Python 3.9 What’s New document (hyperlinked to relevant section) for details.

Major new features of the 3.9 series, compared to 3.8

Many new features for Python 3.9 are still being planned and written. Among the major new features and changes already implemented:

  • (Hey, fellow core developer or user of the rolling release stream, if a feature you find important is missing from this list, let <the release manager> know.)

The next pre-release of Python 3.9 is expected to be 3.8.0b2, currently scheduled for 2020-02-02.

Example announcement text for the 3.9.0b2 release

This is the second preview release of Python 3.9. As a beta release, it is fully binary compatible with the preceding 3.9.0a1 release. It is recommended for production use only in environments with sufficiently robust integration testing and operational monitoring capabilities.

(Remainder as per 3.9.0a1 announcement, with updates for implemented changes and the next expected release being 3.9.0b3)

Example announcement text for 3.9.0a5 (a mid-stream alpha release)

This is the fifth preview release of Python 3.9. As an alpha release, it is NOT fully binary compatible with the preceding 3.9.0b4 release. This release is intended for library and application compatibility testing and the creation of ABI compatible binary artifacts. It is not recommended for use in production environments.

Breaking changes in the full CPython ABI between 3.9.0b4 and 3.9.0a5

  • new field ob_example added to the PyObject struct
  • provisional field tp_example removed from the PyTypeObject struct

Projects that are supporting the rolling release stream and require a rebuild to restore binary compatibility should add the following metadata to their updated release:

Python-Requires: >= "3.9.0b6"; python_version == "3.9" and full_python_version != "3.9.0a5"

(Remainder as per 3.9.0a1 announcement, with updates for implemented changes and the next expected release being 3.9.0b6)

Example announcement text for 3.9.0rc1

This is the first release candidate for Python 3.9. As a release candidate, this release is now feature complete, the full CPython ABI is now frozen, and the pre-release marker has been removed from the ABI compatibility flags. It is recommended for production use only in environments with sufficiently robust integration testing and operational monitoring capabilities.

Preparation for the final 3.9.0 release

With the full CPython ABI now frozen, library developers targeting that ABI are encouraged to build and publish binaries for the stable 3.9.x series.

Application developers that have not been testing against the rolling release stream are encouraged to test their applications against the release candidate and report any compatibility regressions not already mentioned in the Porting Guide (hyperlinked to relevant What’s New section).

A second release candidate is planned for 2021-07-02, and then the final 3.9.0 release is planned for 2021-08-02.

Major new features of the 3.9 series, compared to 3.8

Some of the major new features and changes in this release:

  • (Hey, fellow core developer or user of the rolling release stream, if a feature you find important is missing from this list, let <the release manager> know.)

Motivation

The current CPython pre-release and release management processes were developed in an era where automated continuous integration and operational monitoring systems were still relatively immature. Since that time, many organisations have adopted deployment models that allow them to incorporate new CPython feature releases without adding substantially more risk than they incur for any other code change. Newer deployment models, such as lightweight task specific application containers, also make it easier to combine an application with a language runtime in a CI pipeline, and then keep them together until the entire container image is later replaced by an updated one.

In light of those changes in the wider environment, PEP 602 has proposed reducing the feature delivery latency for the Python standard library and CPython reference interpreter by increasing the frequency of CPython feature releases from every 18-24 months to instead occur every 12 months.

Unfortunately, for many organisations, the cost of adopting a new Python release doesn’t automatically scale down with a reduced number of changes in the release, as the primary costs aren’t associated with resolving any discovered issues; the primary costs are associated with the search for issues. This search may involve manual testing of software systems, human review of written materials, and other activities where the time required scales with the size of the existing system, rather than with the number of changes between the versions of Python.

For third party library developers, costs are primarily associated with the number of distinct Python versions in widespread usage. This currently tends to be influenced by a combination of which releases are still actively maintained by python-dev, and which releases are the latest versions offered by particular redistributors (with the Debian, Ubuntu LTS, and RHEL/CentOS system Python versions being particularly popular development targets). In addition to the basic CI cost of testing against more Python versions, having more variants in widespread use can make it more difficult to determine when a fault report is an actual error in the project, or an issue in the reporting user’s environment.

PEP 602 proposes that affected organisations and projects simply switch to adopting every second or third CPython release, rather than attempting to adopt every release, but that creates its own set of new problems to be resolved, both practical (e.g. deprecations would need to cover more than one release if we’re expecting users to routinely skip releases) and cultural (e.g. with a larger number of versions in active use, there is a much higher chance that open source library maintainers will receive bug reports that only occur on Python versions that they’re not using themselves).

PEP 598 was an initial attempt by one of the authors of this PEP to propose an alternative scheme to reduce feature delivery latency by adopting a semantic versioning style policy that allowed for the incremental delivery of backwards compatible features within a release series, until that series reached feature complete status. That variant still had the undesirable consequence of imposing visible changes on end users that are happy enough with the current release management model.

This PEP takes the view that both PEP 598 and PEP 602 share a common flaw: they are attempting to satisfy the needs of two quite distinct audiences within the constraints of a single release model, which results in conflicting design requirements, and the need for awkward trade-offs between those conflicting requirements. The proposal in this PEP aims to avoid that flaw by proposing the creation of two distinct production-ready release streams, with the existing release stream being largely left alone, while the new release stream is tailored towards the audience that would most benefit from a reduction in feature delivery latency.

Aims of this Proposal

The proposal in this PEP arises from making the following key assumptions:

  • the vast majority of Python’s users aren’t actively clamouring for new language and runtime level features, and instead only upgrade when either the version they were previously using is no longer supported, when their Python provider switches to offering them a newer version by default, or when enough changes that are of interest to them have accumulated to make a compelling case for upgrading
  • for many users of new releases, much of the work that arises when adopting a new release doesn’t arise from compatibility issues at the language level, but from compatibility issues at the component installation level (i.e. filename and installation path changes)
  • that there’s a subset of Python’s user base that would be willing to run production-ready pre-releases (akin to Windows Insider or Debian testing builds) for at least some of their use cases

The core of the proposal in this PEP is changing the CPython pre-release process to produce a rolling stream of incremental feature releases at a regular cadence, and to ensure that most of those builds offer a sufficient level of stability as to be suitable for use in appropriately managed production systems.

By adopting this approach, the proposal aims to provide an improved outcome for almost all Python users and contributors:

  • for users of the new incremental feature release stream, targeting the pre-release phase allows for even lower feature delivery latency than the annual cadence proposed in PEP 602;
  • for core developers working on new features, increased frequency and adoption of pre-releases should improve pre-release feedback cycles;
  • for users of the established release stream, the increased adoption and improved feedback cycles during the pre-release period should result in increased feature maturity at the time of its first X.Y.0 release, as well as higher levels of ecosystem readiness;
  • for Python library maintainers, the rolling stream of pre-releases will hopefully provide more opportunities to identify and resolve design issues before they make it into a full stable release than is offered by the current pre-release management process; and
  • for developers of alternative Python implementations, the rolling stream of pre-releases may provide an additional incentive for extension module authors to migrate from the full CPython ABI to the Python stable ABI, which would also serve to make more of the ecosystem compatible with implementations that don’t emulate the full CPython C API.

That said, it is acknowledged that not all the outcomes of this proposal will be beneficial for all members of the wider Python ecosystem:

  • for Python library maintainers, both this PEP and PEP 602 would likely result in user pressure to support the faster release cadence. While this PEP attempts to mitigate that by clearly marking which pre-releases include potentially breaking changes to the full CPython C ABI, and PEP 602 attempts to mitigate it by keeping the minimum time between full releases at 12 months, it isn’t possible to eliminate this downside completely;
  • for third party extension module maintainers, both this PEP and PEP 602 would likely result in user pressure to start supporting the stable ABI in order to provide wheel archives that work on the new version as soon as it is available. Whether that’s a net negative or not will depend on how the request is presented to them (it could be a positive if the request comes in the form of a courteous contribution to their project from a developer interested in supporting the rolling pre-freeze releases);
  • for some users of the established release stream that rely on the availability of pre-built wheel archives, switching to adopting a new release every 12 months may be an acceptable rate increase, while moving consistently to the 24 month end of the historical 18-24 month cadence would be an undesirable rate reduction relative to the 18-month cycle used for recent releases. Whether this proposal would be a net negative for these users will depend on whether or not we’re able to persuade library maintainers that it’s worth their while to support the upcoming stable release throughout its pre-freeze period, rather than waiting until its API and ABI have been frozen.

Proposal

The majority of the proposed changes in this PEP only affect the handling of pre-release versions. The one change affecting full release versions is a suggested change to their cadence.

Two year cadence for stable releases

With the rolling pre-freeze releases available to users that are looking to use leading edge versions of the reference interpreter and standard library, this PEP proposes that the frequency of X.Y.0 releases be adjusted to publish a new stable release in August every two years (starting in 2021, around two years after the release of Python 3.8.0).

This change is arguably orthogonal to the proposed changes to the handling of the pre-freeze release period, but the connection is that without those pre-release management changes, the downsides of a two-year full release cadence would probably outweigh the upsides, whereas the opposite is true for a 12-month release cadence (i.e. with the pre-release management changes proposed in this PEP in place, the downsides of a 12-month full release cadence would outweigh the upsides).

Merging of the alpha and beta phases into a “pre-freeze” phase

Rather than continuing the status quo where the pre-release alpha and beta phases are distinct and sequential, this PEP proposes that they instead be combined into a single “pre-freeze” phase with a monotonically increasing serial number on the releases.

Rather than denoting distinct phases, the “alpha” and “beta” names would instead indicate whether or not the release contains breaking changes to the full CPython C ABI:

  • “alpha” releases would be “ABI breaking” releases where extension modules built against the full CPython ABI in the preceding pre-release are not necessarily going to load correctly
  • “beta” releases would be “binary compatible” releases, where extension modules built against the full CPython ABI in the preceding pre-release are expected to load correctly, as long as those modules abide by the following additional criteria:
    • the module must not be using any provisional or private C APIs (either from the previous stable release series, or the in development pre-release series) that were removed in this beta release, or were changed in an ABI incompatible way
    • the module must not be using any C APIs that were deprecated in the previous stable release series, and removed in this beta release

Pre-freeze phase duration and cadence

Rather than being released monthly for a period of a few months while preparing a new X.Y.0 release, pre-freeze releases would instead be consistently published every two months.

The only time this would not be the case is during the two month release candidate period for an upcoming X.Y.0 release (see the release candidate section below for more details). This means two otherwise scheduled releases would be skipped (one corresponding with the first release candidate date, one with the final release date).

The pre-freeze phase would typically be expected to start 2 months after the preceding stable X.Y.0 release.

The first pre-freeze release for any new release series will always be X.Y.0a1 (as there is no preceding release with the same ABI version markers to judge binary compatibility against).

Pre-freeze releases would gain an additional flag in their C ABI compatibility markers to avoid binary compatibility issues with the eventual stable release.

Release policy for beta releases

This PEP proposes that the policy for beta releases be set as follows:

  • as with current beta releases, the stable BuildBot fleet is expected to be green prior to preparation and publication of the beta release
  • as with current beta releases, the release manager is expected to review open release blocker issues prior to preparation and publication of the beta release
  • as with current beta releases, any additions to the abi3 stable C ABI would be expected to become a permanent part of that ABI unless and until that stable ABI version is retired completely (Note: there are no current plans to increment the stable ABI version)
  • unlike current beta releases, beta releases under this PEP would not be considered feature complete for the next X.Y.0 release
  • unlike current beta releases, all APIs added since the last CPython feature release (other than additions to the stable C ABI) would be considered provisional
  • unlike current beta releases, beta releases under this PEP would be prepared and published from the master development branch
  • unlike current alpha or beta releases, beta releases under this PEP would be required to be fully ABI compatible with the immediately preceding pre-release in the series (excluding any changes to provisional APIs, or the removal of APIs that were deprecated in the previous release series)

Release policy for alpha releases

This PEP proposes that the policy for alpha releases be set as follows:

  • as with current alpha releases, the stable BuildBot fleet is expected to be green prior to preparation and publication of the alpha release
  • as with current alpha releases, the release manager is expected to review open release blocker issues prior to preparation and publication of the beta release
  • unlike current alpha release, the release manager would be expected to target a similar level of stability to the current beta releases, even for the alpha releases

Under this PEP, an alpha release would be published whenever it isn’t possible to publish a release that satisfies the criteria for a beta release, and allowing some additional time before making the release won’t resolve the issue.

It is expected that the full CPython API changing in a way that breaks ABI compatibility (for example, a field may have been added to or removed from a public struct definition) will be the most likely reason for publishing additional alpha releases beyond the initial compatibility tag defining X.Y.0a1 release, but the decision for any particular release rests with the release manager.

Release candidate policy, phase duration, and cadence

Given the proposed changes to the alpha and beta release phases, the release candidate phase would see the following related adjustments:

  • Feature freeze, ABI freeze, pyc file format freeze, and maintenance branch creation would all correspond with the creation of X.Y.0rc1 (currently these occur across a mixture of X.Y.0b1, the last beta release, and X.Y.0rc1)
  • The X.Y.0 release candidate period would be extended from 3 weeks to 2 months
  • There would normally be two release candidates issued a month apart, but additional candidates may be published at the release manager’s discretion
  • The final X.Y.0 release would occur between 1 and 4 weeks after the final release candidate (depending if additional release candidates were needed after the second)
  • If the final X.Y.0 release is delayed beyond the August target date, the subsequent release series is not affected, and will still be scheduled for August (now slightly less than two years later).

In addition to allowing more time for end user feedback on the release candidate, this adjusted policy also provides additional time for maintainers of Python projects to build and publish pre-built wheel archives for the new stable release series, significantly improving the initial user experience of the X.Y.0 release.

Changes to management of the CPython stable C ABI

The CPython stable ABI [5] makes the commitment that binary extension modules built against any particular CPython release will continue to work on future CPython releases that support the same stable ABI version (this version is currently abi3).

Under the proposed rolling pre-freeze release model, this commitment would be extended to also apply to the beta releases: once an intentional addition to the abi3 stable ABI for the upcoming Python version has been shipped in a beta release, then it will not be removed from future releases for as long as the abi3 stable ABI remains supported.

Two main mechanisms will be available for obtaining community feedback on additions to the stable ABI:

  • the preferred mechanism will be to add new APIs to the full CPython API first, and only promote them to the stable ABI after they have been included in at least one published beta release and received relevant user feedback
  • for APIs where that approach is unavailable for some reason (e.g. some API additions may serve no useful purpose when the full CPython API is available), then developers may request that the release manager mark the next release as an alpha release (even in the absence of an ABI break in the full CPython API), and attempt to obtain further feedback that way

As a slight readability and usability improvement, this PEP also proposes the introduction of aliases for each major stable ABI version:

#define Py_LIMITED_API_3_3 0x03030000
#define Py_LIMITED_API_3_4 0x03040000
#define Py_LIMITED_API_3_5 0x03050000
#define Py_LIMITED_API_3_6 0x03060000
#define Py_LIMITED_API_3_7 0x03070000
#define Py_LIMITED_API_3_8 0x03080000
#define Py_LIMITED_API_3_9 0x03090000
// etc...

These would be used both in extension module code to set the target ABI version:

#define Py_LIMITED_API Py_LIMITED_API_3_8

And also in the CPython interpreter implementation to check which symbols should be made available:

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= Py_LIMITED_API_3_9
// A Python 3.9+ addition to the stable ABI would appear here
#endif

The documentation for the rolling pre-freeze releases and the stable C ABI would make it clear that extension modules built against the stable ABI in a later pre-freeze release may not load correctly on an earlier pre-freeze release.

The documentation for alpha releases and the stable C ABI would make it clear that even extension modules built against the stable ABI in an alpha release may not load correctly on the next release if two alpha releases are published in a row (this situation would ideally be rare).

Changes to management of the full CPython ABI

This PEP proposes two changes to the management of the full CPython ABI.

An explicit commit and NEWS file convention to mark ABI breaking changes

The proposal in this PEP requires that release managers be able to appropriately mark a pre-freeze release as either an alpha or a beta release based on whether or not it includes an ABI breaking change.

To assist in that process, core developers would be requested to include a “(CPython ABI break)” marker at the beginning of all NEWS file snippets for changes that introduce a breaking change in the full CPython C ABI.

The “CPython” marker is included to make it clear that these annotations relate to the full CPython ABI, not the stable ABI.

For commit messages, the shorter marker “(ABI break)” would be placed at the start of the summary line for the commit.

The pre-merge bots will be updated to ensure that if the ABI break marker appears in one of the two locations, it appears in both of them.

If the marker is inadvertently omitted from the initial commit message and NEWS entry, then the commit message marker should be included in the subsequent commit that adds the marker to the NEWS entry.

In addition to being useful for release managers, these markers should also be useful for developers investigating unexpected segfaults when testing against the affected release.

Explicitly marking builds against the pre-freeze ABI

The full CPython ABI has long operated under a policy where binary compatibility only applies within a release series after the ABI has been declared frozen, and only source compatibility applies between different release series.

This policy means that extension modules built against CPython pre-releases prior to the ABI freeze for that release series may not actually load correctly on the final release.

This is due to the fact that the extension module may be relying on provisional or previously deprecated interfaces that were changed or removed in a later alpha or beta release, or it may be due to public structures used by the extension module changing size due to the addition of new fields.

Historically, adoption of alpha and beta releases has been low enough that this hasn’t really been a problem in practice. However, this PEP proposes to actively encourage widespread operational use of beta releases, which makes it desirable to ensure that users of those releases won’t inadvertently publish binary extension modules that cause segfaults for users running the release candidates and final releases.

To that end, this PEP proposes amending the extension module SOABI marker on non-Windows systems to include a new “p” flag for CPython pre-releases, and only switch back to omitting that flag once the ABI for that particular X.Y.0 version has been frozen on entry to the release candidate stage.

With this change, alpha and beta releases of 3.9.0 would get an SOABI tag of cpython-39p, while all release candidates and final builds (for both 3.9.0 and later 3.9.x releases) would get an unqualified SOABI tag of cpython-39

Debug builds would still add the “d” to the end of the tag, giving cpython-39pd for debug builds of pre-releases.

On Windows systems, the suffix for tagged pyd files in pre-release builds would include “p” as a pre-release marker immediately after the version number, giving markers like “cp39p-win_amd64”.

A proposed reference implementation for this change is available at [4] (Note: at time of writing, that implementation had not yet been tested on Windows).

Updating Python-Requires for projects affected by full C ABI changes

When a project first opts in to providing pre-built binary wheels for the rolling pre-freeze release series, they don’t need to do anything special: they would add the rolling release series to their build and test matrices and publish binary archives that are flagged as being compatible with that release series, just as they would if providing pre-built binary wheels after the full CPython ABI freeze for that release series.

However, if the project is affected by a CPython ABI compatibility break in the rolling release stream, then they will need to issue a version update that includes both the new binary build, and a new environment constrained Python-Requires marker.

For example, if a project supporting the rolling release stream was affected by a CPython ABI compatibility break in the 3.9.0a5 release, then they would add the following metadata entry on the version that published the updated binary build:

Python-Requires: >= "3.9.0b6"; python_version == "3.9" and full_python_version != "3.9.0a5"

What this does is add an additional compatibility constraint as part of the published packages, so Python 3.9.0 beta versions prior to 3.9.0b6 won’t consider the updated package as a candidate for installation, and the only alpha release that will consider the package is 3.9.0a5 itself.

Caveats and Limitations

Actual release dates may be scheduled up to a month earlier or later at the discretion of the release manager, based on release team availability, and the timing of other events (e.g. PyCon US, or the annual core developer sprints). However, as one goal of the proposal is to provide a consistent release cadence, adjustments should ideally be rare.

Within a release series, the exact frequency of maintenance releases would still be up to the release manager and the binary release team; this PEP only proposes an expected cadence for pre-releases and X.Y.0 releases.

However, for the sake of the example timelines, the PEP assumes maintenance releases every other month, allowing them to alternate months with the rolling pre-freeze releases.

The release manager and Steering Council would also retain the power to amend various details of the proposal in this PEP. Possible amendments include (but are not limited to):

  • changing the timing for creation of the maintenance branch. If a major change that would require a new alpha release is landed relatively late in the pre-release process, the release manager could potentially choose to branch off from a point prior to that major change. For example, it may make sense to do this if the next scheduled release was intended to be the final beta release or the first release candidate.
  • the criteria for declaring an alpha release could potentially be expanded to include all changes that require a “Porting” entry in the What’s New document
  • rather than declaring alpha releases on an as-needed basis, the release manager could declare some dates as alpha releases in advance, and ask core developers to time their higher risk changes accordingly.

The intent of the concrete proposal in the PEP is to provide a clear illustrative example for reviewers to consider, not to constrain our ability to adjust specific details based on practical experience with the process.

Design Discussion

Why rolling pre-freeze releases over simply doing more frequent X.Y.0 releases?

For large parts of Python’s user base, availability of new CPython feature releases isn’t the limiting factor on their adoption of those new releases (this effect is visible in such metrics as PyPI download metadata).

As such, any proposal based on speeding up full feature releases needs to strike a balance between meeting the needs of users who would be adopting each release as it became available, and those that would now be in a position of adopting every 2nd, 3rd, or 4th release, rather than being able to migrate to almost every release at some point within its lifecycle.

This proposal aims to approach the problem from a different angle by defining a new production-ready release stream that is more specifically tailored to the interests of operating environments that are able to consume new releases as fast as the CPython core team is prepared to produce them.

Is it necessary to keep the “alpha” and “beta” naming scheme?

Using the “a” and “b” initials for the proposed rolling releases is a design constraint imposed by some of the pragmatic aspects of the way CPython version numbers are published.

Specifically, alpha releases, beta releases, and release candidates are reported in some places using the strings “a”, “b”, and “c” respectively, while in others they’re reported using the hex digits 0xA, 0xB, and 0xC. We want to preserve that, while also ensuring that any Python-Requires constraints are expressed against the beta releases rather than the alpha releases (since the latter may not enforce the abi3 stability requirements if two alpha releases occur in succession).

However, there isn’t anything forcing us to say that the “a” stands for “alpha” or the “b” stands for “beta”.

That means that if we wanted to increase adoption amongst folks that were only being put off by the “beta” label, then it may make sense to emphasise the “*A*BI breaking” and “*B*inary compatible” names over the “alpha” and “beta” names, giving:

  • 3.9.0a1: ABI breaking pre-freeze release
  • 3.9.0b2: binary compatible pre-freeze release
  • 3.9.0rc1: release candidate
  • 3.9.0: final release

This iteration of the PEP doesn’t go that far, as limiting initial adoption of the rolling pre-freeze releases to folks that are comfortable with the “beta” label is likely to be a good thing, as it is the early adopters of these releases that are going to encounter any unexpected consequences that occur at the level of the wider Python ecosystem, and we’re going to need them to be willing to take an active part in getting those issues resolved.

Moving away from the “beta” naming would then become an option to keep in mind for the future, assuming the resulting experience is sufficiently positive that we decide the approach is worth continuing.

Why rolling pre-freeze releases rather than alternating between stable and unstable release series?

Rather than using the beta period for rolling releases, another option would be to alternate between traditional stable releases (for 3.8.x, 3.10.x, etc), and release series that used the new rolling release cadence (for 3.9.x, 3.11.x, etc).

This idea suffers from the same core problem as PEP 598 and PEP 602: it imposes changes on end users that are happy with the status quo without offering them any clear compensating benefit.

It’s also affected by one of the main concerns raised against PEP 598: at least some core developers and end users strongly prefer that no particular semantics be assigned to the value of any of the numbers in a release version. These community members instead prefer that all the semantic significance be associated with the position within the release number that is changing.

By contrast, the rolling pre-freeze release proposal aims to address that concern by ensuring that the proposed changes in policy all revolve around whether a particular release is an alpha release, beta release, release candidate, or final release.

Why not use Calendar Versioning for the rolling release stream?

Steve Dower’s initial write-up of this proposal [1] suggested the use of calendar versioning for the rolling release stream (so the first rolling pre-release after Python 3.8.0 would have been Python 2019.12 rather than 3.9.0b1).

Paul Moore pointed out [2] two major practical problems with that proposal:

  • it isn’t going to be clear to users of the calendar-based versions where they stand in relation to the traditionally numbered versions
  • it breaks Python-Requires metadata processing in packaging tools with no clear way of fixing it reliably (since all calendar versions would appear as newer than any standard version)

This PEP aims to address both of those problems by using the established beta version numbers for the rolling releases.

As an example, consider the following question: “Does Python 2021.12 include all the new features released in Python 3.9.0?”. With calendar versioning on the rolling releases, that’s impossible to answer without consulting a release calendar to see when 3.9.0rc1 was branched off from the rolling release series.

By contrast, the equivalent question for rolling pre-freeze releases is straightforward to answer: “Does Python 3.10.0b2 include all the new features released in Python 3.9.0?”. Just from formulating the question, the answer is clearly “Yes, unless they were provisional features that got removed”.

The beta numbering approach also avoids other questions raised by the calendar versioning concept, such as how sys.version_info, PY_VERSION_HEX, site-packages directory naming, and installed Python binary and extension module naming would work.

How would users of the rolling pre-freeze releases detect API changes?

When adding new features, core developers would be strongly encouraged to support feature detection and graceful fallback to alternative approaches via mechanisms that don’t rely on either sys.version_info or runtime code object introspection.

In most cases, a simple hasattr check on the affected module will serve this purpose, but when it doesn’t, alternative approaches would be considered as part of the feature addition. Prior art in this area includes the pickle.HIGHEST_PROTOCOL attribute, the hashlib.algorithms_available set, and the various os.supports_* sets that the os module already offers for platform dependent capability detection.

It would also be possible to add features that need to be explicitly enabled via a __future__ import when first included in the rolling pre-freeze releases, even if that feature flag was subsequently enabled by default before its first appearance in an X.Y.0 release candidate.

The rationale behind these approaches is that explicit detection/enabling like this would make it straightforward for users of the rolling pre-freeze release stream to notice when we remove or change provisional features (e.g. from __future__ imports break on compile if the feature flag no longer exists), or to safely fall back on previous functionality.

The interpreter’s rich attribute lookup machinery means we can also choose to add warnings for provisional or deprecated imports and attributes that we don’t have any practical way to add for checks against the value of sys.version_info.

Why add a new pre-freeze ABI flag to force rebuilds after X.Y.0rc1?

The core development team currently actively discourage the creation of public pre-built binaries for an X.Y series prior to the ABI freeze date.

The reason we do that is to avoid the risk of painful debugging sessions on the stable X.Y.0 release that get traced back to “Oh, our dependency ‘superfast-binary-operation’ was affected by a CPython ABI break in X.Y.0a3, but the project hasn’t published a new build since then”.

With the proposed pre-freeze ABI flag in place, this aspect of the release adoption process continues on essentially unchanged from the status quo: a new CPython X.Y release series hits ABI freeze -> package maintainers publish new binary extension modules for that release series -> end users only get segfaults due to actual bugs, not just builds against an incompatible ABI.

The primary goal of the new pre-freeze ABI flag is then to improve the user experience of the rolling pre-freeze releases themselves, by allowing pre-built binary archives to be published for those releases without risking the problems that currently cause us to actively discourage the publication of binary artifacts prior to ABI freeze.

In the ideal case, package maintainers will only need to publish one pre-freeze binary build at X.Y.0a1, and then a post-freeze build after X.Y.0rc1. The only situations that should require a rebuild in the meantime are those where the project was actually affected by a CPython ABI break in an intervening alpha release.

As a concrete example, consider the scenario where we end up having three releases that include ABI breaks: X.Y.0a1, X.Y.0a5, X.Y.0a7. The X.Y.0a7 ABI is then the ABI that carries through all the subsequent beta releases and into X.Y.0rc1. (This is the scenario illustrated in figure 1)

Forcing everyone to rebuild the world every time there’s an alpha release in the rolling release stream would almost certainly lead to publishers deciding supporting the rolling releases was more trouble than it was worth, so we want to allow modules built against X.Y.0a1 to be loaded against X.Y.0a7, as they’re probably going to be compatible (there are very few projects that use every C API that CPython publishes, and most ABI breaks affect a single specific API).

Once we publish X.Y.0rc1 though, we want to ensure that any binaries that were built against X.Y.0a1 and X.Y.0a4 are completely removed from the end user experience. It would be nice to be able to keep the builds against X.Y.0a7 and any subsequent beta releases (since it turned out those actually were built against the post-freeze ABI, even if we didn’t know that at the time), but losing them isn’t any worse than the status quo.

This means that the pre-freeze flag is “the simplest thing that could possibly work” to solve this problem - it’s just a new ABI flag, and we already have the tools available to deal with ABI flags (both in the interpreter and in package publication and installation tools).

Since the ABI flags have changed relative to the pre-releases, projects don’t even need to publish a new release: they can upload new wheel archives to their existing releases, just as they can today.

A cleverer scheme that was able to retroactively accept everything built against the last alpha or subsequent beta releases would likely be possible, but it isn’t considered necessary for adoption of this PEP, as even if we initially start out with a simple pre-release ABI flag, it would still be possible to devise a more sophisticated approach in the future.

Why allow additional alpha releases after X.Y.0a1?

In an ideal world, all breaking changes to the full CPython ABI would land in X.Y.0a1 alongside the filesystem layout changes, and the ABI for the release series would remain stable after that.

However, recent history doesn’t suggest that we’d be able to actually make that commitment and stick to it, so the PEP assumes that ABI changes will be made progressively throughout the pre-freeze period, and the full lockdown will occur only with the creation of the X.Y.z maintenance branch when preparing X.Y.0rc1.

Implications for CPython core development

The major change for CPython core development is the need to keep the master branch more consistently release ready.

While the main requirement for that would be to keep the stable BuildBot fleet green, there would also be encouragement to keep the development version of the documentation up to date for the benefit of users of the rolling pre-freeze releases. This will include providing draft What’s New entries for changes as they are implemented, although the initial versions may be relatively sparse, and then expanded based on feedback from beta release users.

For core developers working on the CPython C API, there would also be a new requirement to consistently mark ABI breaking changes in their NEWS file snippets.

On the specific topic of the stable ABI, most API designs will be able to go through a process where they’re first introduced as a provisional part of the full CPython API (allowing changes between pre-freeze releases), and only promoted to the stable ABI once developers are confident that the interface is genuinely stable.

It’s only in rare cases where an API serves no useful purpose outside the stable ABI that it may make sense to publish an alpha release containing a provisional stable ABI addition rather than iterating on the design in the provisional CPython API instead.

Implications for Python library development

If this PEP is successful in its aims, then supporting the rolling pre-freeze release stream shouldn’t be subtantially more painful for library authors than supporting the stable releases.

For publishers of pure Python packages, this would be a matter of publishing “py3” tagged wheel archives, and potentially adding the rolling pre-freeze release stream to their test matrix if that option is available to them.

For publishers of binary extension modules, the preferred option would be to target the stable C ABI (if feasible), and thus enjoy an experience similar to that of pure Python packages, where a single pre-built wheel archive is able to cover multiple versions of Python, including the rolling pre-freeze release stream.

This option isn’t going to be viable for all libraries, and the desired outcome for those authors is that they be able to support the rolling releases by building and publishing one additional wheel archive, built against the initial X.Y.0a1 release. The subsequent build against X.Y.0rc1 or later is then the same build that would have been needed if only supporting the final stable release.

Additional wheel builds beyond those two should then only be needed if that particular library is directly affected by an ABI break in any other alpha release that occurs between those two points.

Having a rolling pre-freeze release stream available may also make it more feasible for more CI providers to offer a “CPython beta release” testing option. At the moment, this feature is only available from CI providers that are willing and able to put the necessary time and effort into creating, testing, and publishing their own builds from the CPython master branch (e.g. [6]).

Implications for the proposed Scientific Python ecosystem support period

Based on discussions at SciPy 2019, NEP (NumPy Enhancement Proposal) 29 has been drafted [3] to propose a common convention across the Scientific Python ecosystem for dropping support for older Python versions.

While the exact formulation of that policy is still being discussed, the draft proposal (as of October 20, 2019) recommends that projects support any Python feature release published within the last 42 months, with a minimum of supporting the latest 2 Python feature releases.

For an 18-month feature release cadence, that works out to always supporting at least the two most recent feature releases, and then dropping support for all X.Y.Z releases around 6 months after X.(Y+2).0 is released. This means there is a 6-month period roughly every other year where the three most recent feature releases are supported.

For a 12-month release cadence, it would work out to always supporting at least the three most recent feature releases, and then dropping support for all X.Y.Z releases around 6 months after X.(Y+3).0 is released. This means that for half of each year, the four most recent feature releases would be supported, with the other half of each year hopefully being used to get ready for that year’s feature release.

For a 24-month release cadence, the second clause takes priority over the first, and the recommended Python version support period increases to 48 months from the initial X.Y.0 release in order to consistently support the two most recent CPython feature releases. For projects that also support the rolling release stream, the number of supported feature releases would increase to three.

Release cycle alignment for core development sprints

With the proposal in this PEP, it is expected that the focus of core development sprints would shift slightly based on the current location in the two-year cycle.

In release years, the timing of PyCon US is suitable for new contributors to work on bug fixes and smaller features before the first release candidate goes out, while the Language Summit and core developer discussions can focus on plans for the next release series.

The pre-alpha core development sprint in release years will provide an opportunity to incorporate feedback received on the previous release, either as part of the next maintenance release (for bug fixes and feedback on provisional APIs), or as part of the first alpha release of the next release series (for feedback received on stable APIs).

Those initial alpha releases would also be the preferred target for ABI breaking changes to the full CPython ABI (while changes later in the release cycle would still be permitted as described in this PEP, landing them in the X.Y.0a1 release means that they won’t trigger any additional work for publishers of pre-built binary packages).

The Steering Council elections for the next release cycle are also likely to occur around the same time as the pre-alpha development sprints.

In non-release years, the focus for both events would just be on the upcoming maintenance and pre-freeze releases. These less intense years would hopefully provide an opportunity to tackle various process changes and infrastructure upgrades without impacting the release candidate preparation process.

Release cycle alignment for prominent Linux distributions

Some rolling release Linux distributions (e.g. Arch, Gentoo) may be in a position to consume the new rolling pre-freeze releases proposed in this PEP, but it is expected that most distributions would continue to use the established releases.

The specific dates for final releases proposed in this PEP are chosen to align with the feature freeze schedules for the annual October releases of the Ubuntu and Fedora Linux distributions.

For both Fedora and Ubuntu, it means that the release candidate phase aligns with the development period for a distro release, which is the ideal time for them to test a new version and provide feedback on potential regressions and compatibility concerns.

For Ubuntu, this also means that their April LTS releases will have benefited from a full short-term release cycle using the new system Python version, while still having that CPython release be open to upstream bug fixes for most of the time until the next Ubuntu LTS release.

The one Linux release cycle alignment that is likely to be consistently poor with the specific proposal in this PEP is with Debian, as that has been released in the first half of odd-numbered years since 2005 (roughly 12 months offset from Ubuntu LTS releases).

With the annual release proposal in PEP 602, both Debian and Ubuntu LTS would consistently get a system Python version that is around 6 months old, but would also consistently select different Python versions from each other.

With a two-year cadence, and CPython releases in the latter half of the year, they’re likely to select the same version as each other, but one of them will be choosing a CPython release that is more than 18 months behind the latest beta releases by the time the Linux distribution ships.

If that situation does occur, and is deemed undesirable (but not sufficiently undesirable for Debian to choose to adjust their release timing), then that’s where the additional complexity of the “incremental feature release” proposal in PEP 598 may prove worthwhile.

(Moving CPython releases to the same half of the year as the Debian and Ubuntu LTS releases would potentially help mitigate the problem, but also creates new problems where a slip in the CPython release schedule could directly affect the release schedule for a Linux distribution, or else result in a distribution shipping a Python version that is more than 18 months old)

Implications for simple deployment environments

For the purposes of this PEP, a “simple” deployment environment is any use case where it is straightforward to ensure that all target environments are updated to a new Python release at the same time (or at least in advance of the rollout of new higher level application versions), and any pre-release testing that occurs need only target a single Python micro version.

The simplest such case would be scripting for personal use, where the testing and target environments are the exact same environment.

Similarly simple environments would be containerised web services, where the same Python container is used in the CI pipeline as is used on deployment, and any application that bundles its own Python runtime, rather than relying on a pre-existing Python deployment on the target system.

For these use cases, there is a straightforward mechanism to minimise the impact of this PEP: continue using the stable releases, and ignore the rolling pre-freeze releases.

To actually adopt the rolling pre-freeze releases in these environments, the main challenge will be handling the potential for extension module segfaults when the next pre-freeze release is an alpha release rather than a beta release, indicating that the CPython ABI may have changed in an incompatible way.

If all extension modules in use target the stable ABI, then there’s no problem, and everything will work just as smoothly as it does on the stable releases.

Alternatively, “rebuild and recache all extension modules” could become a standard activity undertaken as part of updating to an alpha release.

Finally, it would also be reasonable to just not worry about it until something actually breaks, and then handle it like any other library compatibility issue found in a new alpha or beta release.

Aside from extension module ABI compatibility, the other main point of additional complexity when using the rolling pre-freeze releases would be “roll-back” compatibility for independently versioned features, such as pickle and SQLite, where use of new or provisional features in the beta stream may create files that are not readable by the stable release. Applications that use these kinds of features and also require the ability to reliably roll-back to a previous stable CPython release would, as today, be advised to avoid adopting pre-release versions.

Implications for complex deployment environments

For the purposes of this PEP, “complex” deployment environments are use cases which don’t meet the “simple deployment” criteria above. They may involve multiple distinct versions of Python, use of a personalised build of Python, or “gatekeepers” who are required to approve use of a new version prior to deployment.

For example, organisations that install Python on their users’ machines as part of a standard operating environment fall into this category, as do those that provide a standard build environment. Distributions such as conda-forge or WinPython that provide collections of consistently built and verified packages are impacted in similar ways.

These organisations tend to either prefer high stability (for example, all of those who are happily using the system Python in a stable Linux distribution like Debian, RHEL/CentOS, or Ubuntu LTS as their preferred Python environment) or fast turnaround (for example, those who regularly contribute toward the latest CPython pre-releases).

In some cases, both usage models may exist within the same organisation for different purposes, such as:

  • using a stable Python environment for mission critical systems, but allowing data scientists to use the latest available version for ad hoc data analysis
  • a hardware manufacturer deploying a stable Python version as part of their production firmware, but using the latest available version in the development and execution of their automated integration tests

Under any release model, each new release of Python generates work for these organisations. This work may involve legal, security or technical reviews of Python itself, assessment and verification of impactful changes, reapplication of patches, recompilation and testing of third-party dependencies, and only then deployment.

Organisations that can take updates quickly should be able to make use of the more frequent beta releases. While each update will still require similar investigative work to what they require today, the volume of work required per release should be reduced as each release will be more similar to the previous than it is under the present model. One advantage of the proposed release-every-2-months model is that organisations can choose their own adoption cadence from adopting every beta release, to adopting one per quarter, or one every 6 months, or one every year. Beyond that, it would likely make more sense to continue using the stable releases instead.

For organisations with stricter evaluations or a preference for stability, the longer release cycle for stable releases will reduce the annual effort required to update, the longer release candidate period will allow more time to do internal testing before the X.Y.0 release, and the greater use by others during the beta period will provide more confidence in the initial releases. Meanwhile, the organisation can confidently upgrade through maintenance releases for a longer time without fear of breaking changes.

Acknowledgements

Thanks to Łukasz Langa for creating PEP 602 and prompting this discussion of possible improvements to the CPython release cadence, and to Kyle Stanley and h-vetinari for constructive feedback on the initial draft of this PEP.

References

1
Steve Dower’s initial “Fast and Stable releases” proposal (https://discuss.python.org/t/pep-602-annual-release-cycle-for-python/2296/20)
2
Paul Moore’s initial comments on Steve’s proposal (https://discuss.python.org/t/pep-602-annual-release-cycle-for-python/2296/37)
3
NEP 29 proposes a common policy for dropping support of old Python versions (https://numpy.org/neps/nep-0029-deprecation_policy.html)
4
Example implementation for a pre-release SOABI flag (https://github.com/ncoghlan/cpython/pull/3)
5
CPython stable ABI documentation (https://docs.python.org/3/c-api/stable.html)
6
Travis CI nightly CPython builds (https://docs.travis-ci.com/user/languages/python/#nightly-build-support)

Source: https://github.com/python/peps/blob/master/pep-0605.rst

Last modified: 2021-09-17 18:18:24 GMT