PEP 553 – Built-in breakpoint()

PEP: 553
Title: Built-in breakpoint()
Author: Barry Warsaw <barry at python.org>
Status: Final
Type: Standards Track
Created: 05-Sep-2017
Python-Version: 3.7
Post-History: 2017-09-05, 2017-09-07, 2017-09-13
Resolution: https://mail.python.org/pipermail/python-dev/2017-October/149705.html

Abstract
Rationale
Proposal
Environment variable
Implementation
Rejected alternatives
- A new keyword
- sys.breakpoint()
Version History
References
Copyright

Abstract

This PEP proposes adding a new built-in function called breakpoint() which enters a Python debugger at the point of the call. Additionally, two new names are added to the sys module to make the choice of which debugger is entered configurable.

Rationale

Python has long had a great debugger in its standard library called pdb. Setting a break point is commonly written like this:

foo()
import pdb; pdb.set_trace()
bar()

Thus after executing foo() and before executing bar(), Python will enter the debugger. However this idiom has several disadvantages.

It’s a lot to type (27 characters).
It’s easy to typo. The PEP author often mistypes this line, e.g. omitting the semicolon, or typing a dot instead of an underscore.
It ties debugging directly to the choice of pdb. There might be other debugging options, say if you’re using an IDE or some other development environment.
Python linters (e.g. flake8 [linters]) complain about this line because it contains two statements. Breaking the idiom up into two lines complicates its use because there are more opportunities for mistakes at clean up time. I.e. you might forget to delete one of those lines when you no longer need to debug the code.

Python developers also have many other debuggers to choose from, but remembering how to invoke them can be problematic. For example, even when IDEs have user interface for setting breakpoints, it may still be more convenient to just edit the code. The APIs for entering the debugger programmatically are inconsistent, so it can be difficult to remember exactly what to type.

We can solve all these problems by providing a universal API for entering the debugger, as proposed in this PEP.

Proposal

The JavaScript language provides a debugger statement [js-debugger] which enters the debugger at the point where the statement appears.

This PEP proposes a new built-in function called breakpoint() which enters a Python debugger at the call site. Thus the example above would be written like so:

foo()
breakpoint()
bar()

Further, this PEP proposes two new name bindings for the sys module, called sys.breakpointhook() and sys.__breakpointhook__. By default, sys.breakpointhook() implements the actual importing and entry into pdb.set_trace(), and it can be set to a different function to change the debugger that breakpoint() enters.

sys.__breakpointhook__ is initialized to the same function as sys.breakpointhook() so that you can always easily reset sys.breakpointhook() to the default value (e.g. by doing sys.breakpointhook = sys.__breakpointhook__). This is exactly the same as how the existing sys.displayhook() / sys.__displayhook__ and sys.excepthook() / sys.__excepthook__ work [hooks].

The signature of the built-in is breakpoint(*args, **kws). The positional and keyword arguments are passed straight through to sys.breakpointhook() and the signatures must match or a TypeError will be raised. The return from sys.breakpointhook() is passed back up to, and returned from breakpoint().

The rationale for this is based on the observation that the underlying debuggers may accept additional optional arguments. For example, IPython allows you to specify a string that gets printed when the break point is entered [ipython-embed]. As of Python 3.7, the pdb module also supports an optional header argument [pdb-header].

Environment variable

The default implementation of sys.breakpointhook() consults a new environment variable called PYTHONBREAKPOINT. This environment variable can have various values:

PYTHONBREAKPOINT=0 disables debugging. Specifically, with this value sys.breakpointhook() returns None immediately.
PYTHONBREAKPOINT= (i.e. the empty string). This is the same as not setting the environment variable at all, in which case pdb.set_trace() is run as usual.
PYTHONBREAKPOINT=some.importable.callable. In this case, sys.breakpointhook() imports the some.importable module and gets the callable object from the resulting module, which it then calls. The value may be a string with no dots, in which case it names a built-in callable, e.g. PYTHONBREAKPOINT=int. (Guido has expressed the preference for normal Python dotted-paths, not setuptools-style entry point syntax [syntax].)

This environment variable allows external processes to control how breakpoints are handled. Some uses cases include:

Completely disabling all accidental breakpoint() calls pushed to production. This could be accomplished by setting PYTHONBREAKPOINT=0 in the execution environment. Another suggestion by reviewers of the PEP was to set PYTHONBREAKPOINT=sys.exit in this case.
IDE integration with specialized debuggers for embedded execution. The IDE would run the program in its debugging environment with PYTHONBREAKPOINT set to their internal debugging hook.

PYTHONBREAKPOINT is re-interpreted every time sys.breakpointhook() is reached. This allows processes to change its value during the execution of a program and have breakpoint() respond to those changes. It is not considered a performance critical section since entering a debugger by definition stops execution. Thus, programs can do the following:

os.environ['PYTHONBREAKPOINT'] = 'foo.bar.baz'
breakpoint()    # Imports foo.bar and calls foo.bar.baz()

Overriding sys.breakpointhook defeats the default consultation of PYTHONBREAKPOINT. It is up to the overriding code to consult PYTHONBREAKPOINT if they want.

If access to the PYTHONBREAKPOINT callable fails in any way (e.g. the import fails, or the resulting module does not contain the callable), a RuntimeWarning is issued, and no breakpoint function is called.

Note that as with all other PYTHON* environment variables, PYTHONBREAKPOINT is ignored when the interpreter is started with -E. This means the default behavior will occur (i.e. pdb.set_trace() will run). There was some discussion about alternatively treating PYTHONBREAKPOINT=0 when -E as in effect, but the opinions were inconclusive, so it was decided that this wasn’t special enough for a special case.

Implementation

A pull request exists with the proposed implementation [impl].

While the actual implementation is in C, the Python pseudo-code for this feature looks roughly like the following:

# In builtins.
def breakpoint(*args, **kws):
    import sys
    missing = object()
    hook = getattr(sys, 'breakpointhook', missing)
    if hook is missing:
        raise RuntimeError('lost sys.breakpointhook')
    return hook(*args, **kws)

# In sys.
def breakpointhook(*args, **kws):
    import importlib, os, warnings
    hookname = os.getenv('PYTHONBREAKPOINT')
    if hookname is None or len(hookname) == 0:
        hookname = 'pdb.set_trace'
    elif hookname == '0':
        return None
    modname, dot, funcname = hookname.rpartition('.')
    if dot == '':
        modname = 'builtins'
    try:
        module = importlib.import_module(modname)
        hook = getattr(module, funcname)
    except:
        warnings.warn(
            'Ignoring unimportable $PYTHONBREAKPOINT: {}'.format(
                hookname),
            RuntimeWarning)
        return None
    return hook(*args, **kws)

__breakpointhook__ = breakpointhook

Rejected alternatives

A new keyword

Originally, the author considered a new keyword, or an extension to an existing keyword such as break here. This is rejected on several fronts.

A brand new keyword would require a __future__ to enable it since almost any new keyword could conflict with existing code. This negates the ease with which you can enter the debugger.
An extended keyword such as break here, while more readable and not requiring a __future__ would tie the keyword extension to this new feature, preventing more useful extensions such as those proposed in PEP 548.
A new keyword would require a modified grammar and likely a new bytecode. Each of these makes the implementation more complex. A new built-in breaks no existing code (since any existing module global would just shadow the built-in) and is quite easy to implement.

sys.breakpoint()

Why not sys.breakpoint()? Requiring an import to invoke the debugger is explicitly rejected because sys is not imported in every module. That just requires more typing and would lead to:

import sys; sys.breakpoint()

which inherits several of the problems this PEP aims to solve.

Version History

2019-10-13
- Add missing return None in except clause to pseudo-code.
2017-09-13
- The PYTHONBREAKPOINT environment variable is made a first class feature.
2017-09-07
- debug() renamed to breakpoint()
- Signature changed to breakpoint(*args, **kws) which is passed straight through to sys.breakpointhook().