Protocol

The following abstract base class defines the protocol for an object to be considered a path object:

import abc
import typing as t


class PathLike(abc.ABC):

    """Abstract base class for implementing the file system path protocol."""

    @abc.abstractmethod
    def __fspath__(self) -> t.Union[str, bytes]:
        """Return the file system path representation of the object."""
        raise NotImplementedError

Objects representing file system paths will implement the __fspath__() method which will return the str or bytes representation of the path. The str representation is the preferred low-level path representation as it is human-readable and what people historically represent paths as.

Standard library changes

It is expected that most APIs in Python’s standard library that currently accept a file system path will be updated appropriately to accept path objects (whether that requires code or simply an update to documentation will vary). The modules mentioned below, though, deserve specific details as they have either fundamental changes that empower the ability to use path objects, or entail additions/removal of APIs.

import typing as t


def fspath(path: t.Union[PathLike, str, bytes]) -> t.Union[str, bytes]:
    """Return the string representation of the path.

    If str or bytes is passed in, it is returned unchanged. If __fspath__()
    returns something other than str or bytes then TypeError is raised. If
    this function is given something that is not str, bytes, or os.PathLike
    then TypeError is raised.
    """
    if isinstance(path, (str, bytes)):
        return path

    # Work from the object's type to match method resolution of other magic
    # methods.
    path_type = type(path)
    try:
        path = path_type.__fspath__(path)
    except AttributeError:
        if hasattr(path_type, '__fspath__'):
            raise
    else:
        if isinstance(path, (str, bytes)):
            return path
        else:
            raise TypeError("expected __fspath__() to return str or bytes, "
                            "not " + type(path).__name__)

    raise TypeError("expected str, bytes or os.PathLike object, not "
                    + path_type.__name__)

/*
    Return the file system path representation of the object.

    If the object is str or bytes, then allow it to pass through with
    an incremented refcount. If the object defines __fspath__(), then
    return the result of that method. All other types raise a TypeError.
*/
PyObject *
PyOS_FSPath(PyObject *path)
{
    _Py_IDENTIFIER(__fspath__);
    PyObject *func = NULL;
    PyObject *path_repr = NULL;

    if (PyUnicode_Check(path) || PyBytes_Check(path)) {
        Py_INCREF(path);
        return path;
    }

    func = _PyObject_LookupSpecial(path, &PyId___fspath__);
    if (NULL == func) {
        return PyErr_Format(PyExc_TypeError,
                            "expected str, bytes or os.PathLike object, "
                            "not %S",
                            path->ob_type);
    }

    path_repr = PyObject_CallFunctionObjArgs(func, NULL);
    Py_DECREF(func);
    if (!PyUnicode_Check(path_repr) && !PyBytes_Check(path_repr)) {
        Py_DECREF(path_repr);
        return PyErr_Format(PyExc_TypeError,
                            "expected __fspath__() to return str or bytes, "
                            "not %S",
                            path_repr->ob_type);
    }

    return path_repr;
}

Other names for the protocol’s method

Various names were proposed during discussions leading to this PEP, including __path__, __pathname__, and __fspathname__. In the end people seemed to gravitate towards __fspath__ for being unambiguous without being unnecessarily long.

Separate str/bytes methods

At one point it was suggested that __fspath__() only return strings and another method named __fspathb__() be introduced to return bytes. The thinking is that by making __fspath__() not be polymorphic it could make dealing with the potential string or bytes representations easier. But the general consensus was that returning bytes will more than likely be rare and that the various functions in the os module are the better abstraction to promote over direct calls to __fspath__().

Providing a path attribute

To help deal with the issue of pathlib.PurePath not inheriting from str, originally it was proposed to introduce a path attribute to mirror what os.DirEntry provides. In the end, though, it was determined that a protocol would provide the same result while not directly exposing an API that most people will never need to interact with directly.

Have __fspath__() only return strings

Much of the discussion that led to this PEP revolved around whether __fspath__() should be polymorphic and return bytes as well as str or only return str. The general sentiment for this view was that bytes are difficult to work with due to their inherent lack of information about their encoding and PEP 383 makes it possible to represent all file system paths using str with the surrogateescape handler. Thus, it would be better to forcibly promote the use of str as the low-level path representation for high-level path objects.

In the end, it was decided that using bytes to represent paths is simply not going to go away and thus they should be supported to some degree. The hope is that people will gravitate towards path objects like pathlib and that will move people away from operating directly with bytes.

A generic string encoding mechanism

At one point there was a discussion of developing a generic mechanism to extract a string representation of an object that had semantic meaning (__str__() does not necessarily return anything of semantic significance beyond what may be helpful for debugging). In the end, it was deemed to lack a motivating need beyond the one this PEP is trying to solve in a specific fashion.

Have __fspath__ be an attribute

It was briefly considered to have __fspath__ be an attribute instead of a method. This was rejected for two reasons. One, historically protocols have been implemented as “magic methods” and not “magic methods and attributes”. Two, there is no guarantee that the lower-level representation of a path object will be pre-computed, potentially misleading users that there was no expensive computation behind the scenes in case the attribute was implemented as a property.

This also indirectly ties into the idea of introducing a path attribute to accomplish the same thing. This idea has an added issue, though, of accidentally having any object with a path attribute meet the protocol’s duck typing. Introducing a new magic method for the protocol helpfully avoids any accidental opting into the protocol.

Provide specific type hinting support

There was some consideration to providing a generic typing.PathLike class which would allow for e.g. typing.PathLike[str] to specify a type hint for a path object which returned a string representation. While potentially beneficial, the usefulness was deemed too small to bother adding the type hint class.

This also removed any desire to have a class in the typing module which represented the union of all acceptable path-representing types as that can be represented with typing.Union[str, bytes, os.PathLike] easily enough and the hope is users will slowly gravitate to path objects only.

Provide os.fspathb()

It was suggested that to mirror the structure of e.g. os.getcwd()/os.getcwdb(), that os.fspath() only return str and that another function named os.fspathb() be introduced that only returned bytes. This was rejected as the purposes of the *b() functions are tied to querying the file system where there is a need to get the raw bytes back. As this PEP does not work directly with data on a file system (but which may be), the view was taken this distinction is unnecessary. It’s also believed that the need for only bytes will not be common enough to need to support in such a specific manner as os.fsencode() will provide similar functionality.

Call __fspath__() off of the instance

An earlier draft of this PEP had os.fspath() calling path.__fspath__() instead of type(path).__fspath__(path). The changed to be consistent with how other magic methods in Python are resolved.