PEP 309 – Partial Function Application

PEP: 309
Title: Partial Function Application
Author: Peter Harris <scav at blueyonder.co.uk>
Status: Final
Type: Standards Track
Created: 08-Feb-2003
Python-Version: 2.5
Post-History: 10-Feb-2003, 27-Feb-2003, 22-Feb-2004, 28-Apr-2006

Note
Abstract
Acceptance
Motivation
Example Implementation
Examples of Use
Abandoned Syntax Proposal
Feedback from comp.lang.python and python-dev
Summary
References
Copyright

Note

Following the acceptance of this PEP, further discussion on python-dev and comp.lang.python revealed a desire for several tools that operated on function objects, but were not related to functional programming. Rather than create a new module for these tools, it was agreed 1 that the “functional” module be renamed to “functools” to reflect its newly-widened focus.

References in this PEP to a “functional” module have been left in for historical reasons.

Abstract

This proposal is for a function or callable class that allows a new callable to be constructed from a callable and a partial argument list (including positional and keyword arguments).

I propose a standard library module called “functional”, to hold useful higher-order functions, including the implementation of partial().

An implementation has been submitted to SourceForge 2.

Acceptance

Patch #941881 was accepted and applied in 2005 for Py2.5. It is essentially as outlined here, a partial() type constructor binding leftmost positional arguments and any keywords. The partial object has three read-only attributes func, args, and keywords. Calls to the partial object can specify keywords that override those in the object itself.

There is a separate and continuing discussion of whether to modify the partial implementation with a __get__ method to more closely emulate the behavior of an equivalent function.

Motivation

In functional programming, function currying is a way of implementing multi-argument functions in terms of single-argument functions. A function with N arguments is really a function with 1 argument that returns another function taking (N-1) arguments. Function application in languages like Haskell and ML works such that a function call:

f x y z

actually means:

(((f x) y) z)

This would be only an obscure theoretical issue except that in actual programming it turns out to be very useful. Expressing a function in terms of partial application of arguments to another function can be both elegant and powerful, and in functional languages it is heavily used.

In some functional languages, (e.g. Miranda) you can use an expression such as (+1) to mean the equivalent of Python’s (lambda x: x + 1).

In general, languages like that are strongly typed, so the compiler always knows the number of arguments expected and can do the right thing when presented with a functor and less arguments than expected.

Python does not implement multi-argument functions by currying, so if you want a function with partially-applied arguments you would probably use a lambda as above, or define a named function for each instance.

However, lambda syntax is not to everyone’s taste, so say the least. Furthermore, Python’s flexible parameter passing using both positional and keyword presents an opportunity to generalise the idea of partial application and do things that lambda cannot.

Example Implementation

Here is one way to do a create a callable with partially-applied arguments in Python. The implementation below is based on improvements provided by Scott David Daniels:

class partial(object):

    def __init__(*args, **kw):
        self = args[0]
        self.fn, self.args, self.kw = (args[1], args[2:], kw)

    def __call__(self, *args, **kw):
        if kw and self.kw:
            d = self.kw.copy()
            d.update(kw)
        else:
            d = kw or self.kw
        return self.fn(*(self.args + args), **d)

(A recipe similar to this has been in the Python Cookbook for some time 3.)

Note that when the object is called as though it were a function, positional arguments are appended to those provided to the constructor, and keyword arguments override and augment those provided to the constructor.

Positional arguments, keyword arguments or both can be supplied at when creating the object and when calling it.

Examples of Use

So partial(operator.add, 1) is a bit like (lambda x: 1 + x). Not an example where you see the benefits, of course.

Note too, that you could wrap a class in the same way, since classes themselves are callable factories for objects. So in some cases, rather than defining a subclass, you can specialise classes by partial application of the arguments to the constructor.

For example, partial(Tkinter.Label, fg='blue') makes Tkinter Labels that have a blue foreground by default.

Here’s a simple example that uses partial application to construct callbacks for Tkinter widgets on the fly:

from Tkinter import Tk, Canvas, Button
import sys
from functional import partial

win = Tk()
c = Canvas(win,width=200,height=50)
c.pack()

for colour in sys.argv[1:]:
    b = Button(win, text=colour,
               command=partial(c.config, bg=colour))
    b.pack(side='left')

win.mainloop()

Abandoned Syntax Proposal

I originally suggested the syntax fn@(*args, **kw), meaning the same as partial(fn, *args, **kw).

The @ sign is used in some assembly languages to imply register indirection, and the use here is also a kind of indirection. f@(x) is not f(x), but a thing that becomes f(x) when you call it.

It was not well-received, so I have withdrawn this part of the proposal. In any case, @ has been taken for the new decorator syntax.

Feedback from comp.lang.python and python-dev

Among the opinions voiced were the following (which I summarise):

Lambda is good enough.
The @ syntax is ugly (unanimous).
It’s really a curry rather than a closure. There is an almost identical implementation of a curry class on ActiveState’s Python Cookbook.
A curry class would indeed be a useful addition to the standard library.
It isn’t function currying, but partial application. Hence the name is now proposed to be partial().
It maybe isn’t useful enough to be in the built-ins.
The idea of a module called functional was well received, and there are other things that belong there (for example function composition).
For completeness, another object that appends partial arguments after those supplied in the function call (maybe called rightcurry) has been suggested.

I agree that lambda is usually good enough, just not always. And I want the possibility of useful introspection and subclassing.

I disagree that @ is particularly ugly, but it may be that I’m just weird. We have dictionary, list and tuple literals neatly differentiated by special punctuation – a way of directly expressing partially-applied function literals is not such a stretch. However, not one single person has said they like it, so as far as I’m concerned it’s a dead parrot.

I concur with calling the class partial rather than curry or closure, so I have amended the proposal in this PEP accordingly. But not throughout: some incorrect references to ‘curry’ have been left in since that’s where the discussion was at the time.

Partially applying arguments from the right, or inserting arguments at arbitrary positions creates its own problems, but pending discovery of a good implementation and non-confusing semantics, I don’t think it should be ruled out.

Carl Banks posted an implementation as a real functional closure:

def curry(fn, *cargs, **ckwargs):
    def call_fn(*fargs, **fkwargs):
        d = ckwargs.copy()
        d.update(fkwargs)
        return fn(*(cargs + fargs), **d)
    return call_fn

which he assures me is more efficient.

I also coded the class in Pyrex, to estimate how the performance might be improved by coding it in C:

cdef class curry:

    cdef object fn, args, kw

    def __init__(self, fn, *args, **kw):
        self.fn=fn
        self.args=args
        self.kw = kw

    def __call__(self, *args, **kw):
        if self.kw:        # from Python Cookbook version
            d = self.kw.copy()
            d.update(kw)
        else:
            d=kw
        return self.fn(*(self.args + args), **d)

The performance gain in Pyrex is less than 100% over the nested function implementation, since to be fully general it has to operate by Python API calls. For the same reason, a C implementation will be unlikely to be much faster, so the case for a built-in coded in C is not very strong.

Summary

I prefer that some means to partially-apply functions and other callables should be present in the standard library.

A standard library module functional should contain an implementation of partial, and any other higher-order functions the community want. Other functions that might belong there fall outside the scope of this PEP though.

Patches for the implementation, documentation and unit tests (SF patches 931005 4, 931007 5, and 931010 6 respectively) have been submitted but not yet checked in.

A C implementation by Hye-Shik Chang has also been submitted, although it is not expected to be included until after the Python implementation has proven itself useful enough to be worth optimising.