@overload vs. @when

The @overload decorator is a common-case shorthand for the more general @when decorator. It allows you to leave out the name of the function you are overloading, at the expense of requiring the target function to be in the local namespace. It also doesn’t support adding additional criteria besides the ones specified via argument annotations. The following function definitions have identical effects, except for name binding side-effects (which will be described below):

from overloading import when

@overload
def flatten(ob: basestring):
    yield ob

@when(flatten)
def flatten(ob: basestring):
    yield ob

@when(flatten)
def flatten_basestring(ob: basestring):
    yield ob

@when(flatten, (basestring,))
def flatten_basestring(ob):
    yield ob

The first definition above will bind flatten to whatever it was previously bound to. The second will do the same, if it was already bound to the when decorator’s first argument. If flatten is unbound or bound to something else, it will be rebound to the function definition as given. The last two definitions above will always bind flatten_basestring to the function definition as given.

Using this approach allows you to both give a method a descriptive name (often useful in tracebacks!) and to reuse the method later.

Except as otherwise specified, all overloading decorators have the same signature and binding rules as @when. They accept a function and an optional “predicate” object.

The default predicate implementation is a tuple of types with positional matching to the overloaded function’s arguments. However, an arbitrary number of other kinds of predicates can be created and registered using the Extension API, and will then be usable with @when and other decorators created by this module (like @before, @after, and @around).

Proceeding to the “Next” Method

If the first parameter of an overloaded function is named __proceed__, it will be passed a callable representing the next most-specific method. For example, this code:

def foo(bar:object, baz:object):
    print "got objects!"

@overload
def foo(__proceed__, bar:int, baz:int):
    print "got integers!"
    return __proceed__(bar, baz)

Will print “got integers!” followed by “got objects!”.

If there is no next most-specific method, __proceed__ will be bound to a NoApplicableMethods instance. When called, a new NoApplicableMethods instance will be raised, with the arguments passed to the first instance.

Similarly, if the next most-specific methods have ambiguous precedence with respect to each other, __proceed__ will be bound to an AmbiguousMethods instance, and if called, it will raise a new instance.

Thus, a method can either check if __proceed__ is an error instance, or simply invoke it. The NoApplicableMethods and AmbiguousMethods error classes have a common DispatchError base class, so isinstance(__proceed__, overloading.DispatchError) is sufficient to identify whether __proceed__ can be safely called.

(Implementation note: using a magic argument name like __proceed__ could potentially be replaced by a magic function that would be called to obtain the next method. A magic function, however, would degrade performance and might be more difficult to implement on non-CPython platforms. Method chaining via magic argument names, however, can be efficiently implemented on any Python platform that supports creating bound methods from functions – one simply recursively binds each function to be chained, using the following function or error as the im_self of the bound method.)

“Before” and “After” Methods

In addition to the simple next-method chaining shown above, it is sometimes useful to have other ways of combining methods. For example, the “observer pattern” can sometimes be implemented by adding extra methods to a function, that execute before or after the normal implementation.

To support these use cases, the overloading module will supply @before, @after, and @around decorators, that roughly correspond to the same types of methods in the Common Lisp Object System (CLOS), or the corresponding “advice” types in AspectJ.

Like @when, all of these decorators must be passed the function to be overloaded, and can optionally accept a predicate as well:

from overloading import before, after

def begin_transaction(db):
    print "Beginning the actual transaction"

@before(begin_transaction)
def check_single_access(db: SingletonDB):
    if db.inuse:
        raise TransactionError("Database already in use")

@after(begin_transaction)
def start_logging(db: LoggableDB):
    db.set_log_level(VERBOSE)

@before and @after methods are invoked either before or after the main function body, and are never considered ambiguous. That is, it will not cause any errors to have multiple “before” or “after” methods with identical or overlapping signatures. Ambiguities are resolved using the order in which the methods were added to the target function.

“Before” methods are invoked most-specific method first, with ambiguous methods being executed in the order they were added. All “before” methods are called before any of the function’s “primary” methods (i.e. normal @overload methods) are executed.

“After” methods are invoked in the reverse order, after all of the function’s “primary” methods are executed. That is, they are executed least-specific methods first, with ambiguous methods being executed in the reverse of the order in which they were added.

The return values of both “before” and “after” methods are ignored, and any uncaught exceptions raised by any methods (primary or other) immediately end the dispatching process. “Before” and “after” methods cannot have __proceed__ arguments, as they are not responsible for calling any other methods. They are simply called as a notification before or after the primary methods.

Thus, “before” and “after” methods can be used to check or establish preconditions (e.g. by raising an error if the conditions aren’t met) or to ensure postconditions, without needing to duplicate any existing functionality.

“Around” Methods

The @around decorator declares a method as an “around” method. “Around” methods are much like primary methods, except that the least-specific “around” method has higher precedence than the most-specific “before” method.

Unlike “before” and “after” methods, however, “Around” methods are responsible for calling their __proceed__ argument, in order to continue the invocation process. “Around” methods are usually used to transform input arguments or return values, or to wrap specific cases with special error handling or try/finally conditions, e.g.:

from overloading import around

@around(commit_transaction)
def lock_while_committing(__proceed__, db: SingletonDB):
    with db.global_lock:
        return __proceed__(db)

They can also be used to replace the normal handling for a specific case, by not invoking the __proceed__ function.

The __proceed__ given to an “around” method will either be the next applicable “around” method, a DispatchError instance, or a synthetic method object that will call all the “before” methods, followed by the primary method chain, followed by all the “after” methods, and return the result from the primary method chain.

Thus, just as with normal methods, __proceed__ can be checked for DispatchError-ness, or simply invoked. The “around” method should return the value returned by __proceed__, unless of course it wishes to modify or replace it with a different return value for the function as a whole.

Custom Combinations

The decorators described above (@overload, @when, @before, @after, and @around) collectively implement what in CLOS is called the “standard method combination” – the most common patterns used in combining methods.

Sometimes, however, an application or library may have use for a more sophisticated type of method combination. For example, if you would like to have “discount” methods that return a percentage off, to be subtracted from the value returned by the primary method(s), you might write something like this:

from overloading import always_overrides, merge_by_default
from overloading import Around, Before, After, Method, MethodList

class Discount(MethodList):
    """Apply return values as discounts"""

    def __call__(self, *args, **kw):
        retval = self.tail(*args, **kw)
        for sig, body in self.sorted():
            retval -= retval * body(*args, **kw)
        return retval

# merge discounts by priority
merge_by_default(Discount)

# discounts have precedence over before/after/primary methods
always_overrides(Discount, Before)
always_overrides(Discount, After)
always_overrides(Discount, Method)

# but not over "around" methods
always_overrides(Around, Discount)

# Make a decorator called "discount" that works just like the
# standard decorators...
discount = Discount.make_decorator('discount')

# and now let's use it...
def price(product):
    return product.list_price

@discount(price)
def ten_percent_off_shoes(product: Shoe)
    return Decimal('0.1')

Similar techniques can be used to implement a wide variety of CLOS-style method qualifiers and combination rules. The process of creating custom method combination objects and their corresponding decorators is described in more detail under the Extension API section.

Note, by the way, that the @discount decorator shown will work correctly with any new predicates defined by other code. For example, if zope.interface were to register its interface types to work correctly as argument annotations, you would be able to specify discounts on the basis of its interface types, not just classes or overloading-defined interface types.

Similarly, if a library like RuleDispatch or PEAK-Rules were to register an appropriate predicate implementation and dispatch engine, one would then be able to use those predicates for discounts as well, e.g.:

from somewhere import Pred  # some predicate implementation

@discount(
    price,
    Pred("isinstance(product,Shoe) and"
         " product.material.name=='Blue Suede'")
)
def forty_off_blue_suede_shoes(product):
    return Decimal('0.4')

The process of defining custom predicate types and dispatching engines is also described in more detail under the Extension API section.

Abstract and Concrete Methods

Note, by the way, that the @abstract decorator is not limited to use in interface definitions; it can be used anywhere that you wish to create an “empty” generic function that initially has no methods. In particular, it need not be used inside a class.

Also note that interface methods need not be abstract; one could, for example, write an interface like this:

class IWriteMapping(Interface):
    @abstract
    def __setitem__(self, key, value):
        """This has to be implemented"""

    def update(self, other:IReadMapping):
        for k, v in IReadMapping(other).items():
            self[k] = v

As long as __setitem__ is defined for some type, the above interface will provide a usable update() implementation. However, if some specific type (or pair of types) has a more efficient way of handling update() operations, an appropriate overload can still be registered for use in that case.

Subclassing and Re-assembly

Interfaces can be subclassed:

class ISizedStack(IStack):
    @abstract
    def __len__(self):
        """Return the number of items on the stack"""

# define __len__ support for ISizedStack
when(ISizedStack.__len__, (list,))(list.__len__)

Or assembled by combining functions from existing interfaces:

class Sizable(Interface):
    __len__ = ISizedStack.__len__

# list now implements Sizable as well as ISizedStack, without
# making any new declarations!

A class can be considered to “adapt to” an interface at a given point in time, if no method defined in the interface is guaranteed to raise a NoApplicableMethods error if invoked on an instance of that class at that point in time.

In normal usage, however, it is “easier to ask forgiveness than permission”. That is, it is easier to simply use an interface on an object by adapting it to the interface (e.g. IStack(mylist)) or invoking interface methods directly (e.g. IStack.push(mylist, 42)), than to try to figure out whether the object is adaptable to (or directly implements) the interface.

Implementing an Interface in a Class

It is possible to declare that a class directly implements an interface, using the declare_implementation() function:

from overloading import declare_implementation

class Stack(object):
    def __init__(self):
        self.data = []
    def push(self, ob):
        self.data.append(ob)
    def pop(self):
        return self.data.pop()

declare_implementation(IStack, Stack)

The declare_implementation() call above is roughly equivalent to the following steps:

when(IStack.push, (Stack,object))(lambda self, ob: self.push(ob))
when(IStack.pop, (Stack,))(lambda self, ob: self.pop())

That is, calling IStack.push() or IStack.pop() on an instance of any subclass of Stack, will simply delegate to the actual push() or pop() methods thereof.

For the sake of efficiency, calling IStack(s) where s is an instance of Stack, may return s rather than an IStack adapter. (Note that calling IStack(x) where x is already an IStack adapter will always return x unchanged; this is an additional optimization allowed in cases where the adaptee is known to directly implement the interface, without adaptation.)

For convenience, it may be useful to declare implementations in the class header, e.g.:

class Stack(metaclass=Implementer, implements=IStack):
    ...

Instead of calling declare_implementation() after the end of the suite.

Interfaces as Type Specifiers

Interface subclasses can be used as argument annotations to indicate what type of objects are acceptable to an overload, e.g.:

@overload
def traverse(g: IGraph, s: IStack):
    g = IGraph(g)
    s = IStack(s)
    # etc....

Note, however, that the actual arguments are not changed or adapted in any way by the mere use of an interface as a type specifier. You must explicitly cast the objects to the appropriate interface, as shown above.

Note, however, that other patterns of interface use are possible. For example, other interface implementations might not support adaptation, or might require that function arguments already be adapted to the specified interface. So the exact semantics of using an interface as a type specifier are dependent on the interface objects you actually use.

For the interface objects defined by this PEP, however, the semantics are as described above. An interface I1 is considered “more specific” than another interface I2, if the set of descriptors in I1’s inheritance hierarchy are a proper superset of the descriptors in I2’s inheritance hierarchy.

So, for example, ISizedStack is more specific than both ISizable and ISizedStack, irrespective of the inheritance relationships between these interfaces. It is purely a question of what operations are included within those interfaces – and the names of the operations are unimportant.

Interfaces (at least the ones provided by overloading) are always considered less-specific than concrete classes. Other interface implementations can decide on their own specificity rules, both between interfaces and other interfaces, and between interfaces and classes.

Non-Method Attributes in Interfaces

The Interface implementation actually treats all attributes and methods (i.e. descriptors) in the same way: their __get__ (and __set__ and __delete__, if present) methods are called with the wrapped (adapted) object as “self”. For functions, this has the effect of creating a bound method linking the generic function to the wrapped object.

For non-function attributes, it may be easiest to specify them using the property built-in, and the corresponding fget, fset, and fdel attributes:

class ILength(Interface):
    @property
    @abstract
    def length(self):
        """Read-only length attribute"""

# ILength(aList).length == list.__len__(aList)
when(ILength.length.fget, (list,))(list.__len__)

Alternatively, methods such as _get_foo() and _set_foo() may be defined as part of the interface, and the property defined in terms of those methods, but this is a bit more difficult for users to implement correctly when creating a class that directly implements the interface, as they would then need to match all the individual method names, not just the name of the property or attribute.