Strategy “Strict dictionary”

This strategy acknowledges that __getitem__ is special in accepting only one object, and the nature of that object must be non-ambiguous in its specification of the axes: it can be either by order, or by name. As a result of this assumption, in presence of keyword arguments, the passed entity is a dictionary and all labels must be specified.

C0. a[1]; a[1,2]      -> idx = 1; idx = (1, 2)
C1. a[Z=3]            -> idx = {"Z": 3}
C2. a[Z=3, R=4]       -> idx = {"Z": 3, "R": 4}
C3. a[1, Z=3]         -> raise SyntaxError
C4. a[1, Z=3, R=4]    -> raise SyntaxError
C5. a[1, 2, Z=3]      -> raise SyntaxError
C6. a[1, 2, Z=3, R=4] -> raise SyntaxError
C7. a[1, Z=3, 2, R=4] -> raise SyntaxError

Strategy “mixed dictionary”

This strategy relaxes the above constraint to return a dictionary containing both numbers and strings as keys.

C0. a[1]; a[1,2]      -> idx = 1; idx = (1, 2)
C1. a[Z=3]            -> idx = {"Z": 3}
C2. a[Z=3, R=4]       -> idx = {"Z": 3, "R": 4}
C3. a[1, Z=3]         -> idx = { 0: 1, "Z": 3}
C4. a[1, Z=3, R=4]    -> idx = { 0: 1, "Z": 3, "R": 4}
C5. a[1, 2, Z=3]      -> idx = { 0: 1, 1: 2, "Z": 3}
C6. a[1, 2, Z=3, R=4] -> idx = { 0: 1, 1: 2, "Z": 3, "R": 4}
C7. a[1, Z=3, 2, R=4] -> idx = { 0: 1, "Z": 3, 2: 2, "R": 4}

Strategy “named tuple”

Return a named tuple for idx instead of a tuple. Keyword arguments would obviously have their stated name as key, and positional argument would have an underscore followed by their order:

C0. a[1]; a[1,2]      -> idx = 1; idx = (_0=1, _1=2)
C1. a[Z=3]            -> idx = (Z=3)
C2. a[Z=3, R=2]       -> idx = (Z=3, R=2)
C3. a[1, Z=3]         -> idx = (_0=1, Z=3)
C4. a[1, Z=3, R=2]    -> idx = (_0=1, Z=3, R=2)
C5. a[1, 2, Z=3]      -> idx = (_0=1, _2=2, Z=3)
C6. a[1, 2, Z=3, R=4] -> (_0=1, _1=2, Z=3, R=4)
C7. a[1, Z=3, 2, R=4] -> (_0=1, Z=3, _1=2, R=4)
                      or (_0=1, Z=3, _2=2, R=4)
                      or raise SyntaxError

The required typename of the namedtuple could be Index or the name of the argument in the function definition, it keeps the ordering and is easy to analyse by using the _fields attribute. It is backward compatible, provided that C0 with more than one entry now passes a namedtuple instead of a plain tuple.

Strategy “New argument contents”

In the current implementation, when many arguments are passed to __getitem__, they are grouped in a tuple and this tuple is passed to __getitem__ as the single argument idx. This strategy keeps the current signature, but expands the range of variability in type and contents of idx to more complex representations.

We identify four possible ways to implement this strategy:

• P1: uses a single dictionary for the keyword arguments.
• P2: uses individual single-item dictionaries.
• P3: similar to P2, but replaces single-item dictionaries with a (key, value) tuple.
• P4: similar to P2, but uses a special and additional new object: keyword()

Some of these possibilities lead to degenerate notations, i.e. indistinguishable from an already possible representation. Once again, the proposed notation becomes syntactic sugar for these representations.

Under this strategy, the old behavior for C0 is unchanged.

C0: a[1]        -> idx = 1                    # integer
    a[1,2]      -> idx = (1,2)                # tuple

In C1, we can use either a dictionary or a tuple to represent key and value pair for the specific indexing entry. We need to have a tuple with a tuple in C1 because otherwise we cannot differentiate a["Z", 3] from a[Z=3].

C1: a[Z=3]      -> idx = {"Z": 3}             # P1/P2 dictionary with single key
                or idx = (("Z", 3),)          # P3 tuple of tuples
                or idx = keyword("Z", 3)      # P4 keyword object

As you can see, notation P1/P2 implies that a[Z=3] and a[{"Z": 3}] will call __getitem__ passing the exact same value, and is therefore syntactic sugar for the latter. Same situation occurs, although with different index, for P3. Using a keyword object as in P4 would remove this degeneracy.

For the C2 case:

C2. a[Z=3, R=4] -> idx = {"Z": 3, "R": 4}     # P1 dictionary/ordereddict
                or idx = ({"Z": 3}, {"R": 4}) # P2 tuple of two single-key dict
                or idx = (("Z", 3), ("R", 4)) # P3 tuple of tuples
                or idx = (keyword("Z", 3),
                          keyword("R", 4) )   # P4 keyword objects

P1 naturally maps to the traditional **kwargs behavior, however it breaks the convention that two or more entries for the index produce a tuple. P2 preserves this behavior, and additionally preserves the order. Preserving the order would also be possible with an OrderedDict as drafted by PEP-468 5.

The remaining cases are here shown:

C3. a[1, Z=3]   -> idx = (1, {"Z": 3})                     # P1/P2
                or idx = (1, ("Z", 3))                     # P3
                or idx = (1, keyword("Z", 3))              # P4

C4. a[1, Z=3, R=4] -> idx = (1, {"Z": 3, "R": 4})          # P1
                   or idx = (1, {"Z": 3}, {"R": 4})        # P2
                   or idx = (1, ("Z", 3), ("R", 4))        # P3
                   or idx = (1, keyword("Z", 3),
                                keyword("R", 4))           # P4

C5. a[1, 2, Z=3]   -> idx = (1, 2, {"Z": 3})               # P1/P2
                   or idx = (1, 2, ("Z", 3))               # P3
                   or idx = (1, 2, keyword("Z", 3))        # P4

C6. a[1, 2, Z=3, R=4] -> idx = (1, 2, {"Z":3, "R": 4})     # P1
                      or idx = (1, 2, {"Z": 3}, {"R": 4})  # P2
                      or idx = (1, 2, ("Z", 3), ("R", 4))  # P3
                      or idx = (1, 2, keyword("Z", 3),
                                      keyword("R", 4))     # P4

C7. a[1, Z=3, 2, R=4] -> idx = (1, 2, {"Z": 3, "R": 4})    # P1. Pack the keyword arguments. Ugly.
                      or raise SyntaxError                 # P1. Same behavior as in function calls.
                      or idx = (1, {"Z": 3}, 2, {"R": 4})  # P2
                      or idx =  (1, ("Z", 3), 2, ("R", 4)) # P3
                      or idx =  (1, keyword("Z", 3),
                                 2, keyword("R", 4))       # P4

Strategy “kwargs argument”

__getitem__ accepts an optional **kwargs argument which should be keyword only. idx also becomes optional to support a case where no non-keyword arguments are allowed. The signature would then be either

__getitem__(self, idx)
__getitem__(self, idx, **kwargs)
__getitem__(self, **kwargs)

Applied to our cases would produce:

C0. a[1,2]            -> idx=(1,2);  kwargs={}
C1. a[Z=3]            -> idx=None ;  kwargs={"Z":3}
C2. a[Z=3, R=4]       -> idx=None ;  kwargs={"Z":3, "R":4}
C3. a[1, Z=3]         -> idx=1    ;  kwargs={"Z":3}
C4. a[1, Z=3, R=4]    -> idx=1    ;  kwargs={"Z":3, "R":4}
C5. a[1, 2, Z=3]      -> idx=(1,2);  kwargs={"Z":3}
C6. a[1, 2, Z=3, R=4] -> idx=(1,2);  kwargs={"Z":3, "R":4}
C7. a[1, Z=3, 2, R=4] -> raise SyntaxError # in agreement to function behavior

Empty indexing a[] of course remains invalid syntax.

Use a method

One could keep the indexing as is, and use a traditional get() method for those cases where basic indexing is not enough. This is a good point, but as already reported in the introduction, methods have a different semantic weight from indexing, and you can’t use slices directly in methods. Compare e.g. a[1:3, Z=2] with a.get(slice(1,3), Z=2).

The authors however recognize this argument as compelling, and the advantage in semantic expressivity of a keyword-based indexing may be offset by a rarely used feature that does not bring enough benefit and may have limited adoption.

Emulate requested behavior by abusing the slice object

This extremely creative method exploits the slice objects’ behavior, provided that one accepts to use strings (or instantiate properly named placeholder objects for the keys), and accept to use “:” instead of “=”.

>>> a["K":3]
slice('K', 3, None)
>>> a["K":3, "R":4]
(slice('K', 3, None), slice('R', 4, None))
>>>

While clearly smart, this approach does not allow easy inquire of the key/value pair, it’s too clever and esotheric, and does not allow to pass a slice as in a[K=1:10:2].

However, Tim Delaney comments

“I really do think that a[b=c, d=e] should just be syntax sugar for a['b':c, 'd':e]. It’s simple to explain, and gives the greatest backwards compatibility. In particular, libraries that already abused slices in this way will just continue to work with the new syntax.”

We think this behavior would produce inconvenient results. The library Pandas uses strings as labels, allowing notation such as

>>> a[:, "A":"F"]

to extract data from column “A” to column “F”. Under the above comment, this notation would be equally obtained with

>>> a[:, A="F"]

which is weird and collides with the intended meaning of keyword in indexing, that is, specifying the axis through conventional names rather than positioning.

Pass a dictionary as an additional index

>>> a[1, 2, {"K": 3}]

this notation, although less elegant, can already be used and achieves similar results. It’s evident that the proposed Strategy “New argument contents” can be interpreted as syntactic sugar for this notation.

Relevance of ordering of keyword arguments

As part of the discussion of this PEP, it’s important to decide if the ordering information of the keyword arguments is important, and if indexes and keys can be ordered in an arbitrary way (e.g. a[1,Z=3,2,R=4]). PEP-468 5 tries to address the first point by proposing the use of an ordereddict, however one would be inclined to accept that keyword arguments in indexing are equivalent to kwargs in function calls, and therefore as of today equally unordered, and with the same restrictions.

Need for homogeneity of behavior

Relative to Strategy “New argument contents”, a comment from Ian Cordasco points out that

“it would be unreasonable for just one method to behave totally differently from the standard behaviour in Python. It would be confusing for only __getitem__ (and ostensibly, __setitem__) to take keyword arguments but instead of turning them into a dictionary, turn them into individual single-item dictionaries.” We agree with his point, however it must be pointed out that __getitem__ is already special in some regards when it comes to passed arguments.

Chris Angelico also states:

“it seems very odd to start out by saying “here, let’s give indexing the option to carry keyword args, just like with function calls”, and then come back and say “oh, but unlike function calls, they’re inherently ordered and carried very differently”.” Again, we agree on this point. The most straightforward strategy to keep homogeneity would be Strategy “kwargs argument”, opening to a **kwargs argument on __getitem__.

One of the authors (Stefano Borini) thinks that only the “strict dictionary” strategy is worth of implementation. It is non-ambiguous, simple, does not force complex parsing, and addresses the problem of referring to axes either by position or by name. The “options” use case is probably best handled with a different approach, and may be irrelevant for this PEP. The alternative “named tuple” is another valid choice.

Having .get() become obsolete for indexing with default fallback

Introducing a “default” keyword could make dict.get() obsolete, which would be replaced by d["key", default=3]. Chris Angelico however states:

“Currently, you need to write __getitem__ (which raises an exception on finding a problem) plus something else, e.g. get(), which returns a default instead. By your proposal, both branches would go inside __getitem__, which means they could share code; but there still need to be two branches.”

Additionally, Chris continues:

“There’ll be an ad-hoc and fairly arbitrary puddle of names (some will go default=, others will say that’s way too long and go def=, except that that’s a keyword so they’ll use dflt= or something…), unless there’s a strong force pushing people to one consistent name.”.

This argument is valid but it’s equally valid for any function call, and is generally fixed by established convention and documentation.

On degeneracy of notation

User Drekin commented: “The case of a[Z=3] and a[{"Z": 3}] is similar to current a[1, 2] and a[(1, 2)]. Even though one may argue that the parentheses are actually not part of tuple notation but are just needed because of syntax, it may look as degeneracy of notation when compared to function call: f(1, 2) is not the same thing as f((1, 2)).”.