Printf-style formatting, via operator

This venerable technique 19 continues to have its uses, such as with byte-based protocols, simplicity in simple cases, and familiarity to many programmers:

>>> 'Hello, user: %(user)s, id: %(id)s, on host: %(hostname)s' % params
'Hello, user: nobody, id: 9, on host: darkstar'

In this form, considering the prerequisite dictionary creation, the technique is verbose, a tad noisy, yet relatively readable. Additional issues are that an operator can only take one argument besides the original string, meaning multiple parameters must be passed in a tuple or dictionary. Also, it is relatively easy to make an error in the number of arguments passed, the expected type, have a missing key, or forget the trailing type, e.g. (s or d).

string.Template Class

The string.Template class from 20 PEP 292 (Simpler String Substitutions) is a purposely simplified design, using familiar shell interpolation syntax, with safe-substitution feature 21, that finds its main use cases in shell and internationalization tools:

Template('Hello, user: $user, id: ${id}, on host: $hostname').substitute(params)

While also verbose, the string itself is readable. Though functionality is limited, it meets its requirements well. It isn’t powerful enough for many cases, and that helps keep inexperienced users out of trouble, as well as avoiding issues with moderately-trusted input (i18n) from third-parties. It unfortunately takes enough code to discourage its use for ad-hoc string interpolation, unless encapsulated in a convenience library 22 such as flufl.i18n.

PEP 215 - String Interpolation

PEP 215 was a former proposal of which this one shares a lot in common. Apparently, the world was not ready for it at the time, but considering recent support in a number of other languages, its day may have come.

The large number of dollar sign ($) characters it included may have led it to resemble Python’s arch-nemesis Perl, and likely contributed to the PEP’s lack of acceptance. It was superseded by the following proposal.

str.format() Method

The str.format() syntax of 16 PEP 3101 is the most recent and modern of the existing options. It is also more powerful and usually easier to read than the others. It avoids many of the drawbacks and limits of the previous techniques.

However, due to its necessary function call and parameter passing, it runs from verbose to very verbose in various situations with string literals:

>>> 'Hello, user: {user}, id: {id}, on host: {hostname}'.format(**params)
'Hello, user: nobody, id: 9, on host: darkstar'

# when using keyword args, var name shortening sometimes needed to fit :/
>>> 'Hello, user: {user}, id: {id}, on host: {host}'.format(user=user,
                                                            id=id,
                                                            host=hostname)
'Hello, user: nobody, id: 9, on host: darkstar'

The verbosity of the method-based approach is illustrated here.

PEP 498 – Literal String Formatting

PEP 498 defines and discusses format strings, as also described in the Abstract above.

It also, somewhat controversially to those first exposed, introduces the idea that format-strings shall be augmented with support for arbitrary expressions. This is discussed further in the Restricting Syntax section under Rejected Ideas.

PEP 501 – Translation ready string interpolation

The complimentary PEP 501 brings internationalization into the discussion as a first-class concern, with its proposal of the i-prefix, string.Template syntax integration compatible with ES6 (Javascript), deferred rendering, and an object return value.

Bash

Bash supports a number of arbitrary, even recursive constructs inside strings:

> echo "user: $USER, id: $((id + 6)) on host: $(echo is $(hostname))"
user: nobody, id: 15 on host: is darkstar

• Explicit interpolation within double quotes.
• Direct environment variable access supported.
• Arbitrary expressions are supported. 4
• External process execution and output capture supported. 5
• Recursive expressions are supported.

Perl

Perl also has arbitrary expression constructs, perhaps not as well known:

say "I have @{[$id + 6]} guanacos.";                # lists
say "I have ${\($id + 6)} guanacos.";               # scalars
say "Hello { @names.join(', ') } how are you?";     # Perl 6 version

• Explicit interpolation within double quotes.
• Arbitrary expressions are supported. 6 7

Ruby

Ruby allows arbitrary expressions in its interpolated strings:

puts "One plus one is two: #{1 + 1}\n"

• Explicit interpolation within double quotes.
• Arbitrary expressions are supported. 8 9
• Possible to change delimiter chars with %.
• See the Reference Implementation(s) section for an implementation in Python.

Others

Let’s look at some less-similar modern languages recently implementing string interpolation.

Scala

Scala interpolation 23 is directed through string prefixes. Each prefix has a different result:

s"Hello, $name ${1 + 1}"                    # arbitrary
f"$name%s is $height%2.2f meters tall"      # printf-style
raw"a\nb"                                   # raw, like r''

These prefixes may also be implemented by the user, by extending Scala’s StringContext class.

• Explicit interpolation within double quotes with literal prefix.
• User implemented prefixes supported.
• Arbitrary expressions are supported.

ES6 (Javascript)

Designers of Template strings 24 faced the same issue as Python where single and double quotes were taken. Unlike Python however, “backticks” were not. Despite their issues 18, they were chosen as part of the ECMAScript 2015 (ES6) standard:

console.log(`Fifteen is ${a + b} and\nnot ${2 * a + b}.`);

Custom prefixes are also supported by implementing a function the same name as the tag:

function tag(strings, ...values) {
    console.log(strings.raw[0]);    // raw string is also available
    return "Bazinga!";
}
tag`Hello ${ a + b } world ${ a * b}`;

• Explicit interpolation within backticks.
• User implemented prefixes supported.
• Arbitrary expressions are supported.

C#, Version 6

C# has a useful new interpolation feature 25 as well, with some ability to customize interpolation 26 via the IFormattable interface:

$"{person.Name, 20} is {person.Age:D3} year{(p.Age == 1 ? "" : "s")} old.";

• Explicit interpolation with double quotes and $ prefix.
• Custom interpolations are available.
• Arbitrary expressions are supported.

Apple’s Swift

Arbitrary interpolation under Swift 27 is available on all strings:

let multiplier = 3
let message = "\(multiplier) times 2.5 is \(Double(multiplier) * 2.5)"
// message is "3 times 2.5 is 7.5"

• Implicit interpolation with double quotes.
• Arbitrary expressions are supported.
• Cannot contain CR/LF.

Additional examples

A number of additional examples of string interpolation may be found at Wikipedia 28.

Now that background and history have been covered, let’s continue on for a solution.

Mitigation via Tools

The idea is that tools or linters such as pyflakes, pylint, or Pycharm, may check inside strings with expressions and mark them up appropriately. As this is a common task with programming languages today, multi-language tools won’t have to implement this feature solely for Python, significantly shortening time to implementation.

Farther in the future, strings might also be checked for constructs that exceed the safety policy of a project.

Internationalization

Though it was highly desired to integrate internationalization support, (see PEP 501), the finer details diverge at almost every point, making a common solution unlikely: 15

• Use-cases differ
• Compile vs. run-time tasks
• Interpolation syntax needs
• Intended audience
• Security policy

Restricting Syntax to str.format() Only

The common arguments against 33 support of arbitrary expressions were:

1. YAGNI 32, “You aren’t gonna need it.”
2. The feature is not congruent with historical Python conservatism.
3. Postpone - can implement in a future version if need is demonstrated.

Support of only str.format() syntax however, was deemed not enough of a solution to the problem. Often a simple length or increment of an object, for example, is desired before printing.

It can be seen in the Implementations in Other Languages section that the developer community at large tends to agree. String interpolation with arbitrary expressions is becoming an industry standard in modern languages due to its utility.

Additional/Custom String-Prefixes

As seen in the Implementations in Other Languages section, many modern languages have extensible string prefixes with a common interface. This could be a way to generalize and reduce lines of code in common situations. Examples are found in ES6 (Javascript), Scala, Nim, and C# (to a lesser extent). This was rejected by the BDFL. 14

Automated Escaping of Input Variables

While helpful in some cases, this was thought to create too much uncertainty of when and where string expressions could be used safely or not. The concept was also difficult to describe to others. 12

Always consider format string variables to be unescaped, unless the developer has explicitly escaped them.

Environment Access and Command Substitution

For systems programming and shell-script replacements, it would be useful to handle environment variables and capture output of commands directly in an expression string. This was rejected as not important enough, and looking too much like bash/perl, which could encourage bad habits. 13