No use of globals() or locals()

In the discussions on python-dev 4, a number of solutions where presented that used locals() and globals() or their equivalents. All of these have various problems. Among these are referencing variables that are not otherwise used in a closure. Consider:

>>> def outer(x):
...     def inner():
...         return 'x={x}'.format_map(locals())
...     return inner
...
>>> outer(42)()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in inner
KeyError: 'x'

This returns an error because the compiler has not added a reference to x inside the closure. You need to manually add a reference to x in order for this to work:

>>> def outer(x):
...     def inner():
...         x
...         return 'x={x}'.format_map(locals())
...     return inner
...
>>> outer(42)()
'x=42'

In addition, using locals() or globals() introduces an information leak. A called routine that has access to the callers locals() or globals() has access to far more information than needed to do the string interpolation.

Guido stated 5 that any solution to better string interpolation would not use locals() or globals() in its implementation. (This does not forbid users from passing locals() or globals() in, it just doesn’t require it, nor does it allow using these functions under the hood.)

Escape sequences

Backslashes may not appear inside the expression portions of f-strings, so you cannot use them, for example, to escape quotes inside f-strings:

>>> f'{\'quoted string\'}'
  File "<stdin>", line 1
SyntaxError: f-string expression part cannot include a backslash

You can use a different type of quote inside the expression:

>>> f'{"quoted string"}'
'quoted string'

Backslash escapes may appear inside the string portions of an f-string.

Note that the correct way to have a literal brace appear in the resulting string value is to double the brace:

>>> f'{{ {4*10} }}'
'{ 40 }'
>>> f'{{{4*10}}}'
'{40}'

Like all raw strings in Python, no escape processing is done for raw f-strings:

>>> fr'x={4*10}\n'
'x=40\\n'

Due to Python’s string tokenizing rules, the f-string f'abc {a['x']} def' is invalid. The tokenizer parses this as 3 tokens: f'abc {a[', x, and ']} def'. Just like regular strings, this cannot be fixed by using raw strings. There are a number of correct ways to write this f-string: with a different quote character:

f"abc {a['x']} def"

Or with triple quotes:

f'''abc {a['x']} def'''

Code equivalence

The exact code used to implement f-strings is not specified. However, it is guaranteed that any embedded value that is converted to a string will use that value’s __format__ method. This is the same mechanism that str.format() uses to convert values to strings.

For example, this code:

f'abc{expr1:spec1}{expr2!r:spec2}def{expr3}ghi'

Might be evaluated as:

'abc' + format(expr1, spec1) + format(repr(expr2), spec2) + 'def' + format(expr3) + 'ghi'

Expression evaluation

The expressions that are extracted from the string are evaluated in the context where the f-string appeared. This means the expression has full access to local and global variables. Any valid Python expression can be used, including function and method calls.

Because the f-strings are evaluated where the string appears in the source code, there is no additional expressiveness available with f-strings. There are also no additional security concerns: you could have also just written the same expression, not inside of an f-string:

>>> def foo():
...   return 20
...
>>> f'result={foo()}'
'result=20'

Is equivalent to:

>>> 'result=' + str(foo())
'result=20'

Expressions are parsed with the equivalent of ast.parse('(' + expression + ')', '<fstring>', 'eval') 7.

Note that since the expression is enclosed by implicit parentheses before evaluation, expressions can contain newlines. For example:

>>> x = 0
>>> f'''{x
... +1}'''
'1'

>>> d = {0: 'zero'}
>>> f'''{d[0
... ]}'''
'zero'

Format specifiers

Format specifiers may also contain evaluated expressions. This allows code such as:

>>> width = 10
>>> precision = 4
>>> value = decimal.Decimal('12.34567')
>>> f'result: {value:{width}.{precision}}'
'result:      12.35'

Once expressions in a format specifier are evaluated (if necessary), format specifiers are not interpreted by the f-string evaluator. Just as in str.format(), they are merely passed in to the __format__() method of the object being formatted.

Concatenating strings

Adjacent f-strings and regular strings are concatenated. Regular strings are concatenated at compile time, and f-strings are concatenated at run time. For example, the expression:

>>> x = 10
>>> y = 'hi'
>>> 'a' 'b' f'{x}' '{c}' f'str<{y:^4}>' 'd' 'e'

yields the value:

'ab10{c}str< hi >de'

While the exact method of this run time concatenation is unspecified, the above code might evaluate to:

'ab' + format(x) + '{c}' + 'str<' + format(y, '^4') + '>de'

Each f-string is entirely evaluated before being concatenated to adjacent f-strings. That means that this:

>>> f'{x' f'}'

Is a syntax error, because the first f-string does not contain a closing brace.

Error handling

Either compile time or run time errors can occur when processing f-strings. Compile time errors are limited to those errors that can be detected when scanning an f-string. These errors all raise SyntaxError.

Unmatched braces:

>>> f'x={x'
  File "<stdin>", line 1
SyntaxError: f-string: expecting '}'

Invalid expressions:

>>> f'x={!x}'
  File "<stdin>", line 1
SyntaxError: f-string: empty expression not allowed

Run time errors occur when evaluating the expressions inside an f-string. Note that an f-string can be evaluated multiple times, and work sometimes and raise an error at other times:

>>> d = {0:10, 1:20}
>>> for i in range(3):
...     print(f'{i}:{d[i]}')
...
0:10
1:20
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
KeyError: 2

or:

>>> for x in (32, 100, 'fifty'):
...   print(f'x = {x:+3}')
...
'x = +32'
'x = +100'
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError: Sign not allowed in string format specifier

Leading and trailing whitespace in expressions is ignored

For ease of readability, leading and trailing whitespace in expressions is ignored. This is a by-product of enclosing the expression in parentheses before evaluation.

Evaluation order of expressions

The expressions in an f-string are evaluated in left-to-right order. This is detectable only if the expressions have side effects:

>>> def fn(l, incr):
...    result = l[0]
...    l[0] += incr
...    return result
...
>>> lst = [0]
>>> f'{fn(lst,2)} {fn(lst,3)}'
'0 2'
>>> f'{fn(lst,2)} {fn(lst,3)}'
'5 7'
>>> lst
[10]

python-ideas discussion

Most of the discussions on python-ideas 8 focused on three issues:

• How to denote f-strings,
• How to specify the location of expressions in f-strings, and
• Whether to allow full Python expressions.

>>> f'Column={col_idx+1}'
>>> f'number of items: {len(items)}'

>>> col_number = col_idx+1
>>> f'Column={col_number}'
>>> n_items = len(items)
>>> f'number of items: {n_items}'

>>> f'mapping is { {a:b for (a, b) in ((1, 2), (3, 4))} }'
'mapping is {1: 2, 3: 4}'

Similar support in other languages

Wikipedia has a good discussion of string interpolation in other programming languages 9. This feature is implemented in many languages, with a variety of syntaxes and restrictions.

Differences between f-string and str.format expressions

There is one small difference between the limited expressions allowed in str.format() and the full expressions allowed inside f-strings. The difference is in how index lookups are performed. In str.format(), index values that do not look like numbers are converted to strings:

>>> d = {'a': 10, 'b': 20}
>>> 'a={d[a]}'.format(d=d)
'a=10'

Notice that the index value is converted to the string 'a' when it is looked up in the dict.

However, in f-strings, you would need to use a literal for the value of 'a':

>>> f'a={d["a"]}'
'a=10'

This difference is required because otherwise you would not be able to use variables as index values:

>>> a = 'b'
>>> f'a={d[a]}'
'a=20'

See 10 for a further discussion. It was this observation that led to full Python expressions being supported in f-strings.

Furthermore, the limited expressions that str.format() understands need not be valid Python expressions. For example:

>>> '{i[";]}'.format(i={'";':4})
'4'

For this reason, the str.format() “expression parser” is not suitable for use when implementing f-strings.

Triple-quoted f-strings

Triple quoted f-strings are allowed. These strings are parsed just as normal triple-quoted strings are. After parsing and decoding, the normal f-string logic is applied, and __format__() is called on each value.

Raw f-strings

Raw and f-strings may be combined. For example, they could be used to build up regular expressions:

>>> header = 'Subject'
>>> fr'{header}:\s+'
'Subject:\\s+'

In addition, raw f-strings may be combined with triple-quoted strings.

No binary f-strings

For the same reason that we don’t support bytes.format(), you may not combine 'f' with 'b' string literals. The primary problem is that an object’s __format__() method may return Unicode data that is not compatible with a bytes string.

Binary f-strings would first require a solution for bytes.format(). This idea has been proposed in the past, most recently in PEP 461 11. The discussions of such a feature usually suggest either

• adding a method such as __bformat__() so an object can control how it is converted to bytes, or
• having bytes.format() not be as general purpose or extensible as str.format().

Both of these remain as options in the future, if such functionality is desired.

!s, !r, and !a are redundant

The !s, !r, and !a conversions are not strictly required. Because arbitrary expressions are allowed inside the f-strings, this code:

>>> a = 'some string'
>>> f'{a!r}'
"'some string'"

Is identical to:

>>> f'{repr(a)}'
"'some string'"

Similarly, !s can be replaced by calls to str() and !a by calls to ascii().

However, !s, !r, and !a are supported by this PEP in order to minimize the differences with str.format(). !s, !r, and !a are required in str.format() because it does not allow the execution of arbitrary expressions.

Lambdas inside expressions

Because lambdas use the ':' character, they cannot appear outside of parentheses in an expression. The colon is interpreted as the start of the format specifier, which means the start of the lambda expression is seen and is syntactically invalid. As there’s no practical use for a plain lambda in an f-string expression, this is not seen as much of a limitation.

If you feel you must use lambdas, they may be used inside of parentheses:

>>> f'{(lambda x: x*2)(3)}'
'6'

Can’t combine with ‘u’

The ‘u’ prefix was added to Python 3.3 in PEP 414 as a means to ease source compatibility with Python 2.7. Because Python 2.7 will never support f-strings, there is nothing to be gained by being able to combine the ‘f’ prefix with ‘u’.