CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 536 – Final Grammar for Literal String Interpolation | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0536/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 536 – Final Grammar for Literal String Interpolation | peps.python.org'> <meta property="og:description" content="PEP 498 introduced Literal String Interpolation (or “f-strings”). The expression portions of those literals however are subject to certain restrictions. This PEP proposes a formal grammar lifting those restrictions, promoting “f-strings” to “f expressi..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0536/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="PEP 498 introduced Literal String Interpolation (or “f-strings”). The expression portions of those literals however are subject to certain restrictions. This PEP proposes a formal grammar lifting those restrictions, promoting “f-strings” to “f expressi..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 536</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 536 – Final Grammar for Literal String Interpolation</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Philipp Angerer <phil.angerer at gmail.com></dd> <dt class="field-even">Status<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Removed from consideration by sponsor or authors">Withdrawn</abbr></dd> <dt class="field-odd">Type<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-even">Created<span class="colon">:</span></dt> <dd class="field-even">11-Dec-2016</dd> <dt class="field-odd">Python-Version<span class="colon">:</span></dt> <dd class="field-odd">3.7</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/thread/FOYKXOFWEINPVQSK2XGEHKXSTEVO5WWA/" title="Python-Ideas thread">18-Aug-2016</a>, <a class="reference external" href="https://mail.python.org/archives/list/python-ideas@python.org/thread/YKKEA5NIMMKHZTMRE5UFHST4WQ4NN3XJ/" title="Python-Ideas thread">23-Dec-2016</a>, <a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/thread/N43O4KNLZW4U7YZC4NVPCETZIVRDUVU2/" title="Python-Dev thread">15-Mar-2019</a></dd> <dt class="field-odd">Resolution<span class="colon">:</span></dt> <dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/pep-536-should-be-marked-as-rejected/35226/4">Discourse message</a></dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#pep-withdrawal">PEP Withdrawal</a></li> <li><a class="reference internal" href="#terminology">Terminology</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p><a class="pep reference internal" href="../pep-0498/" title="PEP 498 – Literal String Interpolation">PEP 498</a> introduced Literal String Interpolation (or “f-strings”). The expression portions of those literals however are subject to certain restrictions. This PEP proposes a formal grammar lifting those restrictions, promoting “f-strings” to “f expressions” or f-literals.</p> <p>This PEP expands upon the f-strings introduced by <a class="pep reference internal" href="../pep-0498/" title="PEP 498 – Literal String Interpolation">PEP 498</a>, so this text requires familiarity with <a class="pep reference internal" href="../pep-0498/" title="PEP 498 – Literal String Interpolation">PEP 498</a>.</p> </section> <section id="pep-withdrawal"> <h2><a class="toc-backref" href="#pep-withdrawal" role="doc-backlink">PEP Withdrawal</a></h2> <p>This PEP has been withdrawn in favour of <a class="pep reference internal" href="../pep-0701/" title="PEP 701 – Syntactic formalization of f-strings">PEP 701</a>. <a class="pep reference internal" href="../pep-0701/" title="PEP 701 – Syntactic formalization of f-strings">PEP 701</a> addresses all important points of this PEP.</p> </section> <section id="terminology"> <h2><a class="toc-backref" href="#terminology" role="doc-backlink">Terminology</a></h2> <p>This text will refer to the existing grammar as “f-strings”, and the proposed one as “f-literals”.</p> <p>Furthermore, it will refer to the <code class="docutils literal notranslate"><span class="pre">{}</span></code>-delimited expressions in f-literals/f-strings as “expression portions” and the static string content around them as “string portions”.</p> </section> <section id="motivation"> <h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2> <p>The current implementation of f-strings in CPython relies on the existing string parsing machinery and a post processing of its tokens. This results in several restrictions to the possible expressions usable within f-strings:</p> <ol class="arabic"> <li>It is impossible to use the quote character delimiting the f-string within the expression portion:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="sa">f</span><span class="s1">'Magic wand: </span><span class="si">{</span><span class="w"> </span><span class="n">bag</span><span class="p">[</span><span class="s1">'wand'</span><span class="p">]</span><span class="w"> </span><span class="si">}</span><span class="s1">'</span> <span class="go"> ^</span> <span class="go">SyntaxError: invalid syntax</span> </pre></div> </div> </li> <li>A previously considered way around it would lead to escape sequences in executed code and is prohibited in f-strings:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>>>> f'Magic wand { bag[\'wand\'] } string' SyntaxError: f-string expression portion cannot include a backslash </pre></div> </div> </li> <li>Comments are forbidden even in multi-line f-strings:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>>>> f'''A complex trick: { ... bag['bag'] # recursive bags! ... }''' SyntaxError: f-string expression part cannot include '#' </pre></div> </div> </li> <li>Expression portions need to wrap <code class="docutils literal notranslate"><span class="pre">':'</span></code> and <code class="docutils literal notranslate"><span class="pre">'!'</span></code> in braces:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="sa">f</span><span class="s1">'Useless use of lambdas: </span><span class="si">{</span><span class="w"> </span><span class="k">lambda</span><span class="w"> </span><span class="n">x</span><span class="si">:</span><span class="s1"> x*2 </span><span class="si">}</span><span class="s1">'</span> <span class="go">SyntaxError: unexpected EOF while parsing</span> </pre></div> </div> </li> </ol> <p>These limitations serve no purpose from a language user perspective and can be lifted by giving f-literals a regular grammar without exceptions and implementing it using dedicated parse code.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>The restrictions mentioned in <a class="reference internal" href="#motivation">Motivation</a> are non-obvious and counter-intuitive unless the user is familiar with the f-literals’ implementation details.</p> <p>As mentioned, a previous version of <a class="pep reference internal" href="../pep-0498/" title="PEP 498 – Literal String Interpolation">PEP 498</a> allowed escape sequences anywhere in f-strings, including as ways to encode the braces delimiting the expression portions and in their code. They would be expanded before the code is parsed, which would have had several important ramifications:</p> <p>#. It would not be clear to human readers which portions are Expressions and which are strings. Great material for an “obfuscated/underhanded Python challenge” #. Syntax highlighters are good in parsing nested grammar, but not in recognizing escape sequences. ECMAScript 2016 (JavaScript) allows escape sequences in its identifiers <a class="footnote-reference brackets" href="#id3" id="id1">[1]</a> and the author knows of no syntax highlighter able to correctly highlight code making use of this.</p> <p>As a consequence, the expression portions would be harder to recognize with and without the aid of syntax highlighting. With the new grammar, it is easy to extend syntax highlighters to correctly parse and display f-literals:</p> <pre><span style=color:#ff5500>f'Magic wand: </span><span style=color:#3daee9>{</span>bag[<span style=color:#bf0303>'wand'</span>]<span style=color:#3daee9>:^10}</span><span style=color:#ff5500>'</span></pre><p>Highlighting expression portions with possible escape sequences would mean to create a modified copy of all rules of the complete expression grammar, accounting for the possibility of escape sequences in key words, delimiters, and all other language syntax. One such duplication would yield one level of escaping depth and have to be repeated for a deeper escaping in a recursive f-literal. This is the case since no highlighting engine known to the author supports expanding escape sequences before applying rules to a certain context. Nesting contexts however is a standard feature of all highlighting engines.</p> <p>Familiarity also plays a role: Arbitrary nesting of expressions without expansion of escape sequences is available in every single other language employing a string interpolation method that uses expressions instead of just variable names. <a class="footnote-reference brackets" href="#id4" id="id2">[2]</a></p> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p><a class="pep reference internal" href="../pep-0498/" title="PEP 498 – Literal String Interpolation">PEP 498</a> specified f-strings as the following, but places restrictions on it:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">f</span> <span class="s1">' <text> { <expression> <optional !s, !r, or !a> <optional : format specifier> } <text> ... '</span> </pre></div> </div> <p>All restrictions mentioned in the PEP are lifted from f-literals, as explained below:</p> <ol class="arabic simple"> <li>Expression portions may now contain strings delimited with the same kind of quote that is used to delimit the f-literal.</li> <li>Backslashes may now appear within expressions just like anywhere else in Python code. In case of strings nested within f-literals, escape sequences are expanded when the innermost string is evaluated.</li> <li>Comments, using the <code class="docutils literal notranslate"><span class="pre">'#'</span></code> character, are possible only in multi-line f-literals, since comments are terminated by the end of the line (which makes closing a single-line f-literal impossible).</li> <li>Expression portions may contain <code class="docutils literal notranslate"><span class="pre">':'</span></code> or <code class="docutils literal notranslate"><span class="pre">'!'</span></code> wherever syntactically valid. The first <code class="docutils literal notranslate"><span class="pre">':'</span></code> or <code class="docutils literal notranslate"><span class="pre">'!'</span></code> that is not part of an expression has to be followed a valid coercion or format specifier.</li> </ol> <p>A remaining restriction not explicitly mentioned by <a class="pep reference internal" href="../pep-0498/" title="PEP 498 – Literal String Interpolation">PEP 498</a> is line breaks in expression portions. Since strings delimited by single <code class="docutils literal notranslate"><span class="pre">'</span></code> or <code class="docutils literal notranslate"><span class="pre">"</span></code> characters are expected to be single line, line breaks remain illegal in expression portions of single line strings.</p> <div class="admonition note"> <p class="admonition-title">Note</p> <p>Is lifting of the restrictions sufficient, or should we specify a more complete grammar?</p> </div> </section> <section id="backwards-compatibility"> <h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2> <p>f-literals are fully backwards compatible to f-strings, and expands the syntax considered legal.</p> </section> <section id="reference-implementation"> <h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2> <p>TBD</p> </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id3" role="doc-footnote"> <dt class="label" id="id3">[<a href="#id1">1</a>]</dt> <dd>ECMAScript <code class="docutils literal notranslate"><span class="pre">IdentifierName</span></code> specification ( <a class="reference external" href="http://ecma-international.org/ecma-262/6.0/#sec-names-and-keywords">http://ecma-international.org/ecma-262/6.0/#sec-names-and-keywords</a> )<p>Yes, <code class="docutils literal notranslate"><span class="pre">const</span> <span class="pre">cthulhu</span> <span class="pre">=</span> <span class="pre">{</span> <span class="pre">H̹̙̦̮͉̩̗̗ͧ̇̏̊̾Eͨ͆͒̆ͮ̃͏̷̮̣̫̤̣Cͯ̂͐͏̨̛͔̦̟͈̻O̜͎͍͙͚̬̝̣̽ͮ͐͗̀ͤ̍̀͢M̴̡̲̭͍͇̼̟̯̦̉̒͠Ḛ̛̙̞̪̗ͥͤͩ̾͑̔͐ͅṮ̴̷̷̗̼͍̿̿̓̽͐H̙̙̔̄͜\u0042:</span> <span class="pre">42</span> <span class="pre">}</span></code> is valid ECMAScript 2016</p> </aside> <aside class="footnote brackets" id="id4" role="doc-footnote"> <dt class="label" id="id4">[<a href="#id2">2</a>]</dt> <dd>Wikipedia article on string interpolation ( <a class="reference external" href="https://en.wikipedia.org/wiki/String_interpolation">https://en.wikipedia.org/wiki/String_interpolation</a> )</aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0536.rst">https://github.com/python/peps/blob/main/peps/pep-0536.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0536.rst">2023-10-26 09:35:14 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#pep-withdrawal">PEP Withdrawal</a></li> <li><a class="reference internal" href="#terminology">Terminology</a></li> <li><a class="reference internal" href="#motivation">Motivation</a></li> <li><a class="reference internal" href="#rationale">Rationale</a></li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li> <li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0536.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>