CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 436 – The Argument Clinic DSL | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0436/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 436 – The Argument Clinic DSL | peps.python.org'> <meta property="og:description" content="This document proposes “Argument Clinic”, a DSL to facilitate argument processing for built-in functions in the implementation of CPython."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0436/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="This document proposes “Argument Clinic”, a DSL to facilitate argument processing for built-in functions in the implementation of CPython."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 436</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> Toggle light / dark / auto colour theme </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 436 – The Argument Clinic DSL</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author:</dt> <dd class="field-odd">Larry Hastings <larry at hastings.org></dd> <dt class="field-even">Discussions-To:</dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/">Python-Dev list</a></dd> <dt class="field-odd">Status:</dt> <dd class="field-odd"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-even">Type:</dt> <dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-odd">Created:</dt> <dd class="field-odd">22-Feb-2013</dd> <dt class="field-even">Python-Version:</dt> <dd class="field-even">3.4</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale-and-goals">Rationale and Goals</a></li> <li><a class="reference internal" href="#dsl-syntax-summary">DSL Syntax Summary</a><ul> <li><a class="reference internal" href="#general-behavior-of-the-argument-clinic-dsl">General Behavior Of the Argument Clinic DSL</a></li> <li><a class="reference internal" href="#module-and-class-declarations">Module and Class Declarations</a></li> <li><a class="reference internal" href="#function-declaration">Function Declaration</a></li> <li><a class="reference internal" href="#parameter-declaration">Parameter Declaration</a></li> <li><a class="reference internal" href="#legacy-converters">Legacy Converters</a></li> <li><a class="reference internal" href="#parameter-docstrings">Parameter Docstrings</a></li> <li><a class="reference internal" href="#special-syntax-for-parameter-lines">Special Syntax For Parameter Lines</a></li> <li><a class="reference internal" href="#function-docstring">Function Docstring</a></li> <li><a class="reference internal" href="#converters">Converters</a></li> <li><a class="reference internal" href="#return-converters">Return Converters</a></li> <li><a class="reference internal" href="#directives">Directives</a></li> </ul> </li> <li><a class="reference internal" href="#python-code">Python Code</a></li> <li><a class="reference internal" href="#output">Output</a></li> <li><a class="reference internal" href="#functions-with-positional-only-parameters">Functions With Positional-Only Parameters</a></li> <li><a class="reference internal" href="#current-status">Current Status</a><ul> <li><a class="reference internal" href="#argument-clinic-programmatic-interfaces">Argument Clinic Programmatic Interfaces</a></li> </ul> </li> <li><a class="reference internal" href="#notes-tbd">Notes / TBD</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> This document proposes “Argument Clinic”, a DSL to facilitate argument processing for built-in functions in the implementation of CPython. </section> <section id="rationale-and-goals"> <h2><a class="toc-backref" href="#rationale-and-goals" role="doc-backlink">Rationale and Goals</a></h2> The primary implementation of Python, “CPython”, is written in a mixture of Python and C. One implementation detail of CPython is what are called “built-in” functions – functions available to Python programs but written in C. When a Python program calls a built-in function and passes in arguments, those arguments must be translated from Python values into C values. This process is called “parsing arguments”. As of CPython 3.3, builtin functions nearly always parse their arguments with one of two functions: the original <code class="docutils literal notranslate">PyArg_ParseTuple()</code>, <a class="footnote-reference brackets" href="#id8" id="id1">[1]</a> and the more modern <code class="docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code>. <a class="footnote-reference brackets" href="#id9" id="id2">[2]</a> The former only handles positional parameters; the latter also accommodates keyword and keyword-only parameters, and is preferred for new code. With either function, the caller specifies the translation for parsing arguments in a “format string”: <a class="footnote-reference brackets" href="#id10" id="id3">[3]</a> each parameter corresponds to a “format unit”, a short character sequence telling the parsing function what Python types to accept and how to translate them into the appropriate C value for that parameter. <code class="docutils literal notranslate">PyArg_ParseTuple()</code> was reasonable when it was first conceived. There were only a dozen or so of these “format units”; each one was distinct, and easy to understand and remember. But over the years the <code class="docutils literal notranslate">PyArg_Parse</code> interface has been extended in numerous ways. The modern API is complex, to the point that it is somewhat painful to use. Consider: <ul class="simple"> <li>There are now forty different “format units”; a few are even three characters long. This makes it difficult for the programmer to understand what the format string says–or even perhaps to parse it–without constantly cross-indexing it with the documentation.</li> <li>There are also six meta-format units that may be buried in the format string. (They are: <code class="docutils literal notranslate">"()|$:;"</code>.)</li> <li>The more format units are added, the less likely it is the implementer can pick an easy-to-use mnemonic for the format unit, because the character of choice is probably already in use. In other words, the more format units we have, the more obtuse the format units become.</li> <li>Several format units are nearly identical to others, having only subtle differences. This makes understanding the exact semantics of the format string even harder, and can make it difficult to figure out exactly which format unit you want.</li> <li>The docstring is specified as a static C string, making it mildly bothersome to read and edit since it must obey C string quoting rules.</li> <li>When adding a new parameter to a function using <code class="docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code>, it’s necessary to touch six different places in the code: <a class="footnote-reference brackets" href="#id11" id="id4">[4]</a><ul> <li>Declaring the variable to store the argument.</li> <li>Passing in a pointer to that variable in the correct spot in <code class="docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code>, also passing in any “length” or “converter” arguments in the correct order.</li> <li>Adding the name of the argument in the correct spot of the “keywords” array passed in to <code class="docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code>.</li> <li>Adding the format unit to the correct spot in the format string.</li> <li>Adding the parameter to the prototype in the docstring.</li> <li>Documenting the parameter in the docstring.</li> </ul> </li> <li>There is currently no mechanism for builtin functions to provide their “signature” information (see <code class="docutils literal notranslate">inspect.getfullargspec</code> and <code class="docutils literal notranslate">inspect.Signature</code>). Adding this information using a mechanism similar to the existing <code class="docutils literal notranslate">PyArg_Parse</code> functions would require repeating ourselves yet again.</li> </ul> The goal of Argument Clinic is to replace this API with a mechanism inheriting none of these downsides: <ul class="simple"> <li>You need specify each parameter only once.</li> <li>All information about a parameter is kept together in one place.</li> <li>For each parameter, you specify a conversion function; Argument Clinic handles the translation from Python value into C value for you.</li> <li>Argument Clinic also allows for fine-tuning of argument processing behavior with parameterized conversion functions.</li> <li>Docstrings are written in plain text. Function docstrings are required; per-parameter docstrings are encouraged.</li> <li>From this, Argument Clinic generates for you all the mundane, repetitious code and data structures CPython needs internally. Once you’ve specified the interface, the next step is simply to write your implementation using native C types. Every detail of argument parsing is handled for you.</li> </ul> Argument Clinic is implemented as a preprocessor. It draws inspiration for its workflow directly from <a class="reference internal" href="#cog" id="id5">[Cog]</a> by Ned Batchelder. To use Clinic, add a block comment to your C source code beginning and ending with special text strings, then run Clinic on the file. Clinic will find the block comment, process the contents, and write the output back into your C source file directly after the comment. The intent is that Clinic’s output becomes part of your source code; it’s checked in to revision control, and distributed with source packages. This means that Python will still ship ready-to-build. It does complicate development slightly; in order to add a new function, or modify the arguments or documentation of an existing function using Clinic, you’ll need a working Python 3 interpreter. Future goals of Argument Clinic include: <ul class="simple"> <li>providing signature information for builtins,</li> <li>enabling alternative implementations of Python to create automated library compatibility tests, and</li> <li>speeding up argument parsing with improvements to the generated code.</li> </ul> </section> <section id="dsl-syntax-summary"> <h2><a class="toc-backref" href="#dsl-syntax-summary" role="doc-backlink">DSL Syntax Summary</a></h2> The Argument Clinic DSL is specified as a comment embedded in a C file, as follows. The “Example” column on the right shows you sample input to the Argument Clinic DSL, and the “Section” column on the left specifies what each line represents in turn. Argument Clinic’s DSL syntax mirrors the Python <code class="docutils literal notranslate">def</code> statement, lending it some familiarity to Python core developers. <div class="highlight-default notranslate"><div class="highlight"><pre>+-----------------------+-----------------------------------------------------------------+ | Section | Example | +-----------------------+-----------------------------------------------------------------+ | Clinic DSL start | /*[clinic] | | Module declaration | module module_name | | Class declaration | class module_name.class_name | | Function declaration | module_name.function_name -> return_annotation | | Parameter declaration | name : converter(param=value) | | Parameter docstring | Lorem ipsum dolor sit amet, consectetur | | | adipisicing elit, sed do eiusmod tempor | | Function docstring | Lorem ipsum dolor sit amet, consectetur adipisicing | | | elit, sed do eiusmod tempor incididunt ut labore et | | Clinic DSL end | [clinic]*/ | | Clinic output | ... | | Clinic output end | /*[clinic end output:<checksum>]*/ | +-----------------------+-----------------------------------------------------------------+ </pre></div> </div> To give some flavor of the proposed DSL syntax, here are some sample Clinic code blocks. This first block reflects the normally preferred style, including blank lines between parameters and per-argument docstrings. It also includes a user-defined converter (<code class="docutils literal notranslate">path_t</code>) created locally: <div class="highlight-default notranslate"><div class="highlight"><pre>/*[clinic] os.stat as os_stat_fn -> stat result path: path_t(allow_fd=1) Path to be examined; can be string, bytes, or open-file-descriptor int. * dir_fd: OS_STAT_DIR_FD_CONVERTER = DEFAULT_DIR_FD If not None, it should be a file descriptor open to a directory, and path should be a relative string; path will then be relative to that directory. follow_symlinks: bool = True If False, and the last element of the path is a symbolic link, stat will examine the symbolic link itself instead of the file the link points to. Perform a stat system call on the given path. {parameters} dir_fd and follow_symlinks may not be implemented on your platform. If they are unavailable, using them will raise a NotImplementedError. It's an error to use dir_fd or follow_symlinks when specifying path as an open file descriptor. [clinic]*/ </pre></div> </div> This second example shows a minimal Clinic code block, omitting all parameter docstrings and non-significant blank lines: <div class="highlight-default notranslate"><div class="highlight"><pre>/*[clinic] os.access path: path mode: int * dir_fd: OS_ACCESS_DIR_FD_CONVERTER = 1 effective_ids: bool = False follow_symlinks: bool = True Use the real uid/gid to test for access to a path. Returns True if granted, False otherwise. {parameters} dir_fd, effective_ids, and follow_symlinks may not be implemented on your platform. If they are unavailable, using them will raise a NotImplementedError. Note that most operations will use the effective uid/gid, therefore this routine can be used in a suid/sgid environment to test if the invoking user has the specified access to the path. [clinic]*/ </pre></div> </div> This final example shows a Clinic code block handling groups of optional parameters, including parameters on the left: <div class="highlight-default notranslate"><div class="highlight"><pre>/*[clinic] curses.window.addch [ y: int Y-coordinate. x: int X-coordinate. ] ch: char Character to add. [ attr: long Attributes for the character. ] / Paint character ch at (y, x) with attributes attr, overwriting any character previously painter at that location. By default, the character position and attributes are the current settings for the window object. [clinic]*/ </pre></div> </div> <section id="general-behavior-of-the-argument-clinic-dsl"> <h3><a class="toc-backref" href="#general-behavior-of-the-argument-clinic-dsl" role="doc-backlink">General Behavior Of the Argument Clinic DSL</a></h3> All lines support <code class="docutils literal notranslate">#</code> as a line comment delimiter except docstrings. Blank lines are always ignored. Like Python itself, leading whitespace is significant in the Argument Clinic DSL. The first line of the “function” section is the function declaration. Indented lines below the function declaration declare parameters, one per line; lines below those that are indented even further are per-parameter docstrings. Finally, the first line dedented back to column 0 end parameter declarations and start the function docstring. Parameter docstrings are optional; function docstrings are not. Functions that specify no arguments may simply specify the function declaration followed by the docstring. </section> <section id="module-and-class-declarations"> <h3><a class="toc-backref" href="#module-and-class-declarations" role="doc-backlink">Module and Class Declarations</a></h3> When a C file implements a module or class, this should be declared to Clinic. The syntax is simple: <div class="highlight-default notranslate"><div class="highlight"><pre>module module_name </pre></div> </div> or <div class="highlight-default notranslate"><div class="highlight"><pre>class module_name.class_name </pre></div> </div> (Note that these are not actually special syntax; they are implemented as <a class="reference internal" href="#directives">Directives</a>.) The module name or class name should always be the full dotted path from the top-level module. Nested modules and classes are supported. </section> <section id="function-declaration"> <h3><a class="toc-backref" href="#function-declaration" role="doc-backlink">Function Declaration</a></h3> The full form of the function declaration is as follows: <div class="highlight-default notranslate"><div class="highlight"><pre>dotted.name [ as legal_c_id ] [ -> return_annotation ] </pre></div> </div> The dotted name should be the full name of the function, starting with the highest-level package (e.g. “os.stat” or “curses.window.addch”). The “as legal_c_id” syntax is optional. Argument Clinic uses the name of the function to create the names of the generated C functions. In some circumstances, the generated name may collide with other global names in the C program’s namespace. The “as legal_c_id” syntax allows you to override the generated name with your own; substitute “legal_c_id” with any legal C identifier. If skipped, the “as” keyword must also be omitted. The return annotation is also optional. If skipped, the arrow (”<code class="docutils literal notranslate">-></code>”) must also be omitted. If specified, the value for the return annotation must be compatible with <code class="docutils literal notranslate">ast.literal_eval</code>, and it is interpreted as a return converter. </section> <section id="parameter-declaration"> <h3><a class="toc-backref" href="#parameter-declaration" role="doc-backlink">Parameter Declaration</a></h3> The full form of the parameter declaration line as follows: <div class="highlight-default notranslate"><div class="highlight"><pre>name: converter [ (parameter=value [, parameter2=value2]) ] [ = default] </pre></div> </div> The “name” must be a legal C identifier. Whitespace is permitted between the name and the colon (though this is not the preferred style). Whitespace is permitted (and encouraged) between the colon and the converter. The “converter” is the name of one of the “converter functions” registered with Argument Clinic. Clinic will ship with a number of built-in converters; new converters can also be added dynamically. In choosing a converter, you are automatically constraining what Python types are permitted on the input, and specifying what type the output variable (or variables) will be. Although many of the converters will resemble the names of C types or perhaps Python types, the name of a converter may be any legal Python identifier. If the converter is followed by parentheses, these parentheses enclose parameter to the conversion function. The syntax mirrors providing arguments a Python function call: the parameter must always be named, as if they were “keyword-only parameters”, and the values provided for the parameters will syntactically resemble Python literal values. These parameters are always optional, permitting all conversion functions to be called without any parameters. In this case, you may also omit the parentheses entirely; this is always equivalent to specifying empty parentheses. The values supplied for these parameters must be compatible with <code class="docutils literal notranslate">ast.literal_eval</code>. The “default” is a Python literal value. Default values are optional; if not specified you must omit the equals sign too. Parameters which don’t have a default are implicitly required. The default value is dynamically assigned, “live” in the generated C code, and although it’s specified as a Python value, it’s translated into a native C value in the generated C code. Few default values are permitted, owing to this manual translation step. If this were a Python function declaration, a parameter declaration would be delimited by either a trailing comma or an ending parenthesis. However, Argument Clinic uses neither; parameter declarations are delimited by a newline. A trailing comma or right parenthesis is not permitted. The first parameter declaration establishes the indent for all parameter declarations in a particular Clinic code block. All subsequent parameters must be indented to the same level. </section> <section id="legacy-converters"> <h3><a class="toc-backref" href="#legacy-converters" role="doc-backlink">Legacy Converters</a></h3> For convenience’s sake in converting existing code to Argument Clinic, Clinic provides a set of legacy converters that match <code class="docutils literal notranslate">PyArg_ParseTuple</code> format units. They are specified as a C string containing the format unit. For example, to specify a parameter “foo” as taking a Python “int” and emitting a C int, you could specify: <div class="highlight-default notranslate"><div class="highlight"><pre>foo : "i" </pre></div> </div> (To more closely resemble a C string, these must always use double quotes.) Although these resemble <code class="docutils literal notranslate">PyArg_ParseTuple</code> format units, no guarantee is made that the implementation will call a <code class="docutils literal notranslate">PyArg_Parse</code> function for parsing. This syntax does not support parameters. Therefore, it doesn’t support any of the format units that require input parameters (<code class="docutils literal notranslate">"O!", "O&", "es", "es#", "et", "et#"</code>). Parameters requiring one of these conversions cannot use the legacy syntax. (You may still, however, supply a default value.) </section> <section id="parameter-docstrings"> <h3><a class="toc-backref" href="#parameter-docstrings" role="doc-backlink">Parameter Docstrings</a></h3> All lines that appear below and are indented further than a parameter declaration are the docstring for that parameter. All such lines are “dedented” until the first line is flush left. </section> <section id="special-syntax-for-parameter-lines"> <h3><a class="toc-backref" href="#special-syntax-for-parameter-lines" role="doc-backlink">Special Syntax For Parameter Lines</a></h3> There are four special symbols that may be used in the parameter section. Each of these must appear on a line by itself, indented to the same level as parameter declarations. The four symbols are: <dl class="simple"> <dt><code class="docutils literal notranslate">*</code></dt><dd>Establishes that all subsequent parameters are keyword-only.</dd> <dt><code class="docutils literal notranslate">[</code></dt><dd>Establishes the start of an optional “group” of parameters. Note that “groups” may nest inside other “groups”. See <a class="reference internal" href="#functions-with-positional-only-parameters">Functions With Positional-Only Parameters</a> below. Note that currently <code class="docutils literal notranslate">[</code> is only legal for use in functions where all parameters are marked positional-only, see <code class="docutils literal notranslate">/</code> below.</dd> <dt><code class="docutils literal notranslate">]</code></dt><dd>Ends an optional “group” of parameters.</dd> <dt><code class="docutils literal notranslate">/</code></dt><dd>Establishes that all the proceeding arguments are positional-only. For now, Argument Clinic does not support functions with both positional-only and non-positional-only arguments. Therefore: if <code class="docutils literal notranslate">/</code> is specified for a function, it must currently always be after the last parameter. Also, Argument Clinic does not currently support default values for positional-only parameters.</dd> </dl> (The semantics of <code class="docutils literal notranslate">/</code> follow a syntax for positional-only parameters in Python once proposed by Guido. <a class="footnote-reference brackets" href="#id12" id="id6">[5]</a> ) </section> <section id="function-docstring"> <h3><a class="toc-backref" href="#function-docstring" role="doc-backlink">Function Docstring</a></h3> The first line with no leading whitespace after the function declaration is the first line of the function docstring. All subsequent lines of the Clinic block are considered part of the docstring, and their leading whitespace is preserved. If the string <code class="docutils literal notranslate">{parameters}</code> appears on a line by itself inside the function docstring, Argument Clinic will insert a list of all parameters that have docstrings, each such parameter followed by its docstring. The name of the parameter is on a line by itself; the docstring starts on a subsequent line, and all lines of the docstring are indented by two spaces. (Parameters with no per-parameter docstring are suppressed.) The entire list is indented by the leading whitespace that appeared before the <code class="docutils literal notranslate">{parameters}</code> token. If the string <code class="docutils literal notranslate">{parameters}</code> doesn’t appear in the docstring, Argument Clinic will append one to the end of the docstring, inserting a blank line above it if the docstring does not end with a blank line, and with the parameter list at column 0. </section> <section id="converters"> <h3><a class="toc-backref" href="#converters" role="doc-backlink">Converters</a></h3> Argument Clinic contains a pre-initialized registry of converter functions. Example converter functions: <dl class="simple"> <dt><code class="docutils literal notranslate">int</code></dt><dd>Accepts a Python object implementing <code class="docutils literal notranslate">__int__</code>; emits a C <code class="docutils literal notranslate">int</code>.</dd> <dt><code class="docutils literal notranslate">byte</code></dt><dd>Accepts a Python int; emits an <code class="docutils literal notranslate">unsigned char</code>. The integer must be in the range [0, 256).</dd> <dt><code class="docutils literal notranslate">str</code></dt><dd>Accepts a Python str object; emits a C <code class="docutils literal notranslate">char *</code>. Automatically encodes the string using the <code class="docutils literal notranslate">ascii</code> codec.</dd> <dt><code class="docutils literal notranslate">PyObject</code></dt><dd>Accepts any object; emits a C <code class="docutils literal notranslate">PyObject *</code> without any conversion.</dd> </dl> All converters accept the following parameters: <dl class="simple"> <dt><code class="docutils literal notranslate">doc_default</code></dt><dd>The Python value to use in place of the parameter’s actual default in Python contexts. In other words: when specified, this value will be used for the parameter’s default in the docstring, and in the <code class="docutils literal notranslate">Signature</code>. (TBD alternative semantics: If the string is a valid Python expression which can be rendered into a Python value using <code class="docutils literal notranslate">eval()</code>, then the result of <code class="docutils literal notranslate">eval()</code> on it will be used as the default in the <code class="docutils literal notranslate">Signature</code>.) Ignored if there is no default.</dd> <dt><code class="docutils literal notranslate">required</code></dt><dd>Normally any parameter that has a default value is automatically optional. A parameter that has “required” set will be considered required (non-optional) even if it has a default value. The generated documentation will also not show any default value.</dd> </dl> Additionally, converters may accept one or more of these optional parameters, on an individual basis: <dl class="simple"> <dt><code class="docutils literal notranslate">annotation</code></dt><dd>Explicitly specifies the per-parameter annotation for this parameter. Normally it’s the responsibility of the conversion function to generate the annotation (if any).</dd> <dt><code class="docutils literal notranslate">bitwise</code></dt><dd>For converters that accept unsigned integers. If the Python integer passed in is signed, copy the bits directly even if it is negative.</dd> <dt><code class="docutils literal notranslate">encoding</code></dt><dd>For converters that accept str. Encoding to use when encoding a Unicode string to a <code class="docutils literal notranslate">char *</code>.</dd> <dt><code class="docutils literal notranslate">immutable</code></dt><dd>Only accept immutable values.</dd> <dt><code class="docutils literal notranslate">length</code></dt><dd>For converters that accept iterable types. Requests that the converter also emit the length of the iterable, passed in to the <code class="docutils literal notranslate">_impl</code> function in a <code class="docutils literal notranslate">Py_ssize_t</code> variable; its name will be this parameter’s name appended with “<code class="docutils literal notranslate">_length</code>”.</dd> <dt><code class="docutils literal notranslate">nullable</code></dt><dd>This converter normally does not accept <code class="docutils literal notranslate">None</code>, but in this case it should. If <code class="docutils literal notranslate">None</code> is supplied on the Python side, the equivalent C argument will be <code class="docutils literal notranslate">NULL</code>. (The <code class="docutils literal notranslate">_impl</code> argument emitted by this converter will presumably be a pointer type.)</dd> <dt><code class="docutils literal notranslate">types</code></dt><dd>A list of strings representing acceptable Python types for this object. There are also four strings which represent Python protocols:<ul class="simple"> <li>“buffer”</li> <li>“mapping”</li> <li>“number”</li> <li>“sequence”</li> </ul> </dd> <dt><code class="docutils literal notranslate">zeroes</code></dt><dd>For converters that accept string types. The converted value should be allowed to have embedded zeroes.</dd> </dl> </section> <section id="return-converters"> <h3><a class="toc-backref" href="#return-converters" role="doc-backlink">Return Converters</a></h3> A return converter conceptually performs the inverse operation of a converter: it converts a native C value into its equivalent Python value. </section> <section id="directives"> <h3><a class="toc-backref" href="#directives" role="doc-backlink">Directives</a></h3> Argument Clinic also permits “directives” in Clinic code blocks. Directives are similar to pragmas in C; they are statements that modify Argument Clinic’s behavior. The format of a directive is as follows: <div class="highlight-default notranslate"><div class="highlight"><pre>directive_name [argument [second_argument [ ... ]]] </pre></div> </div> Directives only take positional arguments. A Clinic code block must contain either one or more directives, or a function declaration. It may contain both, in which case all directives must come before the function declaration. Internally directives map directly to Python callables. The directive’s arguments are passed directly to the callable as positional arguments of type <code class="docutils literal notranslate">str()</code>. Example possible directives include the production, suppression, or redirection of Clinic output. Also, the “module” and “class” keywords are implemented as directives in the prototype. </section> </section> <section id="python-code"> <h2><a class="toc-backref" href="#python-code" role="doc-backlink">Python Code</a></h2> Argument Clinic also permits embedding Python code inside C files, which is executed in-place when Argument Clinic processes the file. Embedded code looks like this: <div class="highlight-default notranslate"><div class="highlight"><pre>/*[python] # this is python code! print("/" + "* Hello world! *" + "/") [python]*/ /* Hello world! */ /*[python end:da39a3ee5e6b4b0d3255bfef95601890afd80709]*/ </pre></div> </div> The <code class="docutils literal notranslate">"/* Hello world! */"</code> line above was generated by running the Python code in the preceding comment. Any Python code is valid. Python code sections in Argument Clinic can also be used to directly interact with Clinic; see <a class="reference internal" href="#argument-clinic-programmatic-interfaces">Argument Clinic Programmatic Interfaces</a>. </section> <section id="output"> <h2><a class="toc-backref" href="#output" role="doc-backlink">Output</a></h2> Argument Clinic writes its output inline in the C file, immediately after the section of Clinic code. For “python” sections, the output is everything printed using <code class="docutils literal notranslate">builtins.print</code>. For “clinic” sections, the output is valid C code, including: <ul class="simple"> <li>a <code class="docutils literal notranslate">#define</code> providing the correct <code class="docutils literal notranslate">methoddef</code> structure for the function</li> <li>a prototype for the “impl” function – this is what you’ll write to implement this function</li> <li>a function that handles all argument processing, which calls your “impl” function</li> <li>the definition line of the “impl” function</li> <li>and a comment indicating the end of output.</li> </ul> The intention is that you write the body of your impl function immediately after the output – as in, you write a left-curly-brace immediately after the end-of-output comment and implement builtin in the body there. (It’s a bit strange at first, but oddly convenient.) Argument Clinic will define the parameters of the impl function for you. The function will take the “self” parameter passed in originally, all the parameters you define, and possibly some extra generated parameters (“length” parameters; also “group” parameters, see next section). Argument Clinic also writes a checksum for the output section. This is a valuable safety feature: if you modify the output by hand, Clinic will notice that the checksum doesn’t match, and will refuse to overwrite the file. (You can force Clinic to overwrite with the “<code class="docutils literal notranslate">-f</code>” command-line argument; Clinic will also ignore the checksums when using the “<code class="docutils literal notranslate">-o</code>” command-line argument.) Finally, Argument Clinic can also emit the boilerplate definition of the PyMethodDef array for the defined classes and modules. </section> <section id="functions-with-positional-only-parameters"> <h2><a class="toc-backref" href="#functions-with-positional-only-parameters" role="doc-backlink">Functions With Positional-Only Parameters</a></h2> A significant fraction of Python builtins implemented in C use the older positional-only API for processing arguments (<code class="docutils literal notranslate">PyArg_ParseTuple()</code>). In some instances, these builtins parse their arguments differently based on how many arguments were passed in. This can provide some bewildering flexibility: there may be groups of optional parameters, which must either all be specified or none specified. And occasionally these groups are on the left! (A representative example: <code class="docutils literal notranslate">curses.window.addch()</code>.) Argument Clinic supports these legacy use-cases by allowing you to specify parameters in groups. Each optional group of parameters is marked with square brackets. Note that these groups are permitted on the right or left of any required parameters! The impl function generated by Clinic will add an extra parameter for every group, “<code class="docutils literal notranslate">int group_{left|right}_<x></code>”, where x is a monotonically increasing number assigned to each group as it builds away from the required arguments. This argument will be nonzero if the group was specified on this call, and zero if it was not. Note that when operating in this mode, you cannot specify default arguments. Also, note that it’s possible to specify a set of groups to a function such that there are several valid mappings from the number of arguments to a valid set of groups. If this happens, Clinic will abort with an error message. This should not be a problem, as positional-only operation is only intended for legacy use cases, and all the legacy functions using this quirky behavior have unambiguous mappings. </section> <section id="current-status"> <h2><a class="toc-backref" href="#current-status" role="doc-backlink">Current Status</a></h2> As of this writing, there is a working prototype implementation of Argument Clinic available online (though the syntax may be out of date as you read this). <a class="footnote-reference brackets" href="#id13" id="id7">[6]</a> The prototype generates code using the existing <code class="docutils literal notranslate">PyArg_Parse</code> APIs. It supports translating to all current format units except the mysterious <code class="docutils literal notranslate">"w*"</code>. Sample functions using Argument Clinic exercise all major features, including positional-only argument parsing. <section id="argument-clinic-programmatic-interfaces"> <h3><a class="toc-backref" href="#argument-clinic-programmatic-interfaces" role="doc-backlink">Argument Clinic Programmatic Interfaces</a></h3> The prototype also currently provides an experimental extension mechanism, allowing adding support for new types on-the-fly. See <code class="docutils literal notranslate">Modules/posixmodule.c</code> in the prototype for an example of its use. In the future, Argument Clinic is expected to be automatable enough to allow querying, modification, or outright new construction of function declarations through Python code. It may even permit dynamically adding your own custom DSL! </section> </section> <section id="notes-tbd"> <h2><a class="toc-backref" href="#notes-tbd" role="doc-backlink">Notes / TBD</a></h2> <ul> <li>The API for supplying inspect.Signature metadata for builtins is currently under discussion. Argument Clinic will add support for the prototype when it becomes viable.</li> <li>Alyssa Coghlan suggests that we a) only support at most one left-optional group per function, and b) in the face of ambiguity, prefer the left group over the right group. This would solve all our existing use cases including range().</li> <li>Optimally we’d want Argument Clinic run automatically as part of the normal Python build process. But this presents a bootstrapping problem; if you don’t have a system Python 3, you need a Python 3 executable to build Python 3. I’m sure this is a solvable problem, but I don’t know what the best solution might be. (Supporting this will also require a parallel solution for Windows.)</li> <li>On a related note: inspect.Signature has no way of representing blocks of arguments, like the left-optional block of <code class="docutils literal notranslate">y</code> and <code class="docutils literal notranslate">x</code> for <code class="docutils literal notranslate">curses.window.addch</code>. How far are we going to go in supporting this admittedly aberrant parameter paradigm?</li> <li>During the PyCon US 2013 Language Summit, there was discussion of having Argument Clinic also generate the actual documentation (in ReST, processed by Sphinx) for the function. The logistics of this are TBD, but it would require that the docstrings be written in ReST, and require that Python ship a ReST -> ascii converter. It would be best to come to a decision about this before we begin any large-scale conversion of the CPython source tree to using Clinic.</li> <li>Guido proposed having the “function docstring” be hand-written inline, in the middle of the output, something like this:<div class="highlight-default notranslate"><div class="highlight"><pre>/*[clinic] ... prototype and parameters (including parameter docstrings) go here [clinic]*/ ... some output ... /*[clinic docstring start]*/ ... hand-edited function docstring goes here <-- you edit this by hand! /*[clinic docstring end]*/ ... more output /*[clinic output end]*/ </pre></div> </div> I tried it this way and don’t like it – I think it’s clumsy. I prefer that everything you write goes in one place, rather than having an island of hand-edited stuff in the middle of the DSL output. </li> <li>Argument Clinic does not support automatic tuple unpacking (the “<code class="docutils literal notranslate">(OOO)</code>” style format string for <code class="docutils literal notranslate">PyArg_ParseTuple()</code>.)</li> <li>Argument Clinic removes some dynamism / flexibility. With <code class="docutils literal notranslate">PyArg_ParseTuple()</code> one could theoretically pass in different encodings at runtime for the “<code class="docutils literal notranslate">es</code>”/”<code class="docutils literal notranslate">et</code>” format units. AFAICT CPython doesn’t do this itself, however it’s possible external users might do this. (Trivia: there are no uses of “<code class="docutils literal notranslate">es</code>” exercised by regrtest, and all the uses of “<code class="docutils literal notranslate">et</code>” exercised are in socketmodule.c, except for one in _ssl.c. They’re all static, specifying the encoding <code class="docutils literal notranslate">"idna"</code>.)</li> </ul> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> The PEP author wishes to thank Ned Batchelder for permission to shamelessly rip off his clever design for Cog–“my favorite tool that I’ve never gotten to use”. Thanks also to everyone who provided feedback on the [bugtracker issue] and on python-dev. Special thanks to Alyssa (Nick) Coghlan and Guido van Rossum for a rousing two-hour in-person deep dive on the topic at PyCon US 2013. </section> <section id="references"> <h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2> <div role="list" class="citation-list"> <div class="citation" id="cog" role="doc-biblioentry"> <dt class="label" id="cog">[<a href="#id5">Cog</a>]</dt> <dd><code class="docutils literal notranslate">Cog</code>: <a class="reference external" href="http://nedbatchelder.com/code/cog/">http://nedbatchelder.com/code/cog/</a></div> </div> <aside class="footnote-list brackets"> <aside class="footnote brackets" id="id8" role="doc-footnote"> <dt class="label" id="id8">[<a href="#id1">1</a>]</dt> <dd><code class="docutils literal notranslate">PyArg_ParseTuple()</code>: <a class="reference external" href="http://docs.python.org/3/c-api/arg.html#PyArg_ParseTuple">http://docs.python.org/3/c-api/arg.html#PyArg_ParseTuple</a></aside> <aside class="footnote brackets" id="id9" role="doc-footnote"> <dt class="label" id="id9">[<a href="#id2">2</a>]</dt> <dd><code class="docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code>: <a class="reference external" href="http://docs.python.org/3/c-api/arg.html#PyArg_ParseTupleAndKeywords">http://docs.python.org/3/c-api/arg.html#PyArg_ParseTupleAndKeywords</a></aside> <aside class="footnote brackets" id="id10" role="doc-footnote"> <dt class="label" id="id10">[<a href="#id3">3</a>]</dt> <dd><code class="docutils literal notranslate">PyArg_</code> format units: <a class="reference external" href="http://docs.python.org/3/c-api/arg.html#strings-and-buffers">http://docs.python.org/3/c-api/arg.html#strings-and-buffers</a></aside> <aside class="footnote brackets" id="id11" role="doc-footnote"> <dt class="label" id="id11">[<a href="#id4">4</a>]</dt> <dd>Keyword parameters for extension functions: <a class="reference external" href="http://docs.python.org/3/extending/extending.html#keyword-parameters-for-extension-functions">http://docs.python.org/3/extending/extending.html#keyword-parameters-for-extension-functions</a></aside> <aside class="footnote brackets" id="id12" role="doc-footnote"> <dt class="label" id="id12">[<a href="#id6">5</a>]</dt> <dd>Guido van Rossum, posting to python-ideas, March 2012: <a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2012-March/014364.html">https://mail.python.org/pipermail/python-ideas/2012-March/014364.html</a> and <a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2012-March/014378.html">https://mail.python.org/pipermail/python-ideas/2012-March/014378.html</a> and <a class="reference external" href="https://mail.python.org/pipermail/python-ideas/2012-March/014417.html">https://mail.python.org/pipermail/python-ideas/2012-March/014417.html</a></aside> <aside class="footnote brackets" id="id13" role="doc-footnote"> <dt class="label" id="id13">[<a href="#id7">6</a>]</dt> <dd>Argument Clinic prototype: <a class="reference external" href="https://bitbucket.org/larry/python-clinic/">https://bitbucket.org/larry/python-clinic/</a></aside> </aside> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> This document has been placed in the public domain. </section> </section> <hr class="docutils" /> Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0436.rst">https://github.com/python/peps/blob/main/peps/pep-0436.rst</a> Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0436.rst">2025-02-01 08:59:27 GMT</a> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale-and-goals">Rationale and Goals</a></li> <li><a class="reference internal" href="#dsl-syntax-summary">DSL Syntax Summary</a><ul> <li><a class="reference internal" href="#general-behavior-of-the-argument-clinic-dsl">General Behavior Of the Argument Clinic DSL</a></li> <li><a class="reference internal" href="#module-and-class-declarations">Module and Class Declarations</a></li> <li><a class="reference internal" href="#function-declaration">Function Declaration</a></li> <li><a class="reference internal" href="#parameter-declaration">Parameter Declaration</a></li> <li><a class="reference internal" href="#legacy-converters">Legacy Converters</a></li> <li><a class="reference internal" href="#parameter-docstrings">Parameter Docstrings</a></li> <li><a class="reference internal" href="#special-syntax-for-parameter-lines">Special Syntax For Parameter Lines</a></li> <li><a class="reference internal" href="#function-docstring">Function Docstring</a></li> <li><a class="reference internal" href="#converters">Converters</a></li> <li><a class="reference internal" href="#return-converters">Return Converters</a></li> <li><a class="reference internal" href="#directives">Directives</a></li> </ul> </li> <li><a class="reference internal" href="#python-code">Python Code</a></li> <li><a class="reference internal" href="#output">Output</a></li> <li><a class="reference internal" href="#functions-with-positional-only-parameters">Functions With Positional-Only Parameters</a></li> <li><a class="reference internal" href="#current-status">Current Status</a><ul> <li><a class="reference internal" href="#argument-clinic-programmatic-interfaces">Argument Clinic Programmatic Interfaces</a></li> </ul> </li> <li><a class="reference internal" href="#notes-tbd">Notes / TBD</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> <li><a class="reference internal" href="#references">References</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0436.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>