CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 256 – Docstring Processing System Framework | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0256/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 256 – Docstring Processing System Framework | peps.python.org'> <meta property="og:description" content="Python lends itself to inline documentation. With its built-in docstring syntax, a limited form of Literate Programming is easy to do in Python. However, there are no satisfactory standard tools for extracting and processing Python docstrings. The la..."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0256/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="Python lends itself to inline documentation. With its built-in docstring syntax, a limited form of Literate Programming is easy to do in Python. However, there are no satisfactory standard tools for extracting and processing Python docstrings. The la..."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 256</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 256 – Docstring Processing System Framework</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">David Goodger <goodger at python.org></dd> <dt class="field-even">Discussions-To<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/mailman/listinfo/doc-sig">Doc-SIG list</a></dd> <dt class="field-odd">Status<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Formally declined and will not be accepted">Rejected</abbr></dd> <dt class="field-even">Type<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd> <dt class="field-odd">Created<span class="colon">:</span></dt> <dd class="field-odd">01-Jun-2001</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">13-Jun-2001</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#rejection-notice">Rejection Notice</a></li> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#road-map-to-the-docstring-peps">Road Map to the Docstring PEPs</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#pydoc-other-existing-systems">PyDoc & Other Existing Systems</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#project-web-site">Project Web Site</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> </ul> </details></section> <section id="rejection-notice"> <h2><a class="toc-backref" href="#rejection-notice" role="doc-backlink">Rejection Notice</a></h2> <p>This proposal seems to have run out of steam.</p> </section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>Python lends itself to inline documentation. With its built-in docstring syntax, a limited form of <a class="reference external" href="http://www.literateprogramming.com/">Literate Programming</a> is easy to do in Python. However, there are no satisfactory standard tools for extracting and processing Python docstrings. The lack of a standard toolset is a significant gap in Python’s infrastructure; this PEP aims to fill the gap.</p> <p>The issues surrounding docstring processing have been contentious and difficult to resolve. This PEP proposes a generic Docstring Processing System (DPS) framework, which separates out the components (program and conceptual), enabling the resolution of individual issues either through consensus (one solution) or through divergence (many). It promotes standard interfaces which will allow a variety of plug-in components (input context readers, markup parsers, and output format writers) to be used.</p> <p>The concepts of a DPS framework are presented independently of implementation details.</p> </section> <section id="road-map-to-the-docstring-peps"> <h2><a class="toc-backref" href="#road-map-to-the-docstring-peps" role="doc-backlink">Road Map to the Docstring PEPs</a></h2> <p>There are many aspects to docstring processing. The “Docstring PEPs” have broken up the issues in order to deal with each of them in isolation, or as close as possible. The individual aspects and associated PEPs are as follows:</p> <ul class="simple"> <li>Docstring syntax. <a class="pep reference internal" href="../pep-0287/" title="PEP 287 – reStructuredText Docstring Format">PEP 287</a>, “reStructuredText Docstring Format”, proposes a syntax for Python docstrings, PEPs, and other uses.</li> <li>Docstring semantics consist of at least two aspects:<ul> <li>Conventions: the high-level structure of docstrings. Dealt with in <a class="pep reference internal" href="../pep-0257/" title="PEP 257 – Docstring Conventions">PEP 257</a>, “Docstring Conventions”.</li> <li>Methodology: rules for the informational content of docstrings. Not addressed.</li> </ul> </li> <li>Processing mechanisms. This PEP (<a class="pep reference internal" href="../pep-0256/" title="PEP 256 – Docstring Processing System Framework">PEP 256</a>) outlines the high-level issues and specification of an abstract docstring processing system (DPS). <a class="pep reference internal" href="../pep-0258/" title="PEP 258 – Docutils Design Specification">PEP 258</a>, “Docutils Design Specification”, is an overview of the design and implementation of one DPS under development.</li> <li>Output styles: developers want the documentation generated from their source code to look good, and there are many different ideas about what that means. <a class="pep reference internal" href="../pep-0258/" title="PEP 258 – Docutils Design Specification">PEP 258</a> touches on “Stylist Transforms”. This aspect of docstring processing has yet to be fully explored.</li> </ul> <p>By separating out the issues, we can form consensus more easily (smaller fights ;-), and accept divergence more readily.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>There are standard inline documentation systems for some other languages. For example, Perl has <a class="reference external" href="http://www.perldoc.com/perl5.6/pod/perlpod.html">POD</a> (“Plain Old Documentation”) and Java has <a class="reference external" href="http://java.sun.com/j2se/javadoc/">Javadoc</a>, but neither of these mesh with the Pythonic way. POD syntax is very explicit, but takes after Perl in terms of readability. Javadoc is HTML-centric; except for “<code class="docutils literal notranslate"><span class="pre">@field</span></code>” tags, raw HTML is used for markup. There are also general tools such as <a class="reference external" href="http://www.helpmaster.com/hlp-developmentaids-autoduck.htm">Autoduck</a> and <a class="reference external" href="http://www-cs-faculty.stanford.edu/~knuth/cweb.html">Web</a> (Tangle & Weave), useful for multiple languages.</p> <p>There have been many attempts to write auto-documentation systems for Python (not an exhaustive list):</p> <ul class="simple"> <li>Marc-Andre Lemburg’s <a class="reference external" href="http://www.egenix.com/files/python/SoftwareDescriptions.html#doc.py">doc.py</a></li> <li>Daniel Larsson’s <a class="reference external" href="http://starship.python.net/crew/danilo/pythondoc/">pythondoc</a> & <a class="reference external" href="http://starship.python.net/crew/danilo/pythondoc/">gendoc</a></li> <li>Doug Hellmann’s <a class="reference external" href="http://happydoc.sourceforge.net/">HappyDoc</a></li> <li>Laurence Tratt’s Crystal (no longer available on the web)</li> <li>Ka-Ping Yee’s <a class="reference external" href="http://docs.python.org/library/pydoc.html">pydoc</a> (pydoc.py is now part of the Python standard library; see below)</li> <li>Tony Ibbs’ <a class="reference external" href="http://www.tibsnjoan.co.uk/docutils.html">docutils</a> (Tony has donated this name to the <a class="reference external" href="http://docutils.sourceforge.net/">Docutils project</a>)</li> <li>Edward Loper’s <a class="reference external" href="http://www.cis.upenn.edu/~edloper/pydoc/">STminus</a> formalization and related efforts</li> </ul> <p>These systems, each with different goals, have had varying degrees of success. A problem with many of the above systems was over-ambition combined with inflexibility. They provided a self-contained set of components: a docstring extraction system, a markup parser, an internal processing system and one or more output format writers with a fixed style. Inevitably, one or more aspects of each system had serious shortcomings, and they were not easily extended or modified, preventing them from being adopted as standard tools.</p> <p>It has become clear (to this author, at least) that the “all or nothing” approach cannot succeed, since no monolithic self-contained system could possibly be agreed upon by all interested parties. A modular component approach designed for extension, where components may be multiply implemented, may be the only chance for success. Standard inter-component APIs will make the DPS components comprehensible without requiring detailed knowledge of the whole, lowering the barrier for contributions, and ultimately resulting in a rich and varied system.</p> <p>Each of the components of a docstring processing system should be developed independently. A “best of breed” system should be chosen, either merged from existing systems, and/or developed anew. This system should be included in Python’s standard library.</p> <section id="pydoc-other-existing-systems"> <h3><a class="toc-backref" href="#pydoc-other-existing-systems" role="doc-backlink">PyDoc & Other Existing Systems</a></h3> <p>PyDoc became part of the Python standard library as of release 2.1. It extracts and displays docstrings from within the Python interactive interpreter, from the shell command line, and from a GUI window into a web browser (HTML). Although a very useful tool, PyDoc has several deficiencies, including:</p> <ul class="simple"> <li>In the case of the GUI/HTML, except for some heuristic hyperlinking of identifier names, no formatting of the docstrings is done. They are presented within <code class="docutils literal notranslate"><span class="pre"><p><small><tt></span></code> tags to avoid unwanted line wrapping. Unfortunately, the result is not attractive.</li> <li>PyDoc extracts docstrings and structural information (class identifiers, method signatures, etc.) from imported module objects. There are security issues involved with importing untrusted code. Also, information from the source is lost when importing, such as comments, “additional docstrings” (string literals in non-docstring contexts; see <a class="pep reference internal" href="../pep-0258/" title="PEP 258 – Docutils Design Specification">PEP 258</a>), and the order of definitions.</li> </ul> <p>The functionality proposed in this PEP could be added to or used by PyDoc when serving HTML pages. The proposed docstring processing system’s functionality is much more than PyDoc needs in its current form. Either an independent tool will be developed (which PyDoc may or may not use), or PyDoc could be expanded to encompass this functionality and <em>become</em> the docstring processing system (or one such system). That decision is beyond the scope of this PEP.</p> <p>Similarly for other existing docstring processing systems, their authors may or may not choose compatibility with this framework. However, if this framework is accepted and adopted as the Python standard, compatibility will become an important consideration in these systems’ future.</p> </section> </section> <section id="specification"> <h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2> <p>The docstring processing system framework is broken up as follows:</p> <ol class="arabic"> <li>Docstring conventions. Documents issues such as:<ul class="simple"> <li>What should be documented where.</li> <li>First line is a one-line synopsis.</li> </ul> <p><a class="pep reference internal" href="../pep-0257/" title="PEP 257 – Docstring Conventions">PEP 257</a> documents some of these issues.</p> </li> <li>Docstring processing system design specification. Documents issues such as:<ul class="simple"> <li>High-level spec: what a DPS does.</li> <li>Command-line interface for executable script.</li> <li>System Python API.</li> <li>Docstring extraction rules.</li> <li>Readers, which encapsulate the input context.</li> <li>Parsers.</li> <li>Document tree: the intermediate internal data structure. The output of the Parser and Reader, and the input to the Writer all share the same data structure.</li> <li>Transforms, which modify the document tree.</li> <li>Writers for output formats.</li> <li>Distributors, which handle output management (one file, many files, or objects in memory).</li> </ul> <p>These issues are applicable to any docstring processing system implementation. <a class="pep reference internal" href="../pep-0258/" title="PEP 258 – Docutils Design Specification">PEP 258</a> documents these issues.</p> </li> <li>Docstring processing system implementation.</li> <li>Input markup specifications: docstring syntax. <a class="pep reference internal" href="../pep-0287/" title="PEP 287 – reStructuredText Docstring Format">PEP 287</a> proposes a standard syntax.</li> <li>Input parser implementations.</li> <li>Input context readers (“modes”: Python source code, PEP, standalone text file, email, etc.) and implementations.</li> <li>Stylists: certain input context readers may have associated stylists which allow for a variety of output document styles.</li> <li>Output formats (HTML, XML, TeX, DocBook, info, etc.) and writer implementations.</li> </ol> <p>Components 1, 2/3/5, and 4 are the subject of individual companion PEPs. If there is another implementation of the framework or syntax/parser, additional PEPs may be required. Multiple implementations of each of components 6 and 7 will be required; the PEP mechanism may be overkill for these components.</p> </section> <section id="project-web-site"> <h2><a class="toc-backref" href="#project-web-site" role="doc-backlink">Project Web Site</a></h2> <p>A SourceForge project has been set up for this work at <a class="reference external" href="http://docutils.sourceforge.net/">http://docutils.sourceforge.net/</a>.</p> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> <section id="acknowledgements"> <h2><a class="toc-backref" href="#acknowledgements" role="doc-backlink">Acknowledgements</a></h2> <p>This document borrows ideas from the archives of the <a class="reference external" href="http://www.python.org/sigs/doc-sig/">Python Doc-SIG</a>. Thanks to all members past & present.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0256.rst">https://github.com/python/peps/blob/main/peps/pep-0256.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0256.rst">2025-02-01 08:59:27 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#rejection-notice">Rejection Notice</a></li> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#road-map-to-the-docstring-peps">Road Map to the Docstring PEPs</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#pydoc-other-existing-systems">PyDoc & Other Existing Systems</a></li> </ul> </li> <li><a class="reference internal" href="#specification">Specification</a></li> <li><a class="reference internal" href="#project-web-site">Project Web Site</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> <li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0256.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>