CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="color-scheme" content="light dark"> <title>PEP 347 – Migrating the Python CVS to Subversion | peps.python.org</title> <link rel="shortcut icon" href="../_static/py.png"> <link rel="canonical" href="https://peps.python.org/pep-0347/"> <link rel="stylesheet" href="../_static/style.css" type="text/css"> <link rel="stylesheet" href="../_static/mq.css" type="text/css"> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light"> <link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark"> <link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss"> <meta property="og:title" content='PEP 347 – Migrating the Python CVS to Subversion | peps.python.org'> <meta property="og:description" content="The Python source code is currently managed in a CVS repository on sourceforge.net. This PEP proposes to move it to a Subversion repository on svn.python.org."> <meta property="og:type" content="website"> <meta property="og:url" content="https://peps.python.org/pep-0347/"> <meta property="og:site_name" content="Python Enhancement Proposals (PEPs)"> <meta property="og:image" content="https://peps.python.org/_static/og-image.png"> <meta property="og:image:alt" content="Python PEPs"> <meta property="og:image:width" content="200"> <meta property="og:image:height" content="200"> <meta name="description" content="The Python source code is currently managed in a CVS repository on sourceforge.net. This PEP proposes to move it to a Subversion repository on svn.python.org."> <meta name="theme-color" content="#3776ab"> </head> <body> <svg xmlns="http://www.w3.org/2000/svg" style="display: none;"> <symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all"> <title>Following system colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="9"></circle> <path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path> </svg> </symbol> <symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all"> <title>Selected dark colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path stroke="none" d="M0 0h24v24H0z" fill="none"></path> <path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path> </svg> </symbol> <symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all"> <title>Selected light colour scheme</title> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="12" r="5"></circle> <line x1="12" y1="1" x2="12" y2="3"></line> <line x1="12" y1="21" x2="12" y2="23"></line> <line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line> <line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line> <line x1="1" y1="12" x2="3" y2="12"></line> <line x1="21" y1="12" x2="23" y2="12"></line> <line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line> <line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line> </svg> </symbol> </svg> <script> document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto" </script> <section id="pep-page-section"> <header> <h1>Python Enhancement Proposals</h1> <ul class="breadcrumbs"> <li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> » </li> <li><a href="../pep-0000/">PEP Index</a> » </li> <li>PEP 347</li> </ul> <button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())"> <svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg> <svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg> <span class="visually-hidden">Toggle light / dark / auto colour theme</span> </button> </header> <article> <section id="pep-content"> <h1 class="page-title">PEP 347 – Migrating the Python CVS to Subversion</h1> <dl class="rfc2822 field-list simple"> <dt class="field-odd">Author<span class="colon">:</span></dt> <dd class="field-odd">Martin von Löwis <martin at v.loewis.de></dd> <dt class="field-even">Discussions-To<span class="colon">:</span></dt> <dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-dev@python.org/">Python-Dev list</a></dd> <dt class="field-odd">Status<span class="colon">:</span></dt> <dd class="field-odd"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd> <dt class="field-even">Type<span class="colon">:</span></dt> <dd class="field-even"><abbr title="Normative PEP describing or proposing a change to a Python community process, workflow or governance">Process</abbr></dd> <dt class="field-odd">Created<span class="colon">:</span></dt> <dd class="field-odd">14-Jul-2004</dd> <dt class="field-even">Post-History<span class="colon">:</span></dt> <dd class="field-even">14-Jul-2004</dd> </dl> <hr class="docutils" /> <section id="contents"> <details><summary>Table of Contents</summary><ul class="simple"> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#moving-to-subversion">Moving to Subversion</a></li> <li><a class="reference internal" href="#moving-to-python-org">Moving to python.org</a></li> </ul> </li> <li><a class="reference internal" href="#migration-procedure">Migration Procedure</a><ul> <li><a class="reference internal" href="#collect-ssh-keys">Collect SSH keys</a></li> <li><a class="reference internal" href="#administrator-access">Administrator Access</a></li> <li><a class="reference internal" href="#downloading-the-cvs-repository">Downloading the CVS Repository</a></li> <li><a class="reference internal" href="#converting-the-cvs-repository">Converting the CVS Repository</a></li> <li><a class="reference internal" href="#publish-the-repository">Publish the Repository</a></li> <li><a class="reference internal" href="#disable-cvs">Disable CVS</a></li> </ul> </li> <li><a class="reference internal" href="#discussion">Discussion</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> </details></section> <section id="abstract"> <h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2> <p>The Python source code is currently managed in a CVS repository on sourceforge.net. This PEP proposes to move it to a Subversion repository on svn.python.org.</p> </section> <section id="rationale"> <h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2> <p>This change has two aspects: moving from CVS to Subversion, and moving from SourceForge to python.org. For each, a rationale will be given.</p> <section id="moving-to-subversion"> <h3><a class="toc-backref" href="#moving-to-subversion" role="doc-backlink">Moving to Subversion</a></h3> <p>CVS has a number of limitations that have been eliminated by Subversion. For the development of Python, the most notable improvements are:</p> <ul class="simple"> <li>the ability to rename files and directories, and to remove directories, while keeping the history of these files.</li> <li>support for change sets (sets of correlated changes to multiple files) through global revision numbers. Change sets are transactional.</li> <li>atomic, fast tagging: a cvs tag might take many minutes; a Subversion tag (svn cp) will complete quickly, and atomically. Likewise, branches are very efficient.</li> <li>support for offline diffs, which is useful when creating patches.</li> </ul> </section> <section id="moving-to-python-org"> <h3><a class="toc-backref" href="#moving-to-python-org" role="doc-backlink">Moving to python.org</a></h3> <p>SourceForge has kindly provided an important infrastructure for the past years. Unfortunately, the attention that SF received has also caused repeated overload situations in the past, to which the SF operators could not always respond in a timely manner. In particular, for CVS, they had to reduce the load on the primary CVS server by introducing a second, read-only CVS server for anonymous access. This server is regularly synchronized, but lags behind the read-write CVS repository between synchronizations. As a result, users without commit access can see recent changes to the repository only after a delay.</p> <p>On python.org, it would be possible to make the repository accessible for anonymous access.</p> </section> </section> <section id="migration-procedure"> <h2><a class="toc-backref" href="#migration-procedure" role="doc-backlink">Migration Procedure</a></h2> <p>To move the Python CVS repository, the following steps need to be executed. The steps are elaborated upon in the following sections.</p> <ol class="arabic simple"> <li>Collect SSH keys for all current committers, along with usernames to appear in commit messages.</li> <li>At the beginning of the migration, announce that the repository on SourceForge closed.</li> <li>24 hours after the last commit, download the CVS repository.</li> <li>Convert the CVS repository into a Subversion repository.</li> <li>Publish the repository with write access for committers, and read-only anonymous access.</li> <li>Disable CVS access on SF.</li> </ol> <section id="collect-ssh-keys"> <h3><a class="toc-backref" href="#collect-ssh-keys" role="doc-backlink">Collect SSH keys</a></h3> <p>After some discussion, svn+ssh was selected as the best method for write access to the repository. Developers can continue to use their SSH keys, but they must be installed on python.org.</p> <p>In order to avoid having to create a new Unix user for each developer, a single account should be used, with command= attributes in the authorized_keys files.</p> <p>The lines in the authorized_keys file should read like this (wrapped for better readability):</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">command</span><span class="o">=</span><span class="s2">"/usr/bin/svnserve --root=/svnroot -t</span> <span class="o">--</span><span class="n">tunnel</span><span class="o">-</span><span class="n">user</span><span class="o">=</span><span class="s1">'<username>'</span><span class="s2">",no-port-forwarding,</span> <span class="n">no</span><span class="o">-</span><span class="n">X11</span><span class="o">-</span><span class="n">forwarding</span><span class="p">,</span><span class="n">no</span><span class="o">-</span><span class="n">agent</span><span class="o">-</span><span class="n">forwarding</span><span class="p">,</span><span class="n">no</span><span class="o">-</span><span class="n">pty</span> <span class="n">ssh</span><span class="o">-</span><span class="n">dss</span> <span class="o"><</span><span class="n">key</span><span class="o">></span> <span class="o"><</span><span class="n">comment</span><span class="o">></span> </pre></div> </div> <p>As the usernames, the real names should be used instead of the SF account names, so that people can be better identified in log messages.</p> </section> <section id="administrator-access"> <h3><a class="toc-backref" href="#administrator-access" role="doc-backlink">Administrator Access</a></h3> <p>Administrator access to the pythondev account should be granted to all current admins of the Python SF project. To distinguish between shell login and svnserve login, admins need to maintain two keys. Using OpenSSH, the following procedure can be used to create a second key:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="o">.</span><span class="n">ssh</span> <span class="n">ssh</span><span class="o">-</span><span class="n">keygen</span> <span class="o">-</span><span class="n">t</span> <span class="n">DSA</span> <span class="o">-</span><span class="n">f</span> <span class="n">pythondev</span> <span class="o">-</span><span class="n">C</span> <span class="o"><</span><span class="n">user</span><span class="o">></span><span class="nd">@pythondev</span> <span class="n">vi</span> <span class="n">config</span> </pre></div> </div> <p>In the config file, the following lines need to be added:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Host</span> <span class="n">pythondev</span> <span class="n">Hostname</span> <span class="n">dinsdale</span><span class="o">.</span><span class="n">python</span><span class="o">.</span><span class="n">org</span> <span class="n">User</span> <span class="n">pythondev</span> <span class="n">IdentityFile</span> <span class="o">~/.</span><span class="n">ssh</span><span class="o">/</span><span class="n">pythondev</span> </pre></div> </div> <p>Then, shell login becomes possible through “ssh pythondev”.</p> </section> <section id="downloading-the-cvs-repository"> <h3><a class="toc-backref" href="#downloading-the-cvs-repository" role="doc-backlink">Downloading the CVS Repository</a></h3> <p>The CVS repository can be downloaded from</p> <blockquote> <div><a class="reference external" href="http://cvs.sourceforge.net/cvstarballs/python-cvsroot.tar.bz2">http://cvs.sourceforge.net/cvstarballs/python-cvsroot.tar.bz2</a></div></blockquote> <p>Since this tarball is generated only once a day, some time must pass after the repository freeze before the tarball can be picked up. It should be verified that the last commit, as recorded on the python-commits mailing list, is indeed included in the tarball.</p> <p>After the conversion, the converted CVS tarball should be kept forever on www.python.org/archive/python-cvsroot-<date>.tar.bz2</p> </section> <section id="converting-the-cvs-repository"> <h3><a class="toc-backref" href="#converting-the-cvs-repository" role="doc-backlink">Converting the CVS Repository</a></h3> <p>The Python CVS repository contains two modules: distutils and python. The python module is further structured into dist and nondist, where dist only contains src (the python code proper). nondist contains various subdirectories.</p> <p>These should be reorganized in the Subversion repository to get shorter URLs, following the <project>/{trunk,tags,branches} structure. A project will be created for each nondist directory, plus for src (called python), plus distutils. Reorganizing the repository is best done in the CVS tree, as shown below.</p> <p>The fsfs backend should be used as the repository format (which requires Subversion 1.1). The fsfs backend has the advantage of being more backup-friendly, as it allows incremental repository backups, without requiring any dump commands to be run.</p> <p>The conversion should be done using the cvs2svn utility, available e.g. in the cvs2svn Debian package. As cvs2svn does not currently support the project/trunk structure, each project needs to be converted separately. To get each conversion result into a separate directory in the target repository, svnadmin load must be used.</p> <p>Subversion has a different view on binary-vs-text files than CVS. To correctly carry the CVS semantics forward, svn:eol-style should be set to native on all files that are not marked binary in the CVS.</p> <p>In summary, the conversion script is:</p> <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>#!/bin/sh rm cvs2svn-* rm -rf python py.new tar xjf python-cvsroot.tar.bz2 rm -rf python/CVSROOT svnadmin create --fs-type fsfs py.new mv python/python python/orig mv python/orig/dist/src python/python mv python/orig/nondist/* python # nondist/nondist is empty rmdir python/nondist rm -rf python/orig for a in python/* do b=`basename $a` cvs2svn -q --dump-only --encoding=latin1 --force-branch=cnri-16-start \ --force-branch=descr-branch --force-branch=release152p1-patches \ --force-tag=r16b1 $a svn mkdir -m"Conversion to SVN" file:///`pwd`/py.new/$b svnadmin load -q --parent-dir $b py.new < cvs2svn-dump rm cvs2svn-dump done </pre></div> </div> <p>Sample results of this conversion are available at</p> <blockquote> <div><a class="reference external" href="http://www.dcl.hpi.uni-potsdam.de/pysvn/">http://www.dcl.hpi.uni-potsdam.de/pysvn/</a></div></blockquote> </section> <section id="publish-the-repository"> <h3><a class="toc-backref" href="#publish-the-repository" role="doc-backlink">Publish the Repository</a></h3> <p>The repository should be published at <a class="reference external" href="http://svn.python.org/projects">http://svn.python.org/projects</a>. Read-write access should be granted to all current SF committers through svn+ssh://pythondev@svn.python.org/; read-only anonymous access through WebDAV should also be granted.</p> <p>As an option, websvn (available e.g. from the Debian websvn package) could be provided. Unfortunately, in the test installation, websvn breaks because it runs out of memory.</p> <p>The current SF project admins should get write access to the authorized_keys2 file of the pythondev account.</p> </section> <section id="disable-cvs"> <h3><a class="toc-backref" href="#disable-cvs" role="doc-backlink">Disable CVS</a></h3> <p>It appears that CVS cannot be disabled entirely. Only the user interface can be removed from the project page; the repository itself remains available. If desired, write access to the python and distutils modules can be disabled through a CVS commitinfo entry.</p> </section> </section> <section id="discussion"> <h2><a class="toc-backref" href="#discussion" role="doc-backlink">Discussion</a></h2> <p>Several alternatives had been suggested to the procedure above. The rejected alternatives are shortly discussed here:</p> <ul> <li>create multiple repositories, one for python and one for distutils. This would have allowed even shorter URLs, but was rejected because a single repository supports moving code across projects.</li> <li>Several people suggested to create the project/trunk structure through standard cvs2svn, followed by renames. This would have the disadvantage that old revisions use different path names than recent revisions; the suggested approach through dump files works without renames.</li> <li>Several people also expressed concern about the administrative overhead that hosting the repository on python.org would cause to pydotorg admins. As a specific alternative, BerliOS has been suggested. The pydotorg admins themselves haven't objected to the additional workload; migrating the repository again if they get overworked is an option.</li> <li>Different authentication strategies were discussed. As alternatives to svn+ssh were suggested<ul class="simple"> <li>Subversion over WebDAV, using SSL and basic authentication, with pydotorg-generated passwords mailed to the user. People did not like that approach, since they would need to store the password on disk (because they can’t remember it); this is a security risk.</li> <li>Subversion over WebDAV, using SSL client certificates. This would work, but would require us to administer a certificate authority.</li> </ul> </li> <li>Instead of hosting this on python.org, people suggested hosting it elsewhere. One issue is whether this alternative should be free or commercial; several people suggested it should better be commercial, to reduce the load on the volunteers. In particular:<ul> <li>Greg Stein suggested <a class="reference external" href="http://www.wush.net/subversion.php">http://www.wush.net/subversion.php</a>. They offer 5 GB for $90/month, with 200 GB download/month. The data is on a RAID drive and fully backed up. Anonymous access and email commit notifications are supported. wush.net elaborated the following details:<ul class="simple"> <li>The machine would be a Virtuozzo Virtual Private Server (VPS), hosted at PowerVPS.</li> <li>The default repository URL would be <a class="reference external" href="http://python.wush.net/svn/projectname/">http://python.wush.net/svn/projectname/</a>, but anything else could be arranged</li> <li>we would get SSH login to the machine, with sudo capabilities.</li> <li>They have a Web interface for management of the various SVN repositories that we want to host, and to manage user accounts. While svn+ssh would be supported, the user interface does not yet support it.</li> <li>For offsite mirroring/backup, they suggest to use rsync instead of download of repository tarballs.</li> </ul> <p>Bob Ippolito reported that they had used wush.net for a commercial project for about 6 months, after which time they left wush.net, because the service was down for three days, with nobody reachable, and no explanation when it came back.</p> </li> </ul> </li> </ul> </section> <section id="copyright"> <h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2> <p>This document has been placed in the public domain.</p> </section> </section> <hr class="docutils" /> <p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0347.rst">https://github.com/python/peps/blob/main/peps/pep-0347.rst</a></p> <p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0347.rst">2025-02-01 08:59:27 GMT</a></p> </article> <nav id="pep-sidebar"> <h2>Contents</h2> <ul> <li><a class="reference internal" href="#abstract">Abstract</a></li> <li><a class="reference internal" href="#rationale">Rationale</a><ul> <li><a class="reference internal" href="#moving-to-subversion">Moving to Subversion</a></li> <li><a class="reference internal" href="#moving-to-python-org">Moving to python.org</a></li> </ul> </li> <li><a class="reference internal" href="#migration-procedure">Migration Procedure</a><ul> <li><a class="reference internal" href="#collect-ssh-keys">Collect SSH keys</a></li> <li><a class="reference internal" href="#administrator-access">Administrator Access</a></li> <li><a class="reference internal" href="#downloading-the-cvs-repository">Downloading the CVS Repository</a></li> <li><a class="reference internal" href="#converting-the-cvs-repository">Converting the CVS Repository</a></li> <li><a class="reference internal" href="#publish-the-repository">Publish the Repository</a></li> <li><a class="reference internal" href="#disable-cvs">Disable CVS</a></li> </ul> </li> <li><a class="reference internal" href="#discussion">Discussion</a></li> <li><a class="reference internal" href="#copyright">Copyright</a></li> </ul> <br> <a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0347.rst">Page Source (GitHub)</a> </nav> </section> <script src="../_static/colour_scheme.js"></script> <script src="../_static/wrap_tables.js"></script> <script src="../_static/sticky_banner.js"></script> </body> </html>