CINXE.COM

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Jamie Lokier: Re: filesystem transactions API</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Jamie Lokier" href="/groupie.php?aid=24792" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2005"> [2005]</a> 聽 <a class="nb" href="/lkml/2005/4"> [Apr]</a> 聽 <a class="nb" href="/lkml/2005/4/27"> [27]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2005/4/27/126" onclick="this.href='/lkml/headers'+'/2005/4/27/126';">[headers]</a>聽 <a href="/lkml/bounce/2005/4/27/126">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/4/24/76">First message in thread</a></li><li><a href="/lkml/2005/4/27/46">Jan Hudec</a><ul><li><a href="/lkml/2005/4/27/94">Ville Herva</a><ul><li class="origin"><a href="/lkml/2005/4/27/150">Jamie Lokier</a><ul><li><a href="/lkml/2005/4/27/150">Bernd Eckenfels</a></li></ul></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Wed, 27 Apr 2005 16:17:58 +0100</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Jamie Lokier <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: filesystem transactions API</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">Ville Herva wrote: > > How do we specify which calls belong to a transaction? By some kind of > > extra file handle? > > > > I'd think having global per-process transaction is not the best way. > > So I think we should have some kind of transaction handle (probably in > > the file handle space) and a way to say that a syscall is done within > > a transaction. To avoid duplicating all syscalls, we could have > > set_active_transaction() operation. > > That's more or less what NTFS does. See the example at > <a href="http://blogs.msdn.com/because_we_can/">http://blogs.msdn.com/because_we_can/</a> That's the obvious choice but it limits the usefulness quite a lot. If we have transactions, then I'd like to be able to do this from a shell: transaction_open t tar xvpSfz blahblah.tar.gz cd blahblah patch -p1 -E < foo.patch # etc. transaction_close $t I'd also like to write inside a single C program: transaction * t = transaction_open (); /* Ordinary complicated filesystem operations here... */ link (a, b); rename (c, d); read, write, stat etc. conf = open ("/etc/blahblah.conf", O_RDONLY); read (conf, ...) close (conf); /* If /etc/blahblah.conf is changed by another program during the transaction, the transaction is invalidated, because the dbm update below is dependent on what was read... */ dbm_open (...); do_dbm_stuff (...); dbm_close (...); /* Whatever this command does, I'd like to include in the transaction. */ system ("perl -pi -e 's/old_value/new_value/g' /etc/another.conf"); transaction_close (t); Fundamentally, if transactions are supported in the kernel then these two usages are easy to offer: 1. Ordinary file system calls as part of a transaction. This allows libraries which are not transaction-aware to be used, such as the dbm example above, and other things like XML parsers/writers. 2. Subprocesses inherit a transaction, so a program can execute complex transactions by using other programs. It's useful, and there is no good reason to disallow that. Nonetheless, there's a need for some kind of transaction handles. A file descriptor representing a transaction seems like a natural fit. Complex programs will want to have multiple transactions at the same time: For example, any program structured using event-driven logic or async I/O may have multiple independent state machines per thread, each wanting to be able to have their own transactions. This suggests a few things: - Transactions have a file descriptor to represent them. - Each thread has a "current transaction" that applies to all filesystem operations. - Concurrent threads will need their own current transactions, even while keeping "current directory" global to the whole process for POSIX reasons. A process wide "current transaction" is too coarse. - Transactions should be automatically nestable: a program or library which uses transactions should itself be callable from a program or library which is using a transaction. - Transactions should record whether they cannot provide transactions for some operation that is attempted (e.g. writing to a file on a remote filesystem), aborting the transaction. - When a transaction aborts due to the actions of _another_ process (or thread) which is outside the transaction, that abort is an event which should be detectable synchronously (by polling the transaction fd) or asynchronously (by a signal - the SIGIO mechanism is fine for this). - An exclusive locking period should be optional, requested by a flag when opening the transaction. Most usages will want the locking period with its default parameters. - Ideally, programs or mechanisms which provide alternative views of part of a filesystem, such as search results (Beagle), tarfs, or mailfs, should be able to update synchronously with transactions that affect whatever the view is watching, so that the view changes are effectively part of the transaction. This does _not_ mean that a transaction must wait for watchers to calculate anything. It does mean a transaction must synchronously and simultaneously invalidate caches held by watchers during the atomic commit. -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a> Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a> </pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2005-04-27 17:21 聽聽 [from the cache] 漏2003-2020 <a href="http://blog.jasper.es/">Jasper Spaans</a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>