CINXE.COM

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Paul Jackson: Re: Kernel SCM saga..</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Paul Jackson" href="/groupie.php?aid=5266" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2005"> [2005]</a> 聽 <a class="nb" href="/lkml/2005/4"> [Apr]</a> 聽 <a class="nb" href="/lkml/2005/4/10"> [10]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2005/4/10/87" onclick="this.href='/lkml/headers'+'/2005/4/10/87';">[headers]</a>聽 <a href="/lkml/bounce/2005/4/10/87">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/4/6/121">First message in thread</a></li><li><a href="/lkml/2005/4/9/149">Paul Jackson</a><ul><li><a href="/lkml/2005/4/10/45">Ingo Molnar</a><ul><li class="origin"><a href="/lkml/2005/4/10/90">Paul Jackson</a><ul><li><a href="/lkml/2005/4/10/90">Ingo Molnar</a><ul><li><a href="/lkml/2005/4/10/94">Paul Jackson</a></li></ul></li></ul></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Sun, 10 Apr 2005 10:38:05 -0700</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Paul Jackson <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: Kernel SCM saga..</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">Ingo wrote:<br />> With default gzip it's 3.3 seconds though,<br />> and that still compresses it down to 57 MB.<br /><br />Interesting. I'm surprised how much a bunch of separate, modest sized<br />files can be compressed.<br /><br />I'm unclear what matters most here.<br /><br />Space on disk certainly isn't much of an issue. Even with Andrew Morton<br />on our side, we still can't grow the kernel as fast as the disk drive<br />manufacturers can grow disk sizes.<br /><br />Main memory size of the compressed history matters to Linus and his top<br />20 lieutenants doing full kernel source patching as a primary mission if<br />they can't fit the source _history_ in main memory. But those people<br />are running 1 GByte or more of RAM - so whether it is 95, 57 or 45<br />MBytes, it fits fine. The rest of us are mostly concerned with whether<br />a kernel build fits in memory.<br /><br />Looking at an arch i386 kernel build tree I have at hand, I see the<br />following disk usage:<br /><br /> 102 MBytes - BitKeeper/*<br /> 287 MBytes - */SCCS/* (outside of already counted BitKeeper/*)<br /> 232 MBytes - checked out source files<br /> 94 MBytes - ELF and other build byproducts<br /> ---<br /> 715 MBytes - Total<br /><br />Converting from bk to git, I guess this becomes:<br /><br /> 97 MBytes - git (zlib)<br /> 232 MBytes - checked out source files<br /> 94 MBytes - ELF and other build byproducts<br /> ---<br /> 423 MBytes - Total<br /><br />Size matters when its a two to one difference, but when we are down to a<br />10% to 15% difference in the Total, its presentation that matters. The<br />above numbers tell me that this is not a pure size issue for local disk<br />or memory usage.<br /><br />What does matter that I can see:<br /><br /> 1) Linus explicitly stated he wanted "a raw zlib compressed blob,<br /> not a gzipped file", to encourage everyone to use the git tools to<br /> access this data. He did not "want people editing repostitory files<br /> by hand." I'm not sure what he gains here - it did annoy me for a<br /> couple hours before I decided fixing my supper was more important.<br /><br /> 2) The time to compress will be noticed by users as a delay when<br /> checking in changes (I'm guessing zlib compresses relatively faster).<br /><br /> 3) The time to copy compressed data over the internet will be<br /> noticed by users when upgrading kernel versions (gzip can<br /> compress smaller).<br /><br /> 4) Decompress times are smaller so don't matter as much.<br /><br /> 5) Zlib has a nice library, and is patent free. I don't know about gzip.<br /><br /> 6) As you note, zlib has rsync-friendly, recovery-friendly Z_PARTIAL_FLUSH.<br /> I don't know about gzip.<br /><br />My guess is that Linus finds (2) and (3) to balance each other, and that<br />(1) decides the point, in favor of zlib. Well, that or a simpler<br />hypothesis, that he found the nice library (5) convenient, and (1)<br />sealed the deal, with the other tradeoffs passing through his<br />subconscious faster than he bothered to verbalize them.<br /><br />You (Ingo) seem in your second message to be encouraging further<br />consideration of gzip, for its improved compression.<br /><br />How will that matter to us, day to day?<br /><br />-- <br /> I won't rest till it's the best ...<br /> Programmer, Linux Scalability<br /> Paul Jackson <pj@engr.sgi.com> 1.650.933.1373, 1.925.600.0401<br />-<br />To unsubscribe from this list: send the line "unsubscribe linux-kernel" in<br />the body of a message to majordomo@vger.kernel.org<br />More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a><br />Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a><br /><br /></pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2005-04-10 19:46 聽聽 [from the cache]<br />漏2003-2020 <a href="http://blog.jasper.es/"><span itemprop="editor">Jasper Spaans</span></a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>