CINXE.COM

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: "Paul E. McKenney": Re: [PATCH] Fix RCU race in access of nohz_cpu_mask</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by "Paul E. McKenney"" href="/groupie.php?aid=4246" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2005"> [2005]</a> 聽 <a class="nb" href="/lkml/2005/12"> [Dec]</a> 聽 <a class="nb" href="/lkml/2005/12/11"> [11]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2005/12/11/193" onclick="this.href='/lkml/headers'+'/2005/12/11/193';">[headers]</a>聽 <a href="/lkml/bounce/2005/12/11/193">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/12/8/155">First message in thread</a></li><li><a href="/lkml/2005/12/10/68">Srivatsa Vaddagiri</a><ul><li><a href="/lkml/2005/12/10/79">Oleg Nesterov</a><ul><li><a href="/lkml/2005/12/11/68">Srivatsa Vaddagiri</a><ul><li><a href="/lkml/2005/12/11/129">Andrew James Wade</a><ul><li><a href="/lkml/2005/12/11/153">Rusty Russell</a></li></ul></li></ul></li><li class="origin"><a href="/lkml/2005/12/11/208">"Paul E. McKenney"</a><ul><li><a href="/lkml/2005/12/11/208">Andrew Morton</a><ul><li><a href="/lkml/2005/12/11/211">"David S. Miller"</a></li><li><a href="/lkml/2005/12/11/213">Paul Mackerras</a></li><li><a href="/lkml/2005/12/12/8">Keith Owens</a></li></ul></li></ul></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Sun, 11 Dec 2005 19:10:53 -0800</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">"Paul E. McKenney" <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: [PATCH] Fix RCU race in access of nohz_cpu_mask</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">On Sat, Dec 10, 2005 at 09:55:35PM +0300, Oleg Nesterov wrote:<br />> Srivatsa Vaddagiri wrote:<br />> > <br />> > On Fri, Dec 09, 2005 at 10:17:38PM +0300, Oleg Nesterov wrote:<br />> > > > rcp->cur++; /* New GP */<br />> > > ><br />> > > > smp_mb();<br />> > ><br />> > > I think I need some education on memory barriers.<br />> > ><br />> > > Does this mb() garantees that the new value of ->cur will be visible<br />> > > on other cpus immediately after smp_mb() (so that rcu_pending() will<br />> > > notice it) ?<br />> > <br />> > AFAIK the new value of ->cur should be visible to other CPUs when smp_mb()<br />> > returns. Here's a definition of smp_mb() from Paul's article:<br />> > <br />> > smp_mb(): "memory barrier" that orders both loads and stores. This means loads<br />> > and stores preceding the memory barrier are committed to memory before any<br />> > loads and stores following the memory barrier.<br />> <br />> Thanks, but this definition talks about ordering, it does not say<br />> anything about the time when stores are really commited to memory<br />> (and does it mean also that cache-lines are invalidated on other<br />> cpus ?).<br /><br />Think of a pair of CPUs (creatively named "CPU 0" and "CPU 1") with<br />a cache-coherent interconnect between them. Then:<br /><br />1. wmb() guarantees that any writes preceding the wmb() will<br /> be seen by the interconnect before any writes following the<br /> wmb(). But this applies -only- to the writes executed by<br /> the CPU doing the wmb().<br /><br />2. rmb() guarantees that any changes seen by the interconnect<br /> preceding the rmb() will be seen by any reads following the<br /> rmb(). Again, this applies only to reads executed by the<br /> CPU doing the wmb(). However, the changes might be due to<br /> any CPU.<br /><br />3. mb() combines the guarantees made by rmb() and wmb().<br /><br />All CPUs but Alpha make stronger guarantees. On those CPUs, you can<br />replace "interconnect" in #1 above with "all CPUs", and you can replace<br />"seen by the interconnect" in #2 above with "caused by any CPU".<br />Rumor has it that more recent Alpha CPUs also have stronger memory<br />barriers, but I have thus far been unable to confirm this.<br /><br />On with the rest of the definitions:<br /><br />4. smp_wmb(), smp_rmb(), and smb_mb() do nothing in UP kernels,<br /> but do the corresponding memory barrier in SMP kernels.<br /><br />5. read_barrier_depends() is rmb() on Alpha, and nop on other<br /> CPUs. Non-Alpha CPUs guarantee that if CPU 0 does the<br /> following, where p->a is initially zero and global_pointer->a<br /> is initially 1:<br /><br /> p->a = 2;<br /> smp_wmb();<br /> global_pointer = p;<br /><br /> and if CPU 1 does:<br /><br /> x = global_pointer->a;<br /><br /> then the value of x will be either 1 or 2, never zero. On Alpha,<br /> strange though it may seem (and it did seem strange to me when I<br /> first encountered it!), the value of x could well be zero.<br /> To get the same guarantee on Alpha as on the other CPUs, the<br /> assignment to x must instead be as follows:<br /><br /> tmp = global_pointer;<br /> read_memory_depends();<br /> x = tmp->a;<br /><br /> or, equivalently:<br /><br /> x = rcu_dereference(global_pointer)->a;<br /><br /> There is an smp_read_barrier_depends() that takes effect only<br /> in an SMP kernel, similar to smp_wmb() and friends.<br /><br />This example may seem quite strange, but the need for the barrier<br />follows directly from the definition #1, which makes its ordering<br />guarantee only to the -interconnect-, -not- to the other CPUs.<br /><br />> > [ <a href="http://www.linuxjournal.com/article/8211">http://www.linuxjournal.com/article/8211</a> ]<br /><br />Hmm... That one does look familiar. ;-)<br /><br />> And thanks for this link, now I have some understanding about<br />> read_barrier_depends() ...<br /><br />Let's just say it required much patience and perseverence on the part<br />of the Alpha architects to explain it to me the first time around. ;-)<br /><br /> Thanx, Paul<br />-<br />To unsubscribe from this list: send the line "unsubscribe linux-kernel" in<br />the body of a message to majordomo@vger.kernel.org<br />More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a><br />Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a><br /><br /></pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2005-12-12 04:12 聽聽 [from the cache]<br />漏2003-2020 <a href="http://blog.jasper.es/"><span itemprop="editor">Jasper Spaans</span></a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>