CINXE.COM

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Srivatsa Vaddagiri: Semantics of smp_mb() [was : Re: [PATCH] Fix RCU race in access of nohz_cpu_mask ]</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Srivatsa Vaddagiri" href="/groupie.php?aid=17382" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32"> </td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> <a class="nb" href="/lkml/2005"> [2005]</a> <a class="nb" href="/lkml/2005/12"> [Dec]</a> <a class="nb" href="/lkml/2005/12/11"> [11]</a> <a class="nb" href="/lkml/last100"> [last100]</a> <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> <a class="nb" href="/lkml/mheaders/2005/12/11/68" onclick="this.href='/lkml/headers'+'/2005/12/11/68';">[headers]</a> <a href="/lkml/bounce/2005/12/11/68">[forward]</a> </div></td><td width="32"> </td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/12/8/155">First message in thread</a></li><li><a href="/lkml/2005/12/10/68">Srivatsa Vaddagiri</a><ul><li><a href="/lkml/2005/12/10/79">Oleg Nesterov</a><ul><li class="origin"><a href="/lkml/2005/12/11/129">Srivatsa Vaddagiri</a><ul><li><a href="/lkml/2005/12/11/129">Andrew James Wade</a><ul><li><a href="/lkml/2005/12/11/153">Rusty Russell</a></li></ul></li></ul></li><li><a href="/lkml/2005/12/11/193">"Paul E. McKenney"</a><ul><li><a href="/lkml/2005/12/11/208">Andrew Morton</a><ul><li><a href="/lkml/2005/12/11/211">"David S. Miller"</a></li><li><a href="/lkml/2005/12/11/213">Paul Mackerras</a></li><li><a href="/lkml/2005/12/12/8">Keith Owens</a></li></ul></li></ul></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Sun, 11 Dec 2005 23:11:14 +0530</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Srivatsa Vaddagiri <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Semantics of smp_mb() [was : Re: [PATCH] Fix RCU race in access of nohz_cpu_mask ]</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">[Changed the subject line to be more generic in the interest of wider audience]<br /><br />We seem to be having some confusion over the exact semantics of smp_mb().<br /><br />Specifically, are all stores preceding smp_mb() guaranteed to have finished<br />(committed to memory/corresponding cache-lines on other CPUs invalidated) <br />*before* successive loads are issued?<br /><br /><br />From IA-32 manual (download.intel.com/design/Pentium4/manuals/25366817.pdf<br />Page 271):<br /><br />"Memory mapped devices and other I/O devices on the bus are often sensitive to <br />the order of writes to their I/O buffers. I/O instructions can be used to (the <br />IN and OUT instructions) impose strong write ordering on such accesses as <br />follows. Prior to executing an I/O instruction, the processor waits for all <br /> ___________________________<br />previous instructions in the program to complete and for all buffered<br />_____________________________________________________________________<br />writes to drain to memory. Only instruction fetch and page tables walks can <br />_________________________<br />pass I/O instructions. Execution of subsequent instructions do not begin until <br />the processor determines that the I/O instruction has been completed."<br /><br />Synchronization mechanisms in multiple-processor systems may depend upon a <br />strong memory-ordering model. Here, a program can use a locking instruction <br />such as the XCHG instruction or the LOCK prefix to insure that a <br />read-modify-write operation on memory is carried out atomically. Locking <br />operations typically operate like I/O operations in that they wait for all <br /> _________________<br />previous instructions to complete and for all buffered writes to drain to <br />_________________________________________________________________________<br />memory (see Section 7.1.2, “Bus Locking”).<br />______<br /><br />Program synchronization can also be carried out with serializing instructions <br />(see Section 7.4). These instructions are typically used at critical procedure <br />or task boundaries to force completion of all previous instructions before a <br />jump to a new section of code or a context switch occurs. Like the I/O and <br />locking instructions, the processor waits until all previous instructions have<br /> ________________________________________________________<br />been completed and all buffered writes have been drained to memory before <br />_________________________________________________________________________<br />executing the serializing instruction."<br />_____________________________________<br /><br /><br />From this, looks like we can interpret that IA-32 processor will wait for all <br />writes to drain to memory (implies cache invalidation on other CPUs?) *before* <br />executing the synchronizing instruction?<br /><br />Question is can this be generalized for other CPUs too?<br /><br /><br />On Sat, Dec 10, 2005 at 09:55:35PM +0300, Oleg Nesterov wrote:<br />> Srivatsa Vaddagiri wrote:<br />> > <br />> > On Fri, Dec 09, 2005 at 10:17:38PM +0300, Oleg Nesterov wrote:<br />> > > > rcp->cur++; /* New GP */<br />> > > ><br />> > > > smp_mb();<br />> > ><br />> > > I think I need some education on memory barriers.<br />> > ><br />> > > Does this mb() garantees that the new value of ->cur will be visible<br />> > > on other cpus immediately after smp_mb() (so that rcu_pending() will<br />> > > notice it) ?<br />> > <br />> > AFAIK the new value of ->cur should be visible to other CPUs when smp_mb()<br />> > returns. Here's a definition of smp_mb() from Paul's article:<br />> > <br />> > smp_mb(): "memory barrier" that orders both loads and stores. This means loads<br />> > and stores preceding the memory barrier are committed to memory before any<br />> > loads and stores following the memory barrier.<br />> <br />> Thanks, but this definition talks about ordering, it does not say<br />> anything about the time when stores are really commited to memory<br />> (and does it mean also that cache-lines are invalidated on other<br />> cpus ?).<br />> <br />> > [ <a href="http://www.linuxjournal.com/article/8211">http://www.linuxjournal.com/article/8211</a> ]<br />> <br />> And thanks for this link, now I have some understanding about<br />> read_barrier_depends() ...<br />> <br />> Oleg.<br /><br />-- <br /><br /><br />Thanks and Regards,<br />Srivatsa Vaddagiri,<br />Linux Technology Center,<br />IBM Software Labs,<br />Bangalore, INDIA - 560017<br />-<br />To unsubscribe from this list: send the line "unsubscribe linux-kernel" in<br />the body of a message to majordomo@vger.kernel.org<br />More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a><br />Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a><br /><br /></pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> </td></tr><tr><td align="right" valign="bottom"> </td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c"> </td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> </td><td class="lm">Last update: 2005-12-11 18:45 [from the cache]<br />©2003-2020 <a href="http://blog.jasper.es/"><span itemprop="editor">Jasper Spaans</span></a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td> </td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>