LKML: Jack Steiner: Re: [PATCH] - Fix memory ordering problem in wake

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Jack Steiner: Re: [PATCH] - Fix memory ordering problem in wake_futex()</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Jack Steiner" href="/groupie.php?aid=5277" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2005"> [2005]</a> 聽 <a class="nb" href="/lkml/2005/12"> [Dec]</a> 聽 <a class="nb" href="/lkml/2005/12/23"> [23]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2005/12/23/141" onclick="this.href='/lkml/headers'+'/2005/12/23/141';">[headers]</a>聽 <a href="/lkml/bounce/2005/12/23/141">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/12/23/92">First message in thread</a></li><li><a href="/lkml/2005/12/23/92">Jack Steiner</a><ul><li><a href="/lkml/2005/12/23/95">Joe Seigh</a></li><li><a href="/lkml/2005/12/23/133">Olof Johansson</a><ul><li class="origin"><a href="/lkml/2005/12/23/146">Jack Steiner</a><ul><li><a href="/lkml/2005/12/23/146">Olof Johansson</a><ul><li><a href="/lkml/2005/12/23/198">Robin Holt</a></li></ul></li></ul></li></ul></li><li><a href="/lkml/2005/12/24/27">Jack Steiner</a><ul><li><a href="/lkml/2005/12/24/44">Olof Johansson</a><ul><li><a href="/lkml/2005/12/27/98">Jack Steiner</a></li></ul></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Fri, 23 Dec 2005 15:32:16 -0600</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Jack Steiner <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: [PATCH] - Fix memory ordering problem in wake_futex()</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">On Fri, Dec 23, 2005 at 02:48:22PM -0600, Olof Johansson wrote:<br />> On Fri, Dec 23, 2005 at 10:38:16AM -0600, Jack Steiner wrote:<br />> > <br />> > Here is a fix for a ugly race condition that occurs in wake_futex() on IA64.<br />> > <br />> > On IA64, locks are released using a "st.rel" instruction. This ensures that<br />> > preceding "stores" are visible before the lock is released but does NOT prevent<br />> > a "store" that follows the "st.rel" from becoming visible before the "st.rel".<br />> > The result is that the task that owns the futex_q continues prematurely. <br />> > <br />> > The failure I saw is the task that owned the futex_q resumed prematurely and<br />> > was context-switch off of the cpu. The task's switch_stack occupied the same<br />> > space of the futex_q. The store to q->lock_ptr overwrote the ar.bspstore in the<br />> > switch_stack. When the task resumed, it ran with a corrupted ar.bspstore.<br />> > Things went downhill from there.<br />> > <br />> > Without the fix, the application fails roughly every 10 minutes. With<br />> > the fix, it ran 16 hours without a failure.<br />> <br />> So what happened to what the comment 10 lines above your patch says?<br />> <br />> /*<br />> * The lock in wake_up_all() is a crucial memory barrier after<br />> * the list_del_init() and also before assigning to q->lock_ptr.<br />> */<br />> <br />> On PPC64, the spinlock unlock path has a sync in there for the very<br />> purpose of adding the write barrier. Maybe the ia64 unlock path is<br />> missing something similar?<br /><br />On IA64, the "sync" instructions are actually part of the ld.acq ot st.rel<br />instructions that are used to set/clear spinlocks.<br /><br />The program order of the instructions is:<br /><br /> cmpxchg.acq // set lock<br /> ..<br /> .. do things<br /> ..<br /> st.rel // release lock<br /><br /> st NULL -> lock_ptr // set flag to allow futex_q to be freed<br /><br /><br />IA64 implements fencing of ld.acq or st.rel instructions as one-directional<br />barriers.<br /><br />For example, st.rel allows stores that FOLLOW the st.rel to be reordered<br />and become visible before the st.rel<br />(I hope this picture survives the various mailers)<br /><br /><br /> ld |<br /> st | | ^ ^<br /> - fence - | |<br /> st.rel --------------|-----|------------------- fence<br /> | |<br /> ld | |<br /> st |<br /><br />I don't understand PPC64 but I suspect it has different memory ordering semantics.<br /><br />-- <br />Thanks<br /><br />Jack Steiner (steiner@sgi.com) 651-683-5302<br />Principal Engineer SGI - Silicon Graphics, Inc.<br /><br /><br />-<br />To unsubscribe from this list: send the line "unsubscribe linux-kernel" in<br />the body of a message to majordomo@vger.kernel.org<br />More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a><br />Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a><br /><br /></pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2005-12-23 22:34 聽聽 [from the cache]<br />漏2003-2020 <a href="http://blog.jasper.es/"><span itemprop="editor">Jasper Spaans</span></a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>

CINXE.COM

LKML: Jack Steiner: Re: [PATCH] - Fix memory ordering problem in wake_futex()