LKML: Ralph Campbell: Re: [openib-general] Re: [PATCH 10/13] [RFC] ipath verbs, part 1

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Ralph Campbell: Re: [openib-general] Re: [PATCH 10/13] [RFC] ipath verbs, part 1</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Ralph Campbell" href="/groupie.php?aid=22552" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2005"> [2005]</a> 聽 <a class="nb" href="/lkml/2005/12"> [Dec]</a> 聽 <a class="nb" href="/lkml/2005/12/19"> [19]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2005/12/19/163" onclick="this.href='/lkml/headers'+'/2005/12/19/163';">[headers]</a>聽 <a href="/lkml/bounce/2005/12/19/163">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/12/16/290">First message in thread</a></li><li><a href="/lkml/2005/12/16/299">Roland Dreier</a><ul><li><a href="/lkml/2005/12/16/295">Roland Dreier</a><ul><li><a href="/lkml/2005/12/16/302">Roland Dreier</a><ul><li><a href="/lkml/2005/12/16/292">Roland Dreier</a><ul><li><a href="/lkml/2005/12/16/300">Roland Dreier</a></li></ul></li></ul></li></ul></li><li><a href="/lkml/2005/12/18/89">"Paul E. McKenney"</a><ul><li><a href="/lkml/2005/12/18/92">Robert Walsh</a></li><li class="origin"><a href="">Ralph Campbell</a></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: [openib-general] Re: [PATCH 10/13] [RFC] ipath verbs, part 1</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Ralph Campbell <></td></tr><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Mon, 19 Dec 2005 12:50:27 -0800</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">The quick answer is the qp_list is traversed w/o the lock held in ipath_ib_rcv(). The intent is to be able to do a lookup on the GID to get a reference to the struct ipath_mcast and then walk the qp_list w/o locks being held while processing the received packets at interrupt level. On Sun, 2005-12-18 at 11:59 -0800, Paul E. McKenney wrote: > On Fri, Dec 16, 2005 at 03:48:55PM -0800, Roland Dreier wrote: > > First half of ipath verbs driver > > Some RCU-related questions interspersed. Basic question is "where is > the lock-free read-side traversal?" > > Thanx, Paul > > > --- > > > > drivers/infiniband/hw/ipath/ipath_verbs.c | 3244 +++++++++++++++++++++++++++++ > > 1 files changed, 3244 insertions(+), 0 deletions(-) > > create mode 100644 drivers/infiniband/hw/ipath/ipath_verbs.c ... > > +/* > > + * Insert the multicast GID into the table and > > + * attach the QP structure. > > + * Return zero if both were added. > > + * Return EEXIST if the GID was already in the table but the QP was added. > > + * Return ESRCH if the QP was already attached and neither structure was added. > > + */ > > +static int ipath_mcast_add(struct ipath_mcast *mcast, > > + struct ipath_mcast_qp *mqp) > > +{ > > + struct rb_node **n = &mcast_tree.rb_node; > > + struct rb_node *pn = NULL; > > + unsigned long flags; > > + > > + spin_lock_irqsave(&mcast_lock, flags); > > + > > + while (*n) { > > + struct ipath_mcast *tmcast; > > + struct ipath_mcast_qp *p; > > + int ret; > > + > > + pn = *n; > > + tmcast = rb_entry(pn, struct ipath_mcast, rb_node); > > + > > + ret = memcmp(mcast->mgid.raw, tmcast->mgid.raw, > > + sizeof(union ib_gid)); > > + if (ret < 0) { > > + n = &pn->rb_left; > > + continue; > > + } > > + if (ret > 0) { > > + n = &pn->rb_right; > > + continue; > > + } > > + > > + /* Search the QP list to see if this is already there. */ > > + list_for_each_entry_rcu(p, &tmcast->qp_list, list) { > > Given that we hold the global mcast_lock, how is RCU helping here? Its not really. I'm just trying to be consistent where ever the qp_list is traversed. > Is there a lock-free read-side traversal path somewhere that I am > missing? The lock free traversal is in ipath_ib_rcv() which is an interrupt routine. > > + if (p->qp == mqp->qp) { > > + spin_unlock_irqrestore(&mcast_lock, flags); > > + return ESRCH; > > + } > > + } > > + list_add_tail_rcu(&mqp->list, &tmcast->qp_list); > > Ditto... > > > + spin_unlock_irqrestore(&mcast_lock, flags); > > + return EEXIST; > > + } > > + > > + list_add_tail_rcu(&mqp->list, &mcast->qp_list); > > Ditto... > > > + spin_unlock_irqrestore(&mcast_lock, flags); > > + > > + atomic_inc(&mcast->refcount); > > + rb_link_node(&mcast->rb_node, pn, n); > > + rb_insert_color(&mcast->rb_node, &mcast_tree); > > + > > + spin_unlock_irqrestore(&mcast_lock, flags); > > + > > + return 0; > > +} -- Ralph Campbell <ralphc@pathscale.com> - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a> Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a> </pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2005-12-19 21:53 聽聽 [from the cache] 漏2003-2020 <a href="http://blog.jasper.es/">Jasper Spaans</a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>

CINXE.COM

LKML: Ralph Campbell: Re: [openib-general] Re: [PATCH 10/13] [RFC] ipath verbs, part 1