CINXE.COM

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Steven Rostedt: Re: [PATCH 00/05] robust per_cpu allocation for modules</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Steven Rostedt" href="/groupie.php?aid=3543" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2006"> [2006]</a> 聽 <a class="nb" href="/lkml/2006/4"> [Apr]</a> 聽 <a class="nb" href="/lkml/2006/4/16"> [16]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2006/4/16/49" onclick="this.href='/lkml/headers'+'/2006/4/16/49';">[headers]</a>聽 <a href="/lkml/bounce/2006/4/16/49">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2006/4/14/137">First message in thread</a></li><li><a href="/lkml/2006/4/16/19">Steven Rostedt</a><ul><li><a href="/lkml/2006/4/16/20">Sam Ravnborg</a></li><li><a href="/lkml/2006/4/16/25">Arnd Bergmann</a><ul><li><a href="/lkml/2006/4/16/33">"Tony Luck"</a></li><li class="origin"><a href="/lkml/2006/4/16/58">Steven Rostedt</a><ul><li><a href="/lkml/2006/4/16/58">Arnd Bergmann</a><ul><li><a href="/lkml/2006/4/16/60">Steven Rostedt</a></li></ul></li></ul></li><li><a href="/lkml/2006/4/17/93">Ravikiran G Thirumalai</a></li></ul></li><li><a href="/lkml/2006/4/17/1">Rusty Russell</a><ul><li><a href="/lkml/2006/4/17/14">Steven Rostedt</a></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: [PATCH 00/05] robust per_cpu allocation for modules</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Steven Rostedt <></td></tr><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Sun, 16 Apr 2006 20:45:50 -0400</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">On Sun, 2006-04-16 at 17:34 +0200, Arnd Bergmann wrote:<br />> On Sunday 16 April 2006 15:40, Steven Rostedt wrote:<br />> > I'll think more about this, but maybe someone else has some crazy ideas<br />> > that can find a solution to this that is both fast and robust.<br />> <br />> Ok, you asked for a crazy idea, you're going to get it ;-)<br />> <br />> You could take a fixed range from the vmalloc area (e.g. 1MB per cpu)<br />> and use that to remap pages on demand when you need per cpu data.<br />> <br />> #define PER_CPU_BASE 0xe000000000000000UL /* arch dependant */<br />> #define PER_CPU_SHIFT 0x100000UL<br />> #define __per_cpu_offset(__cpu) (PER_CPU_BASE + PER_CPU_STRIDE * (__cpu))<br />> #define per_cpu(var, cpu) (*RELOC_HIDE(&per_cpu__##var, __per_cpu_offset(cpu)))<br />> #define __get_cpu_var(var) per_cpu(var, smp_processor_id())<br />> <br />> This is a lot like the current sparc64 implementation already is.<br />> <br /><br />Hmm, interesting idea.<br /><br />> The tricky part here is the remapping of pages. You'd need to <br />> alloc_pages_node() new pages whenever the already reserved space is<br />> not enough for the module you want to load and then map_vm_area()<br />> them into the space reserved for them.<br />> <br />> Advantages of this solution are:<br />> - no dependant load access for per_cpu()<br />> - might be flexible enough to implement a faster per_cpu_ptr()<br />> - can be combined with ia64-style per-cpu remapping<br />> <br />> Disadvantages are:<br />> - you can't use huge tlbs for mapping per cpu data like the<br />> regular linear mapping -> may be slower on some archs<br /><br />> - does not work in real mode, so percpu data can't be used<br />> inside exception handlers on some architectures.<br /><br />This is probably a big issue. I believe interrupt context in hrtimers<br />uses per_cpu variables.<br /><br />> - memory consumption is rather high when PAGE_SIZE is large<br /><br />That's also something that I'm trying to solve. To use the least amount<br />of memory and still have the performance.<br /><br />Now, I've also thought about allocating per_cpu and when a module is<br />loaded, reallocate more memory and copy it again. Use something like<br />the kstopmachine to sync the system so that the CPUS don't update any<br />per_cpu variables while this is happening, so that things can't get out<br />of sync.<br /><br />This shouldn't be too much of an issue, since this would only be done<br />when a module is being loaded, and that is a user event that doesn't<br />happen often.<br /><br />We would still need to use the method of keeping track of what is<br />allocated and freed, so that when a module is unloaded, we can still<br />free the area in the per_cpu data. And reallocate that area if a module<br />is added that uses less or the same amount of memory as what was freed.<br /><br />-- Steve<br /><br /><br />-<br />To unsubscribe from this list: send the line "unsubscribe linux-kernel" in<br />the body of a message to majordomo@vger.kernel.org<br />More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a><br />Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a><br /><br /></pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2006-04-17 02:48 聽聽 [from the cache]<br />漏2003-2020 <a href="http://blog.jasper.es/"><span itemprop="editor">Jasper Spaans</span></a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>