LKML: Paul Jackson: Re: [patch] sched: auto-tune migration costs [was: Re: Industry db benchmark result on recent 2.6 kernels]

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Paul Jackson: Re: [patch] sched: auto-tune migration costs [was: Re: Industry db benchmark result on recent 2.6 kernels]</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Paul Jackson" href="/groupie.php?aid=5266" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2005"> [2005]</a> 聽 <a class="nb" href="/lkml/2005/4"> [Apr]</a> 聽 <a class="nb" href="/lkml/2005/4/3"> [3]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2005/4/3/6" onclick="this.href='/lkml/headers'+'/2005/4/3/6';">[headers]</a>聽 <a href="/lkml/bounce/2005/4/3/6">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2005/4/1/310">First message in thread</a></li><li><a href="/lkml/2005/4/1/310">"Chen, Kenneth W"</a><ul><li><a href="/lkml/2005/4/1/325">Nick Piggin</a></li><li><a href="/lkml/2005/4/2/34">Ingo Molnar</a><ul><li><a href="/lkml/2005/4/2/96">Paul Jackson</a></li><li class="origin"><a href="/lkml/2005/4/3/19">Paul Jackson</a><ul><li><a href="/lkml/2005/4/3/19">Ingo Molnar</a><ul><li><a href="/lkml/2005/4/3/28">Paul Jackson</a></li><li><a href="/lkml/2005/4/3/55">Paul Jackson</a></li><li><a href="/lkml/2005/4/3/145">"Chen, Kenneth W"</a></li></ul></li><li><a href="/lkml/2005/4/3/37">Paul Jackson</a></li></ul></li></ul></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Sat, 2 Apr 2005 21:53:32 -0800</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Paul Jackson <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: [patch] sched: auto-tune migration costs [was: Re: Industry db benchmark result on recent 2.6 kernels]</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">Just so as no else wastes time repeating the little bit I've done so far, and so I don't waste time figuring out what is already known, here's what I have so far, trying out Ingo's "sched: auto-tune migration costs" on ia64 SN2: To get it to compile against 2.6.12-rc1-mm4, I did thus: 1. Manually edited "include/asm-x86_64/topology.h" to remove .cache_hot_time (patch failed due to conflicts with nearby changes to add some *_idx terms). 2. Moved the 394 line block of new code in kernel/sched.c to _before_ the large #ifdef ARCH_HAS_SCHED_DOMAIN, #else, #endif block. The ia64 arch (only) defines ARCH_HAS_SCHED_DOMAIN, so was being denied use of Ingo's code when it was buried in the '#else-#endif' side of this large conditional block. 3. Add "#include <linux/vmalloc.h>" to kernel/sched.c 4. Don't print cpu_khz in the cost matrix header, as cpu_khz is only in a few arch's (x86_64, ppc, i386, arm). Note that (2) was just a superficial fix - it compiles, but the result could easily be insanely stupid and I'd have no clue. I need to read the code some more. Booting on an 8 CPU ia64 SN2, the console output got far enough to show: ============================ begin ============================ Brought up 8 CPUs softlockup thread 7 started up. Total of 8 processors activated (15548.60 BogoMIPS). --------------------- migration cost matrix (max_cache_size: 33554432): --------------------- [00] [01] [02] [03] [04] [05] [06] [07] [00]: - ============================= end ============================= Then it hung for 5 or 10 minutes, and then it blurted out a panic and died. I'll quote the whole panic, including backtrace, in case someone happens to see something obvious. But I'm not asking anyone to think about this yet, unless it amuses them. I can usefully occupy myself reading the code and adding printk's for a while. Note the first 3 chars of the panic message "4.5". This looks like it might be the [00]-[01] entry of Ingo's table, flushed out when the newlines of the panic came through. ============================ begin ============================ 4.5(0)<1>Unable to handle kernel paging request at virtual address 0000000000010008 swapper[1]: Oops 8813272891392 [1] Modules linked in: Pid: 1, CPU 0, comm: swapper psr : 0000101008026018 ifs : 8000000000000288 ip : [<a0000001000d9a30>] Not tainted ip is at queue_work+0xb0/0x1a0 unat: 0000000000000000 pfs : 0000000000000288 rsc : 0000000000000003 rnat: a000000100ab2a50 bsps: 0000000000100000 pr : 5a66666956996a65 ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c8a70033f csd : 0000000000000000 ssd : 0000000000000000 b0 : a0000001000d99b0 b6 : a000000100003320 b7 : a000000100490200 f6 : 1003e0000000000009ff7 f7 : 1003e000418d3645db265 f8 : 1003e000000003b8186ed f9 : 1003e0000000000005f3b f10 : 1003e0000000000001000 f11 : 1003e0000000000000040 r1 : a000000100c9de60 r2 : 0000000000000000 r3 : 0000000000000001 r8 : 0000000000000000 r9 : 0000000000000000 r10 : a000000100969c50 r11 : 0000000000000004 r12 : e00001b03a8d7910 r13 : e00001b03a8d0000 r14 : 0000000000000000 r15 : 0000000000010008 r16 : e00001b03a8d0dc0 r17 : 0000000000010008 r18 : 0000000000000103 r19 : a000000100c32048 r20 : a000000100c32018 r21 : a000000100aa92c8 r22 : e000003003005d90 r23 : e000003003005da8 r24 : a000000100cf2098 r25 : e000003003005db0 r26 : a000000100ab4bf4 r27 : e000003003005d81 r28 : 000000010004b001 r29 : 0000000000000000 r30 : 000000010004b000 r31 : a000000100c32010 Call Trace: [<a000000100010460>] show_stack+0x80/0xa0 sp=e00001b03a8d74d0 bsp=e00001b03a8d1620 [<a000000100010d40>] show_regs+0x860/0x880 sp=e00001b03a8d76a0 bsp=e00001b03a8d15b8 [<a000000100036390>] die+0x170/0x200 sp=e00001b03a8d76b0 bsp=e00001b03a8d1580 [<a00000010005bb20>] ia64_do_page_fault+0x200/0xa40 sp=e00001b03a8d76b0 bsp=e00001b03a8d1520 [<a00000010000b2c0>] ia64_leave_kernel+0x0/0x290 sp=e00001b03a8d7740 bsp=e00001b03a8d1520 [<a0000001000d9a30>] queue_work+0xb0/0x1a0 sp=e00001b03a8d7910 bsp=e00001b03a8d14e0 [<a0000001000db0d0>] schedule_work+0x30/0x60 sp=e00001b03a8d7910 bsp=e00001b03a8d14c8 [<a000000100490230>] blank_screen_t+0x30/0x60 sp=e00001b03a8d7910 bsp=e00001b03a8d14b8 [<a0000001000c8130>] run_timer_softirq+0x2d0/0x4a0 sp=e00001b03a8d7910 bsp=e00001b03a8d1410 [<a0000001000bb920>] __do_softirq+0x220/0x260 sp=e00001b03a8d7930 bsp=e00001b03a8d1378 [<a0000001000bb9e0>] do_softirq+0x80/0xe0 sp=e00001b03a8d7930 bsp=e00001b03a8d1320 [<a0000001000bbc50>] irq_exit+0x90/0xc0 sp=e00001b03a8d7930 bsp=e00001b03a8d1310 [<a00000010000f4b0>] ia64_handle_irq+0x110/0x140 sp=e00001b03a8d7930 bsp=e00001b03a8d12d8 [<a00000010000b2c0>] ia64_leave_kernel+0x0/0x290 sp=e00001b03a8d7930 bsp=e00001b03a8d12d8 [<a000000100844b20>] read_cache+0x40/0x60 sp=e00001b03a8d7b00 bsp=e00001b03a8d12c8 [<a000000100844fb0>] target_handler+0xd0/0xe0 sp=e00001b03a8d7b00 bsp=e00001b03a8d1298 [<a000000100845150>] measure_one+0x190/0x240 sp=e00001b03a8d7b00 bsp=e00001b03a8d1260 [<a000000100845890>] measure_cacheflush_time+0x270/0x420 sp=e00001b03a8d7b30 bsp=e00001b03a8d1200 [<a0000001000a7350>] calibrate_cache_decay+0x710/0x740 sp=e00001b03a8d7b40 bsp=e00001b03a8d1148 [<a000000100056180>] arch_init_sched_domains+0x12c0/0x1e40 sp=e00001b03a8d7b60 bsp=e00001b03a8d0e80 [<a000000100845a60>] sched_init_smp+0x20/0x60 sp=e00001b03a8d7de0 bsp=e00001b03a8d0e70 [<a000000100009570>] init+0x250/0x440 sp=e00001b03a8d7de0 bsp=e00001b03a8d0e38 [<a000000100012940>] kernel_thread_helper+0xe0/0x100 sp=e00001b03a8d7e30 bsp=e00001b03a8d0e10 [<a000000100009120>] start_kernel_thread+0x20/0x40 sp=e00001b03a8d7e30 bsp=e00001b03a8d0e10 <0>Kernel panic - not syncing: Aiee, killing interrupt handler! ============================= end ============================= -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson <pj@engr.sgi.com> 1.650.933.1373, 1.925.600.0401 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at <a href="http://vger.kernel.org/majordomo-info.html">http://vger.kernel.org/majordomo-info.html</a> Please read the FAQ at <a href="http://www.tux.org/lkml/">http://www.tux.org/lkml/</a> </pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2005-04-06 13:31 聽聽 [from the cache] 漏2003-2020 <a href="http://blog.jasper.es/">Jasper Spaans</a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>

CINXE.COM

LKML: Paul Jackson: Re: [patch] sched: auto-tune migration costs [was: Re: Industry db benchmark result on recent 2.6 kernels]