CINXE.COM

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>LKML: Peter Xu: Re: [PATCH 0/5] vfio: Improve DMA mapping performance for huge pfnmaps</title><link href="/css/message.css" rel="stylesheet" type="text/css" /><link href="/css/wrap.css" rel="alternate stylesheet" type="text/css" title="wrap" /><link href="/css/nowrap.css" rel="stylesheet" type="text/css" title="nowrap" /><link href="/favicon.ico" rel="shortcut icon" /><script src="/js/simple-calendar.js" type="text/javascript"></script><script src="/js/styleswitcher.js" type="text/javascript"></script><link rel="alternate" type="application/rss+xml" title="lkml.org : last 100 messages" href="/rss.php" /><link rel="alternate" type="application/rss+xml" title="lkml.org : last messages by Peter Xu" href="/groupie.php?aid=" /><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(["setDoNotTrack", true]); _paq.push(["disableCookies"]); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//m.lkml.org/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script></head><body onload="es.jasper.simpleCalendar.init();" itemscope="itemscope" itemtype="http://schema.org/BlogPosting"><table border="0" cellpadding="0" cellspacing="0"><tr><td width="180" align="center"><a href="/"><img style="border:0;width:135px;height:32px" src="/images/toprowlk.gif" alt="lkml.org" /></a></td><td width="32">聽</td><td class="nb"><div><a class="nb" href="/lkml"> [lkml]</a> 聽 <a class="nb" href="/lkml/2025"> [2025]</a> 聽 <a class="nb" href="/lkml/2025/2"> [Feb]</a> 聽 <a class="nb" href="/lkml/2025/2/6"> [6]</a> 聽 <a class="nb" href="/lkml/last100"> [last100]</a> 聽 <a href="/rss.php"><img src="/images/rss-or.gif" border="0" alt="RSS Feed" /></a></div><div>Views: <a href="#" class="nowrap" onclick="setActiveStyleSheet('wrap');return false;">[wrap]</a><a href="#" class="wrap" onclick="setActiveStyleSheet('nowrap');return false;">[no wrap]</a> 聽 <a class="nb" href="/lkml/mheaders/2025/2/6/1502" onclick="this.href='/lkml/headers'+'/2025/2/6/1502';">[headers]</a>聽 <a href="/lkml/bounce/2025/2/6/1502">[forward]</a>聽 </div></td><td width="32">聽</td></tr><tr><td valign="top"><div class="es-jasper-simpleCalendar" baseurl="/lkml/"></div><div class="threadlist">Messages in this thread</div><ul class="threadlist"><li class="root"><a href="/lkml/2025/2/5/1695">First message in thread</a></li><li><a href="/lkml/2025/2/5/1695">Alex Williamson</a><ul><li><a href="/lkml/2025/2/5/1696">Alex Williamson</a><ul><li><a href="/lkml/2025/2/6/1820">Mitchell Augustin</a></li><li><a href="/lkml/2025/2/14/1302">Jason Gunthorpe</a></li></ul></li><li><a href="/lkml/2025/2/5/1697">Alex Williamson</a><ul><li><a href="/lkml/2025/2/6/1821">Mitchell Augustin</a></li></ul></li><li><a href="/lkml/2025/2/5/1698">Alex Williamson</a><ul><li><a href="/lkml/2025/2/6/1823">Mitchell Augustin</a></li><li><a href="/lkml/2025/2/14/1240">Alex Williamson</a><ul><li><a href="/lkml/2025/2/14/1514">David Hildenbrand</a><ul><li><a href="/lkml/2025/2/17/1504">Alex Williamson</a></li></ul></li></ul></li><li><a href="/lkml/2025/2/14/1376">Jason Gunthorpe</a></li></ul></li><li><a href="/lkml/2025/2/5/1699">Alex Williamson</a><ul><li><a href="/lkml/2025/2/6/1822">Mitchell Augustin</a></li></ul></li><li><a href="/lkml/2025/2/5/1700">Alex Williamson</a><ul><li><a href="/lkml/2025/2/6/1824">Mitchell Augustin</a></li><li><a href="/lkml/2025/2/14/1381">Jason Gunthorpe</a><ul><li><a href="/lkml/2025/2/17/1502">Alex Williamson</a></li></ul></li><li><a href="/lkml/2025/2/14/1412">Matthew Wilcox</a><ul><li><a href="/lkml/2025/2/17/1383">Alex Williamson</a></li></ul></li></ul></li><li class="origin"><a href="">Peter Xu</a></li><li><a href="/lkml/2025/2/6/1825">Mitchell Augustin</a></li></ul></li></ul></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerl.gif" width="32" height="32" alt="/" /></td><td class="c" rowspan="2" valign="top" style="padding-top: 1em"><table><tr><td><table><tr><td class="lp">Date</td><td class="rp" itemprop="datePublished">Thu, 6 Feb 2025 14:14:49 -0500</td></tr><tr><td class="lp">From</td><td class="rp" itemprop="author">Peter Xu <></td></tr><tr><td class="lp">Subject</td><td class="rp" itemprop="name">Re: [PATCH 0/5] vfio: Improve DMA mapping performance for huge pfnmaps</td></tr></table></td><td></td></tr></table><pre itemprop="articleBody">On Wed, Feb 05, 2025 at 04:17:16PM -0700, Alex Williamson wrote:<br />> As GPU BAR sizes increase, the overhead of DMA mapping pfnmap ranges has<br />> become a significant overhead for VMs making use of device assignment.<br />> Not only does each mapping require upwards of a few seconds, but BARs<br />> are mapped in and out of the VM address space multiple times during<br />> guest boot. Also factor in that multi-GPU configurations are<br />> increasingly commonplace and BAR sizes are continuing to increase.<br />> Configurations today can already be delayed minutes during guest boot.<br />> <br />> We've taken steps to make Linux a better guest by batching PCI BAR<br />> sizing operations[1], but it only provides and incremental improvement.<br />> <br />> This series attempts to fully address the issue by leveraging the huge<br />> pfnmap support added in v6.12. When we insert pfnmaps using pud and pmd<br />> mappings, we can later take advantage of the knowledge of the mapping<br />> level page mask to iterate on the relevant mapping stride. In the<br />> commonly achieved optimal case, this results in a reduction of pfn<br />> lookups by a factor of 256k. For a local test system, an overhead of<br />> ~1s for DMA mapping a 32GB PCI BAR is reduced to sub-millisecond (8M<br />> page sized operations reduced to 32 pud sized operations).<br />> <br />> Please review, test, and provide feedback. I hope that mm folks can<br />> ack the trivial follow_pfnmap_args update to provide the mapping level<br />> page mask. Naming is hard, so any preference other than pgmask is<br />> welcome. Thanks,<br />> <br />> Alex<br />> <br />> [1]<a href="https://lore.kernel.org/all/20250120182202.1878581-1-alex.williamson@redhat.com/">https://lore.kernel.org/all/20250120182202.1878581-1-alex.williamson@redhat.com/</a><br />> <br />> <br />> Alex Williamson (5):<br />> vfio/type1: Catch zero from pin_user_pages_remote()<br />> vfio/type1: Convert all vaddr_get_pfns() callers to use vfio_batch<br />> vfio/type1: Use vfio_batch for vaddr_get_pfns()<br />> mm: Provide page mask in struct follow_pfnmap_args<br />> vfio/type1: Use mapping page mask for pfnmaps<br /><br />FWIW:<br /><br />Reviewed-by: Peter Xu <peterx@redhat.com><br /><br />Thanks,<br /><br />-- <br />Peter Xu<br /><br /><br /></pre></td><td width="32" rowspan="2" class="c" valign="top"><img src="/images/icornerr.gif" width="32" height="32" alt="\" /></td></tr><tr><td align="right" valign="bottom"> 聽 </td></tr><tr><td align="right" valign="bottom">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerl.gif" width="32" height="32" alt="\" /></td><td class="c">聽</td><td class="c" valign="bottom" style="padding-bottom: 0px"><img src="/images/bcornerr.gif" width="32" height="32" alt="/" /></td></tr><tr><td align="right" valign="top" colspan="2"> 聽 </td><td class="lm">Last update: 2025-02-06 20:15 聽聽 [from the cache]<br />漏2003-2020 <a href="http://blog.jasper.es/"><span itemprop="editor">Jasper Spaans</span></a>|hosted at <a href="https://www.digitalocean.com/?refcode=9a8e99d24cf9">Digital Ocean</a> and my Meterkast|<a href="http://blog.jasper.es/categories.html#lkml-ref">Read the blog</a></td><td>聽</td></tr></table><script language="javascript" src="/js/styleswitcher.js" type="text/javascript"></script></body></html>