FOSDEM 2021 - PostgreSQL Network Filter for EnvoyProxy

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head profile="http://a9.com/-/spec/opensearch/1.1/" prefix="og: http://ogp.me/ns#"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="generator" content="nanoc"/> <link rel="apple-touch-icon" sizes="120x120" href="/2021/apple-touch-icon.png" /> <link rel="icon" type="image/png" href="/2021/favicon-32x32.png" sizes="32x32" /> <link rel="icon" type="image/png" href="/2021/favicon-16x16.png" sizes="16x16" /> <link rel="manifest" href="/2021/manifest.json" /> <link rel="mask-icon" href="/2021/safari-pinned-tab.svg" color="#af2c93" /> <meta name="msapplication-TileColor" content="#af2c93"/> <meta name="theme-color" content="#ffffff" /> <link media="all" rel="stylesheet" type="text/css" href="/2021/assets/style/fosdem-1df2d7d48d3e43776f7285ddffdbb20e2d15457b668de6b890f46adb6a16bee4.css" /> <link href="/2021/atom.xml" type="application/atom+xml" rel="alternate" title="FOSDEM 2021 ATOM Feed"/> <link href="/2021/rss.xml" type="application/rss+xml" rel="alternate" title="FOSDEM 2021 RSS Feed"/> <link href="/2021/opensearch.xml" type="application/opensearchdescription+xml" title="FOSDEM 2021 Search"/> <title>FOSDEM 2021 - PostgreSQL Network Filter for EnvoyProxy</title> <link rel="schema.DC" href="http://purl.org/DC/elements/1.0/" /> <meta name="DC.Title" content="PostgreSQL Network Filter for EnvoyProxy"/> <meta name="DC.Format" content="text/html" /> <meta name="DC.Language" content="en" /> <meta name="DC.Subject" content="FOSDEM 2021" /> <meta name="DC.Rights" content="http://creativecommons.org/licenses/by-sa/2.0/be/" /> <meta property="og:title" content="FOSDEM 2021 - PostgreSQL Network Filter for EnvoyProxy" /> <meta property="og:type" content="website" /> <meta property="og:image" content="https://fosdem.org/2021/assets/style/logo-gear-7204a6874eb0128932db10ff4030910401ac06f4e907f8b4a40da24ba592b252.png" /> </head> <body class="schedule-event"> <div id="header"> <div class="header-holder"> <h1 class="logo"><a href="/2021/">FOSDEM21</a></h1> <ul id="nav"> <li class="home"><a href="/2021/">Home</a></li> <li><a href="/2021/about/">About</a></li> <li><a href="/2021/news/">News</a></li> <li class="active"><a href="/2021/schedule/">Schedule</a></li> <li><a href="https://stands.fosdem.org">Stands</a></li> <li><a href="/2021/volunteer/">Volunteer</a></li> <li><a href="/2021/practical/">Practical</a></li> </ul>  </div>  </div>  <div class="title-box"> <div class="holder"> <div class="frame"> <div class="text-block"> <h2>Online <span>/</span> <strong>6 & 7 February 2021</strong></h2> <a class="btn-purple" href="/2021/live/">visit</a> </div> <ul class="navbar"> <li><a href="/2021/news/">News</a></li> <li><a href="/2021/about/sponsors/">Sponsors</a></li> <li><a href="/2021/contact/">Contact</a></li> </ul> </div> </div> </div>  <div id="main"> <ul class="breadcrumbs"> <li><a href="/2021/">FOSDEM 2021</a></li><li class="separator">/</li><li><a href="/2021/schedule/">Schedule</a></li><li class="separator">/</li><li><a href="/2021/schedule/events/">Events</a></li><li class="separator">/</li><li>Developer rooms</li><li class="separator">/</li><li><a href="/2021/schedule/track/monitoring_and_observability/">Monitoring and Observability</a></li><li class="separator">/</li><li class="active">PostgreSQL Network Filter for EnvoyProxy</li> </ul> <div id="pagetitles"> <h1>PostgreSQL Network Filter for EnvoyProxy</h1> </div> <ul class="side-box"> <li><i class="icon-road"></i> <strong>Track</strong>: <a href="/2021/schedule/track/monitoring_and_observability/">Monitoring and Observability devroom</a></li> <li><i class="icon-home"></i> <strong>Room</strong>: <a href="/2021/schedule/room/dmonitoring/">D.monitoring</a></li> <li><i class="icon-calendar"></i> <strong>Day</strong>: <a href="/2021/schedule/day/sunday/">Sunday</a></li> <li><i class="icon-play"></i> <strong>Start</strong>: <a class="value-title" title="2021-02-07T11:30:00+01:00" href="/2021/schedule/day/sunday/#1130">11:30</a></li> <li><i class="icon-stop"></i> <strong>End</strong>: <a class="value-title" title="2021-02-07T12:05:00+01:00" href="/2021/schedule/day/sunday/#1205">12:05</a></li> <li> <i class="icon-facetime-video"></i> <strong>Video with Q&A</strong>: <a href="https://chat.fosdem.org/#/room/#monitoring:fosdem.org">D.monitoring</a> </li> <li id="live-stream-with-q-and-a" class="visible-desktop"> <i class="icon-film"></i> <strong>Video only</strong>: <a href="https://live.fosdem.org/watch/dmonitoring">D.monitoring</a> </li> <li> <i class="icon-comment"></i> <strong>Chat</strong>: <a href="https://matrix.to/#/#monitoring:fosdem.org?web-instance[element.io]=chat.fosdem.org">Join the conversation!</a> </li> </ul> <br style="clear: both;"/> <div class="event-blurb"> <div class="event-abstract"><p>How do you monitor Postgres? What information can you get out of it, and to what degree does this information help to troubleshoot operational issues? What if you want/need to log all the queries? That may bring heavy trafficked databases down.</p> <p>At OnGres we’re obsessed with improving PostgreSQL’s observability. So we worked together with Tetrate folks on an Envoy’s Network Filter extension for PostgreSQL, to provide and extend observability of the traffic inout a cluster infrastructure. This extension is public and open source. You can use it anywhere you use Envoy. It allows you to capture automated metrics and to debug network traffic. This talk will be a technical deep-dive into PostgreSQL’s protocol decoding, Envoy proxy filters and will cover all the capabilities of the tool and its usage and deployment in any environment.</p></div> <div class="event-description"><p>Envoy [1] is a high performance C++ distributed proxy designed for single services and applications, as well as a communication bus and “universal data plane” designed for large microservice “service mesh” architectures. Built on the learnings of solutions such as NGINX, HAProxy, hardware load balancers, and cloud load balancers, Envoy runs alongside every application and abstracts the network by providing common features in a platform-agnostic manner. When all service traffic in an infrastructure flows via an Envoy mesh, it becomes easy to visualize problem areas via consistent observability, tune overall performance, and add substrate features in a single place.</p> <p>Envoy can be used to proxy connections to PostgreSQL instances and in this talk we’ll see how we improve PostgreSQL observability without impacting the performance of the database and without needing to install and/or configure a bunch of things like logs, pg<em>stat</em>statements, etc, using a Network Filter [2] for PostgreSQL we developed that decodes frontend and backend protocol to get transparently some metrics and metadata about it operation.</p> <p>Roadmap: * [WIP] SSL termination and monitoring [3] [4] * Integrate Postgres parser to improve dynamic metadata and per-query tracking * Individual (per-query) tracking of query performance * Traffic mirroring for Postgres major upgrade testing and validations</p> <p>[1] https://www.envoyproxy.io/ [2] https://www.envoyproxy.io/docs/envoy/latest/intro/arch<em>overview/other</em>protocols/postgres#arch-overview-postgres [3] https://github.com/envoyproxy/envoy/issues/10942 [4] https://github.com/envoyproxy/envoy/issues/9577</p></div> </div> <div class="video"> <video preload="none" controls="controls"> <source src="https://video.fosdem.org/2021/D.monitoring/postgresql_filter_envoy.webm" type='video/webm; codecs="vp9, opus"' /> </video> </div> <h3>Speakers</h3> <table> <tbody> <tr> <td><a href="/2021/schedule/speaker/fabrizio_mello/"><img class="thumb" height="32" width="32" src="/2021/schedule/speaker/fabrizio_mello/41e20905cb4305ca7ca2d64a51f60976e19ec0deeb10094132d3be0c6709e2c3.png" alt="Photo of Fabrízio Mello"/></a></td> <td><a href="/2021/schedule/speaker/fabrizio_mello/">Fabrízio Mello</a></td> </tr> <tr> <td><a href="/2021/schedule/speaker/alvaro_hernandez/"><img class="thumb" height="32" width="32" src="/2021/schedule/speaker/alvaro_hernandez/a5d22268ea04759af630ec1bee1198882034f897a23102f9533dd98c1215a712.jpg" alt="Photo of Alvaro Hernandez"/></a></td> <td><a href="/2021/schedule/speaker/alvaro_hernandez/">Alvaro Hernandez</a></td> </tr> </tbody> </table> <h3>Attachments</h3> <ul class="event-attachments unstyled"> <li><i class="icon-download"></i> <a href="/2021/schedule/event/postgresql_filter_envoy/attachments/slides/4703/export/events/attachments/postgresql_filter_envoy/slides/4703/PostgreSQL_Network_Filter_EnvoyProxy.pdf">Slides (slides)</a></li> </ul> <h3>Links</h3> <ul> <li><a href="https://envoycon2020.sched.com/event/ecd1/postgresql-network-filter-for-envoyproxy">Presentation at EnvoyCon 2020</a></li> <li><a href="https://video.fosdem.org/2021/D.monitoring/postgresql_filter_envoy.webm">Video recording (WebM/VP9)</a></li> <li><a href="https://video.fosdem.org/2021/D.monitoring/postgresql_filter_envoy.mp4">Video recording (mp4)</a></li> <li><a href="https://submission.fosdem.org/feedback/11502.php">Submit feedback</a></li> </ul> </div>  <div id="footer"> <div class="footer-holder"> <div class="panel-block"> <div class="panel-column"> <h3>FOSDEM</h3> <ul class="panel"> <li class="home"><a accesskey="1" href="/2021/">Home</a></li> <li><a accesskey="2" href="/2021/news/">News</a></li> <li><a href="/2021/about/">About</a></li> <li><a href="/2021/about/sponsors/">Sponsors</a></li> <li><a href="/2021/support/donate/">Donate</a></li> <li><a href="/2021/tshirts/">T-shirts and hoodies</a></li> <li><a accesskey="5" href="/2021/faq/">FAQ</a></li> <li><a href="/2021/archives/">Archives</a></li> </ul> </div>  <div class="panel-column"> <h3>This year</h3> <ul class="panel"> <li><a href="/2021/live/">Visit</a></li> <li><a accesskey="s" href="/2021/schedule/">Schedule</a></li> <li><a href="/2021/stands/">Stands</a></li> <li><a href="/2021/certification/">Certification exams</a></li> <li><a href="/2021/tshirts/">T-shirts and hoodies</a></li> <li><a href="/2021/volunteer/">Volunteer</a></li> <li><a href="/2021/fringe/">Fringe</a></li> </ul> </div>  <div class="panel-column"> <h3>Practical information</h3> <ul class="panel"> <li><a href="/2021/practical/online/">FOSDEM 2021 Online</a></li> <li><a href="/2021/practical/matrix_troubleshooting/">Troubleshooting Matrix at FOSDEM 2021</a></li> <li><a href="/2021/tshirts/">T-shirts and hoodies</a></li> <li><a href="/2021/practical/accessibility/">Accessibility</a></li> <li><a href="/2021/practical/conduct/">Code of Conduct</a></li> <li><a href="/2021/practical/services/">During the Event</a></li> </ul> </div>  <div class="panel-column"> <h3>Media and press</h3> <ul class="panel"> <li><a href="/2021/social/">Social media</a></li> </ul> </div>  </div>  <div class="company-info"> <div class="company-info-logo"></div> <strong class="logo-small"><a href="/2021/schedule/">FOSDEM21</a></strong> <p>Online <span>/</span> <strong>6 & 7 February 2021</strong></p> </div> <div class="license-info"> <div class="license-info-logo"></div> This work is licensed under the Creative Commons Attribution 2.0 Belgium Licence.<br/> To view a copy of this licence, visit <a class="quiet" rel="license" href="http://creativecommons.org/licenses/by/2.0/be/deed.en">http://creativecommons.org/licenses/by/2.0/be/deed.en</a><br/> or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.<br /> All content such as talks and biographies is the sole responsibility of the speaker. </div> </div> </div> </body> </html>

CINXE.COM

FOSDEM 2021 - PostgreSQL Network Filter for EnvoyProxy