CINXE.COM

<!DOCTYPE html>   <html class="no-js" lang="en" >  <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>mitmproxy 2.0.2 documentation</title> <link rel="shortcut icon" href="_static/favicon.ico"/> <link rel="stylesheet" href="_static/css/theme.css" type="text/css" /> <link rel="stylesheet" href="_static/theme_overrides.css" type="text/css" /> <link rel="index" title="Index" href="genindex.html"/> <link rel="search" title="Search" href="search.html"/> <link rel="top" title="None" href="index.html#document-index"/> <script src="_static/js/modernizr.min.js"></script>   <link rel="canonical" href="http://docs.mitmproxy.org/en/v2.0.2/" /> <link rel="stylesheet" href="https://media.readthedocs.org/css/readthedocs-doc-embed.css" type="text/css" />  </head> <body class="wy-body-for-nav" role="document"> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search"> <a href="https://mitmproxy.org/" style="margin-bottom: 7px; background: none !important;"> <button class="btn btn-info"> <i class="fa fa-arrow-left"></i> Return to mitmproxy.org </button> </a> <a href="index.html#document-index"> <img src="_static/mitmproxy-docs.png" class="logo" /> </a> <div class="version"> v2.0.2 </div> </div> <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-introduction">Introduction</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-install">Installation</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-certinstall">About Certificates</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-howmitmproxy">How mitmproxy works</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-modes">Modes of Operation</a></li> </ul> <p class="caption"><span class="caption-text">Tools</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-mitmproxy">mitmproxy</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-mitmdump">mitmdump</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-mitmweb">mitmweb</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-config">Configuration</a></li> </ul> <p class="caption"><span class="caption-text">Features</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/anticache">Anticache</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/filters">Filter expressions</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/replacements">Replacements</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/clientreplay">Client-side replay</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/serverreplay">Server-side replay</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/setheaders">Set Headers</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/passthrough">Ignore Domains</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/proxyauth">Proxy Authentication</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/reverseproxy">Reverse Proxy</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/responsestreaming">Response Streaming</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/socksproxy">SOCKS Mode</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/sticky">Sticky cookies and auth</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/tcpproxy">TCP Proxy</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/upstreamproxy">Upstream proxy mode</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-features/upstreamcerts">Upstream Certificates</a></li> </ul> <p class="caption"><span class="caption-text">Transparent Proxying</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-transparent">Transparent Proxying</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-transparent/linux">Linux</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-transparent/osx">OSX</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-transparent/openbsd">OpenBSD</a></li> </ul> <p class="caption"><span class="caption-text">Scripting</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-scripting/overview">Overview</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-scripting/events">Events</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-scripting/api">API</a></li> </ul> <p class="caption"><span class="caption-text">Tutorials</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-tutorials/30second">Client playback: a 30 second example</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-tutorials/gamecenter">Setting highscores on Apple’s GameCenter</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-tutorials/transparent-dhcp">Transparently proxify virtual machines</a></li> </ul> <p class="caption"><span class="caption-text">Pathod & Pathoc</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-pathod/intro">Pathology 101</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-pathod/language">language spec</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-pathod/library">pathod library</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-pathod/test">pathod.test</a></li> </ul> <p class="caption"><span class="caption-text">Development</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="index.html#document-dev/contributing">Contributing</a></li> <li class="toctree-l1"><a class="reference internal" href="index.html#document-dev/sslkeylogfile">TLS Master Secrets</a></li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> <i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="index.html#document-index">mitmproxy docs</a> </nav> <div class="wy-nav-content"> <div class="rst-content"> <div role="navigation" aria-label="breadcrumbs navigation"> <ul class="wy-breadcrumbs"> <li><a href="index.html#document-index">Docs</a> »</li> <li>mitmproxy 2.0.2 documentation</li> </ul> <hr/> </div> <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <div class="section" id="introduction"> <h1>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">露</a></h1> <p><strong>mitmproxy</strong> is an interactive man-in-the-middle proxy for HTTP and HTTPS with a console interface.</p> <p><strong>mitmdump</strong> is the command-line version of mitmproxy. Think tcpdump for HTTP.</p> <p><strong>mitmweb</strong> is a web-based interface for mitmproxy.</p> <p>Documentation, tutorials and distribution packages can be found on the mitmproxy website: <a class="reference external" href="https://mitmproxy.org/">mitmproxy.org</a></p> <p class="rubric">Features</p> <ul class="simple"> <li>Intercept HTTP & HTTPS requests and responses and modify them on the fly</li> <li>Save complete HTTP conversations for later replay and analysis</li> <li>Replay the client-side of an HTTP conversations</li> <li>Replay HTTP responses of a previously recorded server</li> <li>Reverse proxy mode to forward traffic to a specified server</li> <li>Transparent proxy mode on OSX and Linux</li> <li>Make scripted changes to HTTP traffic using Python</li> <li>SSL/TLS certificates for interception are generated on the fly</li> <li>And much, much more...</li> </ul> <div class="toctree-wrapper compound"> <span id="document-introduction"></span><div class="section" id="introduction"> <h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">露</a></h2> <p><strong>mitmproxy</strong> is an interactive man-in-the-middle proxy for HTTP and HTTPS with a console interface.</p> <p><strong>mitmdump</strong> is the command-line version of mitmproxy. Think tcpdump for HTTP.</p> <p><strong>mitmweb</strong> is a web-based interface for mitmproxy.</p> <p>Documentation, tutorials and distribution packages can be found on the mitmproxy website: <a class="reference external" href="https://mitmproxy.org/">mitmproxy.org</a></p> <p class="rubric">Features</p> <ul class="simple"> <li>Intercept HTTP & HTTPS requests and responses and modify them on the fly</li> <li>Save complete HTTP conversations for later replay and analysis</li> <li>Replay the client-side of an HTTP conversations</li> <li>Replay HTTP responses of a previously recorded server</li> <li>Reverse proxy mode to forward traffic to a specified server</li> <li>Transparent proxy mode on OSX and Linux</li> <li>Make scripted changes to HTTP traffic using Python</li> <li>SSL/TLS certificates for interception are generated on the fly</li> <li>And much, much more...</li> </ul> </div> <span id="document-install"></span><div class="section" id="installation"> <span id="install"></span><h2>Installation<a class="headerlink" href="#installation" title="Permalink to this headline">露</a></h2> <p>Please follow the steps for your operating system.</p> <p>Once installation is complete, you can run <a class="reference internal" href="index.html#mitmproxy"><span class="std std-ref">mitmproxy</span></a>, <a class="reference internal" href="index.html#mitmdump"><span class="std std-ref">mitmdump</span></a> or <a class="reference internal" href="index.html#mitmweb"><span class="std std-ref">mitmweb</span></a> from a terminal.</p> <div class="section" id="installation-on-macos"> <span id="install-macos"></span><h3>Installation on macOS<a class="headerlink" href="#installation-on-macos" title="Permalink to this headline">露</a></h3> <p>You can use Homebrew to install everything:</p> <div class="code bash highlight-default"><div class="highlight"><pre><span></span><span class="n">brew</span> <span class="n">install</span> <span class="n">mitmproxy</span> </pre></div> </div> <p>Or you can download the pre-built binary packages from our <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/releases">releases</a>.</p> </div> <div class="section" id="installation-on-windows"> <span id="install-windows"></span><h3>Installation on Windows<a class="headerlink" href="#installation-on-windows" title="Permalink to this headline">露</a></h3> <p>The recommended way to install mitmproxy on Windows is to use the installer provided at <a class="reference external" href="https://mitmproxy.org/">mitmproxy.org</a>. After installation, you’ll find shortcuts for <a class="reference internal" href="index.html#mitmweb"><span class="std std-ref">mitmweb</span></a> (the web-based interface) and <a class="reference internal" href="index.html#mitmdump"><span class="std std-ref">mitmdump</span></a> in the start menu. Both executables are added to your PATH and can be invoked from the command line.</p> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">Mitmproxy’s console interface is not supported on Windows, but you can use mitmweb (the web-based interface) and mitmdump.</p> </div> </div> <div class="section" id="installation-on-linux"> <span id="install-linux"></span><h3>Installation on Linux<a class="headerlink" href="#installation-on-linux" title="Permalink to this headline">露</a></h3> <p>The recommended way to run mitmproxy on Linux is to use the pre-built binaries provided at <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/releases">releases</a>.</p> <p>Our pre-built binaries provide you with the latest version of mitmproxy, a self-contained Python 3.5 environment and a recent version of OpenSSL that supports HTTP/2. Of course, you can also install mitmproxy from source if you prefer that (see <a class="reference internal" href="#install-advanced"><span class="std std-ref">Advanced Installation</span></a>).</p> </div> <div class="section" id="advanced-installation"> <span id="install-advanced"></span><h3>Advanced Installation<a class="headerlink" href="#advanced-installation" title="Permalink to this headline">露</a></h3> <div class="section" id="docker-images"> <span id="install-docker"></span><h4>Docker Images<a class="headerlink" href="#docker-images" title="Permalink to this headline">露</a></h4> <p>You can also use the official mitmproxy images from <a class="reference external" href="https://hub.docker.com/r/mitmproxy/mitmproxy/">DockerHub</a>. That being said, our portable binaries are just as easy to install and even easier to use. 馃槉</p> </div> <div class="section" id="installation-on-arch-linux"> <span id="install-arch"></span><h4>Installation on Arch Linux<a class="headerlink" href="#installation-on-arch-linux" title="Permalink to this headline">露</a></h4> <p>mitmproxy has been added into the [community] repository. Use pacman to install it:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">pacman</span> <span class="o">-</span><span class="n">S</span> <span class="n">mitmproxy</span> </pre></div> </div> </div> <div class="section" id="installation-from-source-on-ubuntu"> <span id="install-source-ubuntu"></span><h4>Installation from Source on Ubuntu<a class="headerlink" href="#installation-from-source-on-ubuntu" title="Permalink to this headline">露</a></h4> <p>Ubuntu comes with Python but we need to install pip3, python3-dev and several libraries. This was tested on a fully patched installation of Ubuntu 16.04.</p> <div class="code bash highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">python3</span><span class="o">-</span><span class="n">dev</span> <span class="n">python3</span><span class="o">-</span><span class="n">pip</span> <span class="n">libffi</span><span class="o">-</span><span class="n">dev</span> <span class="n">libssl</span><span class="o">-</span><span class="n">dev</span> <span class="n">sudo</span> <span class="n">pip3</span> <span class="n">install</span> <span class="n">mitmproxy</span> <span class="c1"># or pip3 install --user mitmproxy</span> </pre></div> </div> <p>On older Ubuntu versions, e.g., <strong>12.04</strong> and <strong>14.04</strong>, you may need to install a newer version of Python. mitmproxy requires Python 3.5 or higher. Please take a look at <a class="reference external" href="https://github.com/yyuu/pyenv">pyenv</a>. Make sure to have an up-to-date version of pip by running <code class="docutils literal"><span class="pre">pip3</span> <span class="pre">install</span> <span class="pre">-U</span> <span class="pre">pip</span></code>.</p> </div> <div class="section" id="installation-from-source-on-fedora"> <span id="install-source-fedora"></span><h4>Installation from Source on Fedora<a class="headerlink" href="#installation-from-source-on-fedora" title="Permalink to this headline">露</a></h4> <p>Fedora comes with Python but we need to install pip3, python3-dev and several libraries. This was tested on a fully patched installation of Fedora 24.</p> <div class="code bash highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">dnf</span> <span class="n">install</span> <span class="n">make</span> <span class="n">gcc</span> <span class="n">redhat</span><span class="o">-</span><span class="n">rpm</span><span class="o">-</span><span class="n">config</span> <span class="n">python3</span><span class="o">-</span><span class="n">devel</span> <span class="n">python3</span><span class="o">-</span><span class="n">pip</span> <span class="n">libffi</span><span class="o">-</span><span class="n">devel</span> <span class="n">openssl</span><span class="o">-</span><span class="n">devel</span> <span class="n">sudo</span> <span class="n">pip3</span> <span class="n">install</span> <span class="n">mitmproxy</span> <span class="c1"># or pip3 install --user mitmproxy</span> </pre></div> </div> <p>Make sure to have an up-to-date version of pip by running <code class="docutils literal"><span class="pre">pip3</span> <span class="pre">install</span> <span class="pre">-U</span> <span class="pre">pip</span></code>.</p> </div> <div class="section" id="installation-from-source-on-windows"> <span id="install-source-windows"></span><h4>馃惐馃捇 Installation from Source on Windows<a class="headerlink" href="#installation-from-source-on-windows" title="Permalink to this headline">露</a></h4> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">Mitmproxy’s console interface is not supported on Windows, but you can use mitmweb (the web-based interface) and mitmdump.</p> </div> <p>First, install the latest version of Python 3.5 or later from the <a class="reference external" href="https://www.python.org/downloads/windows/">Python website</a>. During installation, make sure to select <cite>Add Python to PATH</cite>.</p> <p>Mitmproxy has no other dependencies on Windows. You can now install mitmproxy by running</p> <div class="code powershell highlight-default"><div class="highlight"><pre><span></span><span class="n">pip3</span> <span class="n">install</span> <span class="n">mitmproxy</span> </pre></div> </div> </div> <div class="section" id="latest-development-version"> <span id="install-dev-version"></span><h4>Latest Development Version<a class="headerlink" href="#latest-development-version" title="Permalink to this headline">露</a></h4> <p>If you would like to install mitmproxy directly from the master branch on GitHub or would like to get set up to contribute to the project, install the dependencies as you would for a regular installation from source. Then see the project’s <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/master/README.rst">README</a> on GitHub. You can check your system information by running: <code class="docutils literal"><span class="pre">mitmproxy</span> <span class="pre">--version</span></code></p> </div> </div> </div> <span id="document-certinstall"></span><div class="section" id="about-certificates"> <span id="certinstall"></span><h2>About Certificates<a class="headerlink" href="#about-certificates" title="Permalink to this headline">露</a></h2> <div class="section" id="introduction"> <h3>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">露</a></h3> <p>Mitmproxy can decrypt encrypted traffic on the fly, as long as the client trusts its built-in certificate authority. Usually this means that the mitmproxy CA certificates have to be installed on the client device.</p> </div> <div class="section" id="quick-setup"> <h3>Quick Setup<a class="headerlink" href="#quick-setup" title="Permalink to this headline">露</a></h3> <p>By far the easiest way to install the mitmproxy certificates is to use the built-in certificate installation app. To do this, just start mitmproxy and configure your target device with the correct proxy settings. Now start a browser on the device, and visit the magic domain <strong>mitm.it</strong>. You should see something like this:</p> <img alt="_images/certinstall-webapp.png" src="_images/certinstall-webapp.png" /> <p>Click on the relevant icon, follow the setup instructions for the platform you’re on and you are good to go.</p> </div> <div class="section" id="installing-the-mitmproxy-ca-certificate-manually"> <h3>Installing the mitmproxy CA certificate manually<a class="headerlink" href="#installing-the-mitmproxy-ca-certificate-manually" title="Permalink to this headline">露</a></h3> <p>Sometimes using the quick install app is not an option - Java or the iOS Simulator spring to mind - or you just need to do it manually for some other reason. Below is a list of pointers to manual certificate installation documentation for some common platforms.</p> <p>The mitmproxy CA cert is located in <code class="docutils literal"><span class="pre">~/.mitmproxy</span></code> after it has been generated at the first start of mitmproxy.</p> <div class="section" id="ios"> <h4>iOS<a class="headerlink" href="#ios" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="http://jasdev.me/intercepting-ios-traffic">http://jasdev.me/intercepting-ios-traffic</a></p> <p>and <a class="reference external" href="http://web.archive.org/web/20150920082614/http://kb.mit.edu/confluence/pages/viewpage.action?pageId=152600377">http://web.archive.org/web/20150920082614/http://kb.mit.edu/confluence/pages/viewpage.action?pageId=152600377</a></p> </div> <div class="section" id="ios-simulator"> <h4>iOS Simulator<a class="headerlink" href="#ios-simulator" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="https://github.com/ADVTOOLS/ADVTrustStore#how-to-use-advtruststore">https://github.com/ADVTOOLS/ADVTrustStore#how-to-use-advtruststore</a></p> </div> <div class="section" id="java"> <h4>Java<a class="headerlink" href="#java" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="http://docs.oracle.com/cd/E19906-01/820-4916/geygn/index.html">http://docs.oracle.com/cd/E19906-01/820-4916/geygn/index.html</a></p> </div> <div class="section" id="android-android-simulator"> <h4>Android/Android Simulator<a class="headerlink" href="#android-android-simulator" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="http://wiki.cacert.org/FAQ/ImportRootCert#Android_Phones_.26_Tablets">http://wiki.cacert.org/FAQ/ImportRootCert#Android_Phones_.26_Tablets</a></p> </div> <div class="section" id="windows"> <h4>Windows<a class="headerlink" href="#windows" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="http://windows.microsoft.com/en-ca/windows/import-export-certificates-private-keys#1TC=windows-7">http://windows.microsoft.com/en-ca/windows/import-export-certificates-private-keys#1TC=windows-7</a></p> </div> <div class="section" id="windows-automated"> <h4>Windows (automated)<a class="headerlink" href="#windows-automated" title="Permalink to this headline">露</a></h4> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">certutil</span><span class="o">.</span><span class="n">exe</span> <span class="o">-</span><span class="n">importpfx</span> <span class="n">Root</span> <span class="n">mitmproxy</span><span class="o">-</span><span class="n">ca</span><span class="o">-</span><span class="n">cert</span><span class="o">.</span><span class="n">p12</span> </pre></div> </div> <p>See also: <a class="reference external" href="https://technet.microsoft.com/en-us/library/cc732443.aspx">https://technet.microsoft.com/en-us/library/cc732443.aspx</a></p> </div> <div class="section" id="mac-os-x"> <h4>Mac OS X<a class="headerlink" href="#mac-os-x" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="https://support.apple.com/kb/PH7297?locale=en_US">https://support.apple.com/kb/PH7297?locale=en_US</a></p> </div> <div class="section" id="ubuntu-debian"> <h4>Ubuntu/Debian<a class="headerlink" href="#ubuntu-debian" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="http://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/94861#94861">http://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/94861#94861</a></p> </div> <div class="section" id="mozilla-firefox"> <h4>Mozilla Firefox<a class="headerlink" href="#mozilla-firefox" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="https://wiki.mozilla.org/MozillaRootCertificate#Mozilla_Firefox">https://wiki.mozilla.org/MozillaRootCertificate#Mozilla_Firefox</a></p> </div> <div class="section" id="chrome-on-linux"> <h4>Chrome on Linux<a class="headerlink" href="#chrome-on-linux" title="Permalink to this headline">露</a></h4> <p>See <a class="reference external" href="https://code.google.com/p/chromium/wiki/LinuxCertManagement">https://code.google.com/p/chromium/wiki/LinuxCertManagement</a></p> </div> </div> <div class="section" id="the-mitmproxy-certificate-authority"> <h3>The mitmproxy certificate authority<a class="headerlink" href="#the-mitmproxy-certificate-authority" title="Permalink to this headline">露</a></h3> <p>The first time <strong>mitmproxy</strong> or <strong>mitmdump</strong> is run, the mitmproxy Certificate Authority (CA) is created in the config directory (<code class="docutils literal"><span class="pre">~/.mitmproxy</span></code> by default). This CA is used for on-the-fly generation of dummy certificates for each of the SSL sites that your client visits. Since your browser won’t trust the mitmproxy CA out of the box, you will see an SSL certificate warning every time you visit a new SSL domain through mitmproxy. When you are testing a single site through a browser, just accepting the bogus SSL cert manually is not too much trouble, but there are a many circumstances where you will want to configure your testing system or browser to trust the mitmproxy CA as a signing root authority. For security reasons, the mitmproxy CA is generated uniquely on the first start and is not shared between mitmproxy installations on different devices.</p> <div class="section" id="certificate-pinning"> <h4>Certificate Pinning<a class="headerlink" href="#certificate-pinning" title="Permalink to this headline">露</a></h4> <p>Some applications employ <a class="reference external" href="http://security.stackexchange.com/questions/29988/what-is-certificate-pinning/">Certificate Pinning</a> to prevent man-in-the-middle attacks. This means that <strong>mitmproxy</strong> and <strong>mitmdump’s</strong> certificates will not be accepted by these applications without modifying them. It is recommended to use the <a class="reference internal" href="index.html#passthrough"><span class="std std-ref">Ignore Domains</span></a> feature in order to prevent <strong>mitmproxy</strong> and <strong>mitmdump</strong> from intercepting traffic to these specific domains. If you want to intercept the pinned connections, you need to patch the application manually. For Android and (jailbroken) iOS devices, various tools exist to accomplish this.</p> </div> </div> <div class="section" id="ca-and-cert-files"> <h3>CA and cert files<a class="headerlink" href="#ca-and-cert-files" title="Permalink to this headline">露</a></h3> <p>The files created by mitmproxy in the .mitmproxy directory are as follows:</p> <table border="1" class="docutils"> <colgroup> <col width="22%" /> <col width="78%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>mitmproxy-ca.pem</td> <td>The certificate <strong>and the private key</strong> in PEM format.</td> </tr> <tr class="row-even"><td>mitmproxy-ca-cert.pem</td> <td>The certificate in PEM format. Use this to distribute on most non-Windows platforms.</td> </tr> <tr class="row-odd"><td>mitmproxy-ca-cert.p12</td> <td>The certificate in PKCS12 format. For use on Windows.</td> </tr> <tr class="row-even"><td>mitmproxy-ca-cert.cer</td> <td>Same file as .pem, but with an extension expected by some Android devices.</td> </tr> </tbody> </table> </div> <div class="section" id="using-a-custom-certificate"> <h3>Using a custom certificate<a class="headerlink" href="#using-a-custom-certificate" title="Permalink to this headline">露</a></h3> <p>You can use your own certificate by passing the <code class="docutils literal"><span class="pre">--cert</span> <span class="pre">[domain=]path_to_certificate</span></code> option to mitmproxy. Mitmproxy then uses the provided certificate for interception of the specified domain instead of generating a certificate signed by its own CA.</p> <p>The certificate file is expected to be in the PEM format. You can include intermediary certificates right below your leaf certificate, so that your PEM file roughly looks like this:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>-----BEGIN PRIVATE KEY----- <private key> -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- <cert> -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- <intermediary cert (optional)> -----END CERTIFICATE----- </pre></div> </div> <p>For example, you can generate a certificate in this format using these instructions:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">openssl</span> <span class="n">genrsa</span> <span class="o">-</span><span class="n">out</span> <span class="n">cert</span><span class="o">.</span><span class="n">key</span> <span class="mi">2048</span> <span class="gp">>>> </span><span class="n">openssl</span> <span class="n">req</span> <span class="o">-</span><span class="n">new</span> <span class="o">-</span><span class="n">x509</span> <span class="o">-</span><span class="n">key</span> <span class="n">cert</span><span class="o">.</span><span class="n">key</span> <span class="o">-</span><span class="n">out</span> <span class="n">cert</span><span class="o">.</span><span class="n">crt</span> <span class="go"> (Specify the mitm domain as Common Name, e.g. *.google.com)</span> <span class="gp">>>> </span><span class="n">cat</span> <span class="n">cert</span><span class="o">.</span><span class="n">key</span> <span class="n">cert</span><span class="o">.</span><span class="n">crt</span> <span class="o">></span> <span class="n">cert</span><span class="o">.</span><span class="n">pem</span> </pre></div> </div> <p>Now, you can run mitmproxy with the generated certificate:</p> <p><strong>For all domain names</strong></p> <p><code class="docutils literal"><span class="pre">>>>mitmproxy</span> <span class="pre">--cert</span> <span class="pre">*=cert.pem</span></code></p> <p><strong>For specific domain names</strong></p> <p><code class="docutils literal"><span class="pre">>>>mitmproxy</span> <span class="pre">--cert</span> <span class="pre">*.example.com=cert.pem</span></code></p> <p><strong>Note:</strong> <code class="docutils literal"><span class="pre">*.example.com</span></code> is for all the subdomains. You can also use <code class="docutils literal"><span class="pre">www.example.com</span></code> for a particular subdomain.</p> </div> <div class="section" id="using-a-custom-certificate-authority"> <h3>Using a custom certificate authority<a class="headerlink" href="#using-a-custom-certificate-authority" title="Permalink to this headline">露</a></h3> <p>By default, mitmproxy will use <code class="docutils literal"><span class="pre">~/.mitmproxy/mitmproxy-ca.pem</span></code> as the certificate authority to generate certificates for all domains for which no custom certificate is provided (see above). You can use your own certificate authority by passing the <code class="docutils literal"><span class="pre">--cadir</span> <span class="pre">DIRECTORY</span></code> option to mitmproxy. Mitmproxy will then look for <code class="docutils literal"><span class="pre">mitmproxy-ca.pem</span></code> in the specified directory. If no such file exists, it will be generated automatically.</p> </div> <div class="section" id="using-a-client-side-certificate"> <h3>Using a client side certificate<a class="headerlink" href="#using-a-client-side-certificate" title="Permalink to this headline">露</a></h3> <p>You can use a client certificate by passing the <code class="docutils literal"><span class="pre">--client-certs</span> <span class="pre">DIRECTORY|FILE</span></code> option to mitmproxy. Using a directory allows certs to be selected based on hostname, while using a filename allows a single specific certificate to be used for all SSL connections. Certificate files must be in the PEM format and should contain both the unencrypted private key and the certificate.</p> <div class="section" id="multiple-certs-by-hostname"> <h4>Multiple certs by Hostname<a class="headerlink" href="#multiple-certs-by-hostname" title="Permalink to this headline">露</a></h4> <p>If you’ve specified a directory to <code class="docutils literal"><span class="pre">--client-certs</span></code>, then the following behavior will be taken:</p> <p>If you visit example.org, mitmproxy looks for a file named <code class="docutils literal"><span class="pre">example.org.pem</span></code> in the specified directory and uses this as the client cert.</p> </div> </div> </div> <span id="document-howmitmproxy"></span><div class="section" id="how-mitmproxy-works"> <h2>How mitmproxy works<a class="headerlink" href="#how-mitmproxy-works" title="Permalink to this headline">露</a></h2> <p>Mitmproxy is an enormously flexible tool. Knowing exactly how the proxying process works will help you deploy it creatively, and take into account its fundamental assumptions and how to work around them. This document explains mitmproxy’s proxy mechanism in detail, starting with the simplest unencrypted explicit proxying, and working up to the most complicated interaction - transparent proxying of TLS-protected traffic <a class="footnote-reference" href="#tls" id="id1">[1]</a> in the presence of <a class="reference external" href="https://en.wikipedia.org/wiki/Server_Name_Indication">Server Name Indication</a>.</p> <div class="section" id="explicit-http"> <h3>Explicit HTTP<a class="headerlink" href="#explicit-http" title="Permalink to this headline">露</a></h3> <p>Configuring the client to use mitmproxy as an explicit proxy is the simplest and most reliable way to intercept traffic. The proxy protocol is codified in the <a class="reference external" href="https://tools.ietf.org/html/rfc7230">HTTP RFC</a>, so the behaviour of both the client and the server is well defined, and usually reliable. In the simplest possible interaction with mitmproxy, a client connects directly to the proxy, and makes a request that looks like this:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>GET http://example.com/index.html HTTP/1.1 </pre></div> </div> <p>This is a proxy GET request - an extended form of the vanilla HTTP GET request that includes a schema and host specification, and it includes all the information mitmproxy needs to proceed.</p> <img alt="_images/how-mitmproxy-works-explicit.png" class="align-center" src="_images/how-mitmproxy-works-explicit.png" /> <ol class="arabic simple"> <li>The client connects to the proxy and makes a request.</li> <li>Mitmproxy connects to the upstream server and simply forwards the request on.</li> </ol> </div> <div class="section" id="explicit-https"> <h3>Explicit HTTPS<a class="headerlink" href="#explicit-https" title="Permalink to this headline">露</a></h3> <p>The process for an explicitly proxied HTTPS connection is quite different. The client connects to the proxy and makes a request that looks like this:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>CONNECT example.com:443 HTTP/1.1 </pre></div> </div> <p>A conventional proxy can neither view nor manipulate an TLS-encrypted data stream, so a CONNECT request simply asks the proxy to open a pipe between the client and server. The proxy here is just a facilitator - it blindly forwards data in both directions without knowing anything about the contents. The negotiation of the TLS connection happens over this pipe, and the subsequent flow of requests and responses are completely opaque to the proxy.</p> <div class="section" id="the-mitm-in-mitmproxy"> <h4>The MITM in mitmproxy<a class="headerlink" href="#the-mitm-in-mitmproxy" title="Permalink to this headline">露</a></h4> <p>This is where mitmproxy’s fundamental trick comes into play. The MITM in its name stands for Man-In-The-Middle - a reference to the process we use to intercept and interfere with these theoretically opaque data streams. The basic idea is to pretend to be the server to the client, and pretend to be the client to the server, while we sit in the middle decoding traffic from both sides. The tricky part is that the <a class="reference external" href="https://en.wikipedia.org/wiki/Certificate_authority">Certificate Authority</a> system is designed to prevent exactly this attack, by allowing a trusted third-party to cryptographically sign a server’s certificates to verify that they are legit. If this signature doesn’t match or is from a non-trusted party, a secure client will simply drop the connection and refuse to proceed. Despite the many shortcomings of the CA system as it exists today, this is usually fatal to attempts to MITM an TLS connection for analysis. Our answer to this conundrum is to become a trusted Certificate Authority ourselves. Mitmproxy includes a full CA implementation that generates interception certificates on the fly. To get the client to trust these certificates, we <a class="reference internal" href="index.html#certinstall"><span class="std std-ref">register mitmproxy as a trusted CA with the device manually</span></a>.</p> </div> <div class="section" id="complication-1-what-s-the-remote-hostname"> <h4>Complication 1: What’s the remote hostname?<a class="headerlink" href="#complication-1-what-s-the-remote-hostname" title="Permalink to this headline">露</a></h4> <p>To proceed with this plan, we need to know the domain name to use in the interception certificate - the client will verify that the certificate is for the domain it’s connecting to, and abort if this is not the case. At first blush, it seems that the CONNECT request above gives us all we need - in this example, both of these values are “example.com”. But what if the client had initiated the connection as follows:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>CONNECT 10.1.1.1:443 HTTP/1.1 </pre></div> </div> <p>Using the IP address is perfectly legitimate because it gives us enough information to initiate the pipe, even though it doesn’t reveal the remote hostname.</p> <p>Mitmproxy has a cunning mechanism that smooths this over - <a class="reference internal" href="index.html#upstreamcerts"><span class="std std-ref">upstream certificate sniffing</span></a>. As soon as we see the CONNECT request, we pause the client part of the conversation, and initiate a simultaneous connection to the server. We complete the TLS handshake with the server, and inspect the certificates it used. Now, we use the Common Name in the upstream certificates to generate the dummy certificate for the client. Voila, we have the correct hostname to present to the client, even if it was never specified.</p> </div> <div class="section" id="complication-2-subject-alternative-name"> <h4>Complication 2: Subject Alternative Name<a class="headerlink" href="#complication-2-subject-alternative-name" title="Permalink to this headline">露</a></h4> <p>Enter the next complication. Sometimes, the certificate Common Name is not, in fact, the hostname that the client is connecting to. This is because of the optional <a class="reference external" href="https://en.wikipedia.org/wiki/SubjectAltName">Subject Alternative Name</a> field in the certificate that allows an arbitrary number of alternative domains to be specified. If the expected domain matches any of these, the client will proceed, even though the domain doesn’t match the certificate CN. The answer here is simple: when we extract the CN from the upstream cert, we also extract the SANs, and add them to the generated dummy certificate.</p> </div> <div class="section" id="complication-3-server-name-indication"> <h4>Complication 3: Server Name Indication<a class="headerlink" href="#complication-3-server-name-indication" title="Permalink to this headline">露</a></h4> <p>One of the big limitations of vanilla TLS is that each certificate requires its own IP address. This means that you couldn’t do virtual hosting where multiple domains with independent certificates share the same IP address. In a world with a rapidly shrinking IPv4 address pool this is a problem, and we have a solution in the form of the <a class="reference external" href="https://en.wikipedia.org/wiki/Server_Name_Indication">Server Name Indication</a> extension to the TLS protocols. This lets the client specify the remote server name at the start of the TLS handshake, which then lets the server select the right certificate to complete the process.</p> <p>SNI breaks our upstream certificate sniffing process, because when we connect without using SNI, we get served a default certificate that may have nothing to do with the certificate expected by the client. The solution is another tricky complication to the client connection process. After the client connects, we allow the TLS handshake to continue until just <strong>after</strong> the SNI value has been passed to us. Now we can pause the conversation, and initiate an upstream connection using the correct SNI value, which then serves us the correct upstream certificate, from which we can extract the expected CN and SANs.</p> </div> <div class="section" id="putting-it-all-together"> <h4>Putting it all together<a class="headerlink" href="#putting-it-all-together" title="Permalink to this headline">露</a></h4> <p>Lets put all of this together into the complete explicitly proxied HTTPS flow.</p> <img alt="_images/how-mitmproxy-works-explicit-https.png" class="align-center" src="_images/how-mitmproxy-works-explicit-https.png" /> <ol class="arabic simple"> <li>The client makes a connection to mitmproxy, and issues an HTTP CONNECT request.</li> <li>Mitmproxy responds with a <code class="docutils literal"><span class="pre">200</span> <span class="pre">Connection</span> <span class="pre">Established</span></code>, as if it has set up the CONNECT pipe.</li> <li>The client believes it’s talking to the remote server, and initiates the TLS connection. It uses SNI to indicate the hostname it is connecting to.</li> <li>Mitmproxy connects to the server, and establishes an TLS connection using the SNI hostname indicated by the client.</li> <li>The server responds with the matching certificate, which contains the CN and SAN values needed to generate the interception certificate.</li> <li>Mitmproxy generates the interception cert, and continues the client TLS handshake paused in step 3.</li> <li>The client sends the request over the established TLS connection.</li> <li>Mitmproxy passes the request on to the server over the TLS connection initiated in step 4.</li> </ol> </div> </div> <div class="section" id="transparent-http"> <h3>Transparent HTTP<a class="headerlink" href="#transparent-http" title="Permalink to this headline">露</a></h3> <p>When a transparent proxy is used, the connection is redirected into a proxy at the network layer, without any client configuration being required. This makes transparent proxying ideal for those situations where you can’t change client behaviour - proxy-oblivious Android applications being a common example.</p> <p>To achieve this, we need to introduce two extra components. The first is a redirection mechanism that transparently reroutes a TCP connection destined for a server on the Internet to a listening proxy server. This usually takes the form of a firewall on the same host as the proxy server - <a class="reference external" href="http://www.netfilter.org/">iptables</a> on Linux or <a class="reference external" href="https://en.wikipedia.org/wiki/PF_(firewall)">pf</a> on OSX. Once the client has initiated the connection, it makes a vanilla HTTP request, which might look something like this:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>GET /index.html HTTP/1.1 </pre></div> </div> <p>Note that this request differs from the explicit proxy variation, in that it omits the scheme and hostname. How, then, do we know which upstream host to forward the request to? The routing mechanism that has performed the redirection keeps track of the original destination for us. Each routing mechanism has a different way of exposing this data, so this introduces the second component required for working transparent proxying: a host module that knows how to retrieve the original destination address from the router. In mitmproxy, this takes the form of a built-in set of <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/tree/master/mitmproxy/platform">modules</a> that know how to talk to each platform’s redirection mechanism. Once we have this information, the process is fairly straight-forward.</p> <img alt="_images/how-mitmproxy-works-transparent.png" class="align-center" src="_images/how-mitmproxy-works-transparent.png" /> <ol class="arabic simple"> <li>The client makes a connection to the server.</li> <li>The router redirects the connection to mitmproxy, which is typically listening on a local port of the same host. Mitmproxy then consults the routing mechanism to establish what the original destination was.</li> <li>Now, we simply read the client’s request...</li> <li>... and forward it upstream.</li> </ol> </div> <div class="section" id="transparent-https"> <h3>Transparent HTTPS<a class="headerlink" href="#transparent-https" title="Permalink to this headline">露</a></h3> <p>The first step is to determine whether we should treat an incoming connection as HTTPS. The mechanism for doing this is simple - we use the routing mechanism to find out what the original destination port is. All incoming connections pass through different layers which can determin the actual protocol to use. Automatic TLS detection works for SSLv3, TLS 1.0, TLS 1.1, and TLS 1.2 by looking for a <em>ClientHello</em> message at the beginning of each connection. This works independently of the used TCP port.</p> <p>From here, the process is a merger of the methods we’ve described for transparently proxying HTTP, and explicitly proxying HTTPS. We use the routing mechanism to establish the upstream server address, and then proceed as for explicit HTTPS connections to establish the CN and SANs, and cope with SNI.</p> <img alt="_images/how-mitmproxy-works-transparent-https.png" class="align-center" src="_images/how-mitmproxy-works-transparent-https.png" /> <ol class="arabic simple"> <li>The client makes a connection to the server.</li> <li>The router redirects the connection to mitmproxy, which is typically listening on a local port of the same host. Mitmproxy then consults the routing mechanism to establish what the original destination was.</li> <li>The client believes it’s talking to the remote server, and initiates the TLS connection. It uses SNI to indicate the hostname it is connecting to.</li> <li>Mitmproxy connects to the server, and establishes an TLS connection using the SNI hostname indicated by the client.</li> <li>The server responds with the matching certificate, which contains the CN and SAN values needed to generate the interception certificate.</li> <li>Mitmproxy generates the interception cert, and continues the client TLS handshake paused in step 3.</li> <li>The client sends the request over the established TLS connection.</li> <li>Mitmproxy passes the request on to the server over the TLS connection initiated in step 4.</li> </ol> <p class="rubric">Footnotes</p> <table class="docutils footnote" frame="void" id="tls" rules="none"> <colgroup><col class="label" /><col /></colgroup> <tbody valign="top"> <tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td>The use of “TLS” refers to both SSL (outdated and insecure) and TLS (1.0 and up) in the generic sense, unless otherwise specified.</td></tr> </tbody> </table> </div> </div> <span id="document-modes"></span><div class="section" id="modes-of-operation"> <span id="modes"></span><h2>Modes of Operation<a class="headerlink" href="#modes-of-operation" title="Permalink to this headline">露</a></h2> <p>Mitmproxy has four modes of operation that allow you to use mitmproxy in a variety of scenarios:</p> <ul class="simple"> <li><strong>Regular</strong> (the default)</li> <li><strong>Transparent</strong></li> <li><strong>Reverse Proxy</strong></li> <li><strong>Upstream Proxy</strong></li> </ul> <p>Now, which one should you pick? Use this flow chart:</p> <img alt="_images/proxy-modes-flowchart.png" class="align-center" src="_images/proxy-modes-flowchart.png" /> <div class="section" id="regular-proxy"> <h3>Regular Proxy<a class="headerlink" href="#regular-proxy" title="Permalink to this headline">露</a></h3> <p>Mitmproxy’s regular mode is the simplest and the easiest to set up.</p> <ol class="arabic simple"> <li>Start mitmproxy.</li> <li>Configure your client to use mitmproxy by explicitly setting an HTTP proxy.</li> <li>Quick Check: You should already be able to visit an unencrypted HTTP site through the proxy.</li> <li>Open the magic domain <strong>mitm.it</strong> and install the certificate for your device.</li> </ol> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">Unfortunately, some applications bypass the system HTTP proxy settings - Android applications are a common example. In these cases, you need to use mitmproxy’s transparent mode.</p> </div> <p>If you are proxying an external device, your network will probably look like this:</p> <img alt="_images/proxy-modes-regular.png" class="align-center" src="_images/proxy-modes-regular.png" /> <p>The square brackets signify the source and destination IP addresses. Your client explicitly connects to mitmproxy and mitmproxy explicitly connects to the target server.</p> </div> <div class="section" id="transparent-proxy"> <h3>Transparent Proxy<a class="headerlink" href="#transparent-proxy" title="Permalink to this headline">露</a></h3> <p>In transparent mode, traffic is directed into a proxy at the network layer, without any client configuration required. This makes transparent proxying ideal for situations where you can’t change client behaviour. In the graphic below, a machine running mitmproxy has been inserted between the router and the internet:</p> <img alt="_images/proxy-modes-transparent-1.png" class="align-center" src="_images/proxy-modes-transparent-1.png" /> <p>The square brackets signify the source and destination IP addresses. Round brackets mark the next hop on the <em>Ethernet/data link</em> layer. This distinction is important: when the packet arrives at the mitmproxy machine, it must still be addressed to the target server. This means that Network Address Translation should not be applied before the traffic reaches mitmproxy, since this would remove the target information, leaving mitmproxy unable to determine the real destination.</p> <img alt="_images/proxy-modes-transparent-wrong.png" class="align-center" src="_images/proxy-modes-transparent-wrong.png" /> <div class="section" id="common-configurations"> <h4>Common Configurations<a class="headerlink" href="#common-configurations" title="Permalink to this headline">露</a></h4> <p>There are many ways to configure your network for transparent proxying. We’ll look at two common scenarios:</p> <ol class="arabic simple"> <li>Configuring the client to use a custom gateway/router/”next hop”</li> <li>Implementing custom routing on the router</li> </ol> <p>In most cases, the first option is recommended due to its ease of use.</p> <div class="section" id="a-custom-gateway"> <h5>(a) Custom Gateway<a class="headerlink" href="#a-custom-gateway" title="Permalink to this headline">露</a></h5> <p>One simple way to get traffic to the mitmproxy machine with the destination IP intact, is to simply configure the client with the mitmproxy box as the default gateway.</p> <img alt="_images/proxy-modes-transparent-2.png" class="align-center" src="_images/proxy-modes-transparent-2.png" /> <p>In this scenario, we would:</p> <ol class="arabic simple"> <li>Configure the proxy machine for transparent mode. You can find instructions in the <a class="reference internal" href="index.html#transparent"><span class="std std-ref">Transparent Proxying</span></a> section.</li> <li>Configure the client to use the proxy machine’s IP as the default gateway.</li> <li>Quick Check: At this point, you should already be able to visit an unencrypted HTTP site over the proxy.</li> <li>Open the magic domain <strong>mitm.it</strong> and install the certificate for your device.</li> </ol> <p>Setting the custom gateway on clients can be automated by serving the settings out to clients over DHCP. This lets set up an interception network where all clients are proxied automatically, which can save time and effort.</p> <div class="note admonition"> <p class="first admonition-title">Troubleshooting Transparent Mode</p> <p>Incorrect transparent mode configurations are a frequent source of error. If it doesn’t work for you, try the following things:</p> <ul class="simple"> <li>Open mitmproxy’s event log (press <code class="kbd docutils literal"><span class="pre">e</span></code>) - do you see clientconnect messages? If not, the packets are not arriving at the proxy. One common cause is the occurrence of ICMP redirects, which means that your machine is telling the client that there’s a faster way to the internet by contacting your router directly (see the <a class="reference internal" href="index.html#transparent"><span class="std std-ref">Transparent Proxying</span></a> section on how to disable them). If in doubt, <a class="reference external" href="https://wireshark.org/">Wireshark</a> may help you to see whether something arrives at your machine or not.</li> <li>Make sure you have not explicitly configured an HTTP proxy on the client. This is not needed in transparent mode.</li> <li>Re-check the instructions in the <a class="reference internal" href="index.html#transparent"><span class="std std-ref">Transparent Proxying</span></a> section. Anything you missed?</li> </ul> <p class="last">If you encounter any other pitfalls that should be listed here, please let us know!</p> </div> </div> <div class="section" id="b-custom-routing"> <h5>(b) Custom Routing<a class="headerlink" href="#b-custom-routing" title="Permalink to this headline">露</a></h5> <p>In some cases, you may need more fine-grained control of which traffic reaches the mitmproxy instance, and which doesn’t. You may, for instance, choose only to divert traffic to some hosts into the transparent proxy. There are a huge number of ways to accomplish this, and much will depend on the router or packet filter you’re using. In most cases, the configuration will look like this:</p> <img alt="_images/proxy-modes-transparent-3.png" class="align-center" src="_images/proxy-modes-transparent-3.png" /> </div> </div> </div> <div class="section" id="reverse-proxy"> <h3>Reverse Proxy<a class="headerlink" href="#reverse-proxy" title="Permalink to this headline">露</a></h3> <p>mitmproxy is usually used with a client that uses the proxy to access the Internet. Using reverse proxy mode, you can use mitmproxy to act like a normal HTTP server:</p> <img alt="_images/proxy-modes-reverse.png" class="align-center" src="_images/proxy-modes-reverse.png" /> <p>There are various use-cases:</p> <ul class="simple"> <li>Say you have an internal API running at <a class="reference external" href="http://example.local/">http://example.local/</a>. You could now set up mitmproxy in reverse proxy mode at <a class="reference external" href="http://debug.example.local/">http://debug.example.local/</a> and dynamically point clients to this new API endpoint, which provides them with the same data and you with debug information. Similarly, you could move your real server to a different IP/port and set up mitmproxy in the original place to debug and or redirect all sessions.</li> <li>Say you’re a web developer working on <a class="reference external" href="http://example.com/">http://example.com/</a> (with a development version running on <a class="reference external" href="http://localhost:8000/">http://localhost:8000/</a>). You can modify your hosts file so that example.com points to 127.0.0.1 and then run mitmproxy in reverse proxy mode on port 80. You can test your app on the example.com domain and get all requests recorded in mitmproxy.</li> <li>Say you have some toy project that should get SSL support. Simply set up mitmproxy as a reverse proxy on port 443 and you’re done (<code class="docutils literal"><span class="pre">mitmdump</span> <span class="pre">-p</span> <span class="pre">443</span> <span class="pre">-R</span> <span class="pre">http://localhost:80/</span></code>). Mitmproxy auto-detects TLS traffic and intercepts it dynamically. There are better tools for this specific task, but mitmproxy is very quick and simple way to set up an SSL-speaking server.</li> <li>Want to add a non-SSL-capable compression proxy in front of your server? You could even spawn a mitmproxy instance that terminates SSL (<code class="docutils literal"><span class="pre">-R</span> <span class="pre">http://...</span></code>), point it to the compression proxy and let the compression proxy point to a SSL-initiating mitmproxy (<code class="docutils literal"><span class="pre">-R</span> <span class="pre">https://...</span></code>), which then points to the real server. As you see, it’s a fairly flexible thing.</li> </ul> <div class="warning admonition"> <p class="first admonition-title">Caveat: Interactive Use</p> <p class="last">Reverse Proxy mode is usually not sufficient to create a copy of an interactive website at different URL. The HTML served to the client remains unchanged - as soon as the user clicks on an non-relative URL (or downloads a non-relative image resource), traffic no longer passes through mitmproxy.</p> </div> </div> <div class="section" id="upstream-proxy"> <h3>Upstream Proxy<a class="headerlink" href="#upstream-proxy" title="Permalink to this headline">露</a></h3> <p>If you want to chain proxies by adding mitmproxy in front of a different proxy appliance, you can use mitmproxy’s upstream mode. In upstream mode, all requests are unconditionally transferred to an upstream proxy of your choice.</p> <img alt="_images/proxy-modes-upstream.png" class="align-center" src="_images/proxy-modes-upstream.png" /> <p>mitmproxy supports both explicit HTTP and explicit HTTPS in upstream proxy mode. You could in theory chain multiple mitmproxy instances in a row, but that doesn’t make any sense in practice (i.e. outside of our tests).</p> </div> </div> </div> <div class="toctree-wrapper compound"> <span id="document-mitmproxy"></span><div class="section" id="mitmproxy"> <span id="id1"></span><h2>mitmproxy<a class="headerlink" href="#mitmproxy" title="Permalink to this headline">露</a></h2> <p><strong>mitmproxy</strong> is a console tool that allows interactive examination and modification of HTTP traffic. It differs from mitmdump in that all flows are kept in memory, which means that it’s intended for taking and manipulating small-ish samples. Use the <code class="kbd docutils literal"><span class="pre">?</span></code> shortcut key to view, context-sensitive documentation from any <strong>mitmproxy</strong> screen.</p> <div class="section" id="flow-list"> <h3>Flow list<a class="headerlink" href="#flow-list" title="Permalink to this headline">露</a></h3> <p>The flow list shows an index of captured flows in chronological order.</p> <img alt="_images/mitmproxy.png" src="_images/mitmproxy.png" /> <ul class="simple"> <li><strong>1</strong>: A GET request, returning a 302 Redirect response.</li> <li><strong>2</strong>: A GET request, returning 16.75kb of text/html data.</li> <li><strong>3</strong>: A replayed request.</li> <li><strong>4</strong>: Intercepted flows are indicated with orange text. The user may edit these flows, and then accept them (using the <code class="kbd docutils literal"><span class="pre">a</span></code> key) to continue. In this case, the request has been intercepted on the way to the server.</li> <li><strong>5</strong>: A response intercepted from the server on the way to the client.</li> <li><strong>6</strong>: The event log can be toggled on and off using the <code class="kbd docutils literal"><span class="pre">e</span></code> shortcut key. This pane shows events and errors that may not result in a flow that shows up in the flow pane.</li> <li><strong>7</strong>: Flow count.</li> <li><strong>8</strong>: Various information on mitmproxy’s state. In this case, we have an interception pattern set to <code class="docutils literal"><span class="pre">.*</span></code>.</li> <li><strong>9</strong>: Bind address indicator - mitmproxy is listening on port 8080 of all interfaces.</li> </ul> </div> <div class="section" id="flow-view"> <h3>Flow view<a class="headerlink" href="#flow-view" title="Permalink to this headline">露</a></h3> <p>The <strong>Flow View</strong> lets you inspect and manipulate a single flow:</p> <img alt="_images/mitmproxy-flowview.png" src="_images/mitmproxy-flowview.png" /> <ul class="simple"> <li><strong>1</strong>: Flow summary.</li> <li><strong>2</strong>: The Request/Response tabs, showing you which part of the flow you are currently viewing. In the example above, we’re viewing the Response. Hit <code class="kbd docutils literal"><span class="pre">tab</span></code> to switch between the Response and the Request.</li> <li><strong>3</strong>: Headers.</li> <li><strong>4</strong>: Body.</li> <li><strong>5</strong>: View Mode indicator. In this case, we’re viewing the body in <strong>hex</strong> mode. The other available modes are <strong>pretty</strong>, which uses a number of heuristics to show you a friendly view of various content types, and <strong>raw</strong>, which shows you exactly what’s there without any changes. You can change modes using the <code class="kbd docutils literal"><span class="pre">m</span></code> key.</li> </ul> </div> <div class="section" id="grid-editor"> <h3>Grid Editor<a class="headerlink" href="#grid-editor" title="Permalink to this headline">露</a></h3> <p>Much of the data that we’d like to interact with in mitmproxy is structured. For instance, headers, queries and form data can all be thought of as a list of key/value pairs. Mitmproxy has a built-in editor that lays this type of data out in a grid for easy manipulation.</p> <p>At the moment, the Grid Editor is used in four parts of mitmproxy:</p> <blockquote> <div><ul class="simple"> <li>Editing request or response headers (<code class="kbd docutils literal"><span class="pre">e</span></code> for edit, then <code class="kbd docutils literal"><span class="pre">h</span></code> for headers in flow view)</li> <li>Editing a query string (<code class="kbd docutils literal"><span class="pre">e</span></code> for edit, then <code class="kbd docutils literal"><span class="pre">q</span></code> for query in flow view)</li> <li>Editing a URL-encoded form (<code class="kbd docutils literal"><span class="pre">e</span></code> for edit, then <code class="kbd docutils literal"><span class="pre">f</span></code> for form in flow view)</li> <li>Editing replacement patterns (<code class="kbd docutils literal"><span class="pre">o</span></code> for options, then <code class="kbd docutils literal"><span class="pre">R</span></code> for Replacement Patterns)</li> </ul> </div></blockquote> <p>If there is is no data, an empty editor will be started to let you add some. Here is the editor showing the headers from a request:</p> <img alt="_images/mitmproxy-kveditor.png" src="_images/mitmproxy-kveditor.png" /> <p>To edit, navigate to the key or value you want to modify using the arrow or vi navigation keys, and press enter. The background color will change to show that you are in edit mode for the specified field:</p> <img alt="_images/mitmproxy-kveditor-editmode.png" src="_images/mitmproxy-kveditor-editmode.png" /> <p>Modify the field as desired, then press escape to exit edit mode when you’re done. You can also add a row (<code class="kbd docutils literal"><span class="pre">a</span></code> key), delete a row (<code class="kbd docutils literal"><span class="pre">d</span></code> key), spawn an external editor on a field (<code class="kbd docutils literal"><span class="pre">e</span></code> key). Be sure to consult the context-sensitive help (<code class="kbd docutils literal"><span class="pre">?</span></code> key) for more.</p> </div> <div class="section" id="example-interception"> <h3>Example: Interception<a class="headerlink" href="#example-interception" title="Permalink to this headline">露</a></h3> <p><strong>mitmproxy</strong>‘s interception functionality lets you pause an HTTP request or response, inspect and modify it, and then accept it to send it on to the server or client.</p> <div class="section" id="set-an-interception-pattern"> <h4>1: Set an interception pattern<a class="headerlink" href="#set-an-interception-pattern" title="Permalink to this headline">露</a></h4> <img alt="_images/mitmproxy-intercept-filt.png" src="_images/mitmproxy-intercept-filt.png" /> <p>We press <code class="kbd docutils literal"><span class="pre">i</span></code> to set an interception pattern. In this case, the <code class="docutils literal"><span class="pre">~q</span></code> filter pattern tells <strong>mitmproxy</strong> to intercept all requests. For complete filter syntax, see the <a class="reference internal" href="index.html#filters"><span class="std std-ref">Filter expressions</span></a> section of the documentation, or the built-in help function in <strong>mitmproxy</strong>.</p> </div> <div class="section" id="intercepted-connections-are-indicated-with-orange-text"> <h4>2: Intercepted connections are indicated with orange text:<a class="headerlink" href="#intercepted-connections-are-indicated-with-orange-text" title="Permalink to this headline">露</a></h4> <img alt="_images/mitmproxy-intercept-mid.png" src="_images/mitmproxy-intercept-mid.png" /> </div> <div class="section" id="you-can-now-view-and-modify-the-request"> <h4>3: You can now view and modify the request:<a class="headerlink" href="#you-can-now-view-and-modify-the-request" title="Permalink to this headline">露</a></h4> <img alt="_images/mitmproxy-intercept-options.png" src="_images/mitmproxy-intercept-options.png" /> <p>In this case, we viewed the request by selecting it, pressed <code class="kbd docutils literal"><span class="pre">e</span></code> for “edit” and <code class="kbd docutils literal"><span class="pre">m</span></code> for “method” to change the HTTP request method.</p> </div> <div class="section" id="accept-the-intercept-to-continue"> <h4>4: Accept the intercept to continue:<a class="headerlink" href="#accept-the-intercept-to-continue" title="Permalink to this headline">露</a></h4> <img alt="_images/mitmproxy-intercept-result.png" src="_images/mitmproxy-intercept-result.png" /> <p>Finally, we press <code class="kbd docutils literal"><span class="pre">a</span></code> to accept the modified request, which is then sent on to the server. In this case, we changed the request from an HTTP GET to OPTIONS, and Google’s server has responded with a 405 “Method not allowed”.</p> </div> </div> </div> <span id="document-mitmdump"></span><div class="section" id="mitmdump"> <span id="id1"></span><h2>mitmdump<a class="headerlink" href="#mitmdump" title="Permalink to this headline">露</a></h2> <p><strong>mitmdump</strong> is the command-line companion to mitmproxy. It provides tcpdump-like functionality to let you view, record, and programmatically transform HTTP traffic. See the <code class="docutils literal"><span class="pre">--help</span></code> flag output for complete documentation.</p> <div class="section" id="examples"> <h3>Examples<a class="headerlink" href="#examples" title="Permalink to this headline">露</a></h3> <div class="section" id="saving-traffic"> <h4>Saving traffic<a class="headerlink" href="#saving-traffic" title="Permalink to this headline">露</a></h4> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">w</span> <span class="n">outfile</span> </pre></div> </div> <p>Start up mitmdump in proxy mode, and write all traffic to <strong>outfile</strong>.</p> </div> <div class="section" id="filtering-saved-traffic"> <h4>Filtering saved traffic<a class="headerlink" href="#filtering-saved-traffic" title="Permalink to this headline">露</a></h4> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">nr</span> <span class="n">infile</span> <span class="o">-</span><span class="n">w</span> <span class="n">outfile</span> <span class="s2">"~m post"</span> </pre></div> </div> <p>Start mitmdump without binding to the proxy port (<code class="docutils literal"><span class="pre">-n</span></code>), read all flows from infile, apply the specified filter expression (only match POSTs), and write to outfile.</p> </div> <div class="section" id="client-replay"> <h4>Client replay<a class="headerlink" href="#client-replay" title="Permalink to this headline">露</a></h4> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">nc</span> <span class="n">outfile</span> </pre></div> </div> <p>Start mitmdump without binding to the proxy port (<code class="docutils literal"><span class="pre">-n</span></code>), then replay all requests from outfile (<code class="docutils literal"><span class="pre">-c</span> <span class="pre">filename</span></code>). Flags combine in the obvious way, so you can replay requests from one file, and write the resulting flows to another:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">nc</span> <span class="n">srcfile</span> <span class="o">-</span><span class="n">w</span> <span class="n">dstfile</span> </pre></div> </div> <p>See the <a class="reference internal" href="index.html#clientreplay"><span class="std std-ref">Client-side replay</span></a> section for more information.</p> </div> <div class="section" id="running-a-script"> <h4>Running a script<a class="headerlink" href="#running-a-script" title="Permalink to this headline">露</a></h4> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">s</span> <span class="n">examples</span><span class="o">/</span><span class="n">add_header</span><span class="o">.</span><span class="n">py</span> </pre></div> </div> <p>This runs the <strong>add_header.py</strong> example script, which simply adds a new header to all responses.</p> </div> <div class="section" id="scripted-data-transformation"> <h4>Scripted data transformation<a class="headerlink" href="#scripted-data-transformation" title="Permalink to this headline">露</a></h4> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">ns</span> <span class="n">examples</span><span class="o">/</span><span class="n">add_header</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">r</span> <span class="n">srcfile</span> <span class="o">-</span><span class="n">w</span> <span class="n">dstfile</span> </pre></div> </div> <p>This command loads flows from <strong>srcfile</strong>, transforms it according to the specified script, then writes it back to <strong>dstfile</strong>.</p> </div> </div> </div> <span id="document-mitmweb"></span><div class="section" id="mitmweb"> <span id="id1"></span><h2>mitmweb<a class="headerlink" href="#mitmweb" title="Permalink to this headline">露</a></h2> <p><strong>mitmweb</strong> is mitmproxy’s web-based user interface that allows interactive examination and modification of HTTP traffic. Like mitmproxy, it differs from mitmdump in that all flows are kept in memory, which means that it’s intended for taking and manipulating small-ish samples.</p> <div class="admonition warning"> <p class="first admonition-title">Warning</p> <p class="last">Mitmweb is currently in beta. We consider it stable for all features currently exposed in the UI, but it still misses a lot of mitmproxy’s features.</p> </div> <img alt="_images/mitmweb.png" src="_images/mitmweb.png" /> </div> <span id="document-config"></span><div class="section" id="configuration"> <span id="config"></span><h2>Configuration<a class="headerlink" href="#configuration" title="Permalink to this headline">露</a></h2> <p>Mitmproxy is configured with a <a class="reference external" href="http://www.yaml.org/start.html">YAML</a> file, located at <code class="docutils literal"><span class="pre">~/.mitmproxy/config.yaml</span></code>. We’ll have complete documentation for all supported options in the next release in the meantime, please consult the <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/master/mitmproxy/options.py">source</a> for a complete list of options and types.</p> </div> </div> <div class="toctree-wrapper compound"> <span id="document-features/anticache"></span><div class="section" id="anticache"> <span id="id1"></span><h2>Anticache<a class="headerlink" href="#anticache" title="Permalink to this headline">露</a></h2> <p>When the <code class="docutils literal"><span class="pre">--anticache</span></code> option is passed to mitmproxy, it removes headers (<code class="docutils literal"><span class="pre">if-none-match</span></code> and <code class="docutils literal"><span class="pre">if-modified-since</span></code>) that might elicit a <code class="docutils literal"><span class="pre">304</span> <span class="pre">not</span> <span class="pre">modified</span></code> response from the server. This is useful when you want to make sure you capture an HTTP exchange in its totality. It’s also often used during <a class="reference internal" href="index.html#clientreplay"><span class="std std-ref">Client-side replay</span></a>, when you want to make sure the server responds with complete data.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--anticache</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">a</span></code></td> </tr> </tbody> </table> </div> <span id="document-features/filters"></span><div class="section" id="filter-expressions"> <span id="filters"></span><h2>Filter expressions<a class="headerlink" href="#filter-expressions" title="Permalink to this headline">露</a></h2> <p>Many commands in <strong class="program">mitmproxy</strong> and <strong class="program">mitmdump</strong> take a filter expression. Filter expressions consist of the following operators:</p> <table border="1" class="docutils"> <colgroup> <col width="50%" /> <col width="50%" /> </colgroup> <thead valign="bottom"> <tr class="row-odd"><th class="head">Expression</th> <th class="head">Description</th> </tr> </thead> <tbody valign="top"> <tr class="row-even"><td>~a</td> <td>Match asset in response: CSS, Javascript, Flash, images.</td> </tr> <tr class="row-odd"><td>~b regex</td> <td>Body</td> </tr> <tr class="row-even"><td>~bq regex</td> <td>Request body</td> </tr> <tr class="row-odd"><td>~bs regex</td> <td>Response body</td> </tr> <tr class="row-even"><td>~c int</td> <td>HTTP response code</td> </tr> <tr class="row-odd"><td>~d regex</td> <td>Domain</td> </tr> <tr class="row-even"><td>~dst regex</td> <td>Match destination address</td> </tr> <tr class="row-odd"><td>~e</td> <td>Match error</td> </tr> <tr class="row-even"><td>~h regex</td> <td>Header</td> </tr> <tr class="row-odd"><td>~hq regex</td> <td>Request header</td> </tr> <tr class="row-even"><td>~hs regex</td> <td>Response header</td> </tr> <tr class="row-odd"><td>~http</td> <td>Match HTTP flows</td> </tr> <tr class="row-even"><td>~m regex</td> <td>Method</td> </tr> <tr class="row-odd"><td>~marked</td> <td>Match marked flows</td> </tr> <tr class="row-even"><td>~q</td> <td>Match request with no response</td> </tr> <tr class="row-odd"><td>~s</td> <td>Match response</td> </tr> <tr class="row-even"><td>~src regex</td> <td>Match source address</td> </tr> <tr class="row-odd"><td>~t regex</td> <td>Content-type header</td> </tr> <tr class="row-even"><td>~tcp</td> <td>Match TCP flows</td> </tr> <tr class="row-odd"><td>~tq regex</td> <td>Request Content-Type header</td> </tr> <tr class="row-even"><td>~ts regex</td> <td>Response Content-Type header</td> </tr> <tr class="row-odd"><td>~u regex</td> <td>URL</td> </tr> <tr class="row-even"><td>!</td> <td>unary not</td> </tr> <tr class="row-odd"><td>&</td> <td>and</td> </tr> <tr class="row-even"><td>|</td> <td>or</td> </tr> <tr class="row-odd"><td>(...)</td> <td>grouping</td> </tr> </tbody> </table> <ul class="simple"> <li>Regexes are Python-style</li> <li>Regexes can be specified as quoted strings</li> <li>Header matching (~h, ~hq, ~hs) is against a string of the form “name: value”.</li> <li>Strings with no operators are matched against the request URL.</li> <li>The default binary operator is &.</li> </ul> <div class="section" id="examples"> <h3>Examples<a class="headerlink" href="#examples" title="Permalink to this headline">露</a></h3> <p>URL containing “google.com”:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>google\.com </pre></div> </div> <p>Requests whose body contains the string “test”:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>~q ~b test </pre></div> </div> <p>Anything but requests with a text/html content type:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>!(~q & ~t "text/html") </pre></div> </div> </div> </div> <span id="document-features/replacements"></span><div class="section" id="replacements"> <span id="id1"></span><h2>Replacements<a class="headerlink" href="#replacements" title="Permalink to this headline">露</a></h2> <p>Mitmproxy lets you specify an arbitrary number of patterns that define text replacements within flows. Each pattern has 3 components: a filter that defines which flows a replacement applies to, a regular expression that defines what gets replaced, and a target value that defines what is substituted in.</p> <p>Replace hooks fire when either a client request or a server response is received. Only the matching flow component is affected: so, for example, if a replace hook is triggered on server response, the replacement is only run on the Response object leaving the Request intact. You control whether the hook triggers on the request, response or both using the filter pattern. If you need finer-grained control than this, it’s simple to create a script using the replacement API on Flow components.</p> <p>Replacement hooks are extremely handy in interactive testing of applications. For instance you can use a replace hook to replace the text “XSS” with a complicated XSS exploit, and then “inject” the exploit simply by interacting with the application through the browser. When used with tools like Firebug and mitmproxy’s own interception abilities, replacement hooks can be an amazingly flexible and powerful feature.</p> <div class="section" id="on-the-command-line"> <h3>On the command-line<a class="headerlink" href="#on-the-command-line" title="Permalink to this headline">露</a></h3> <p>The replacement hook command-line options use a compact syntax to make it easy to specify all three components at once. The general form is as follows:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>/patt/regex/replacement </pre></div> </div> <p>Here, <strong>patt</strong> is a mitmproxy filter expression, <strong>regex</strong> is a valid Python regular expression, and <strong>replacement</strong> is a string literal. The first character in the expression (<code class="docutils literal"><span class="pre">/</span></code> in this case) defines what the separation character is. Here’s an example of a valid expression that replaces “foo” with “bar” in all requests:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>:~q:foo:bar </pre></div> </div> <p>In practice, it’s pretty common for the replacement literal to be long and complex. For instance, it might be an XSS exploit that weighs in at hundreds or thousands of characters. To cope with this, there’s a variation of the replacement hook specifier that lets you load the replacement text from a file. So, you might start <strong>mitmdump</strong> as follows:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">--</span><span class="n">replace</span><span class="o">-</span><span class="n">from</span><span class="o">-</span><span class="n">file</span> <span class="p">:</span><span class="o">~</span><span class="n">q</span><span class="p">:</span><span class="n">foo</span><span class="p">:</span><span class="o">~/</span><span class="n">xss</span><span class="o">-</span><span class="n">exploit</span> </pre></div> </div> <p>This will load the replacement text from the file <code class="docutils literal"><span class="pre">~/xss-exploit</span></code>.</p> <p>Both the <code class="docutils literal"><span class="pre">--replace</span></code> and <code class="docutils literal"><span class="pre">--replace-from-file</span></code> flags can be passed multiple times.</p> </div> <div class="section" id="interactively"> <h3>Interactively<a class="headerlink" href="#interactively" title="Permalink to this headline">露</a></h3> <p>The <code class="kbd docutils literal"><span class="pre">R</span></code> shortcut key in the mitmproxy options menu (<code class="kbd docutils literal"><span class="pre">o</span></code>) lets you add and edit replacement hooks using a built-in editor. The context-sensitive help (<code class="kbd docutils literal"><span class="pre">?</span></code>) has complete usage information.</p> <table border="1" class="docutils"> <colgroup> <col width="44%" /> <col width="56%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--replace</span></code>, <code class="docutils literal"><span class="pre">--replace-from-file</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">R</span></code></td> </tr> </tbody> </table> </div> </div> <span id="document-features/clientreplay"></span><div class="section" id="client-side-replay"> <span id="clientreplay"></span><h2>Client-side replay<a class="headerlink" href="#client-side-replay" title="Permalink to this headline">露</a></h2> <p>Client-side replay does what it says on the tin: you provide a previously saved HTTP conversation, and mitmproxy replays the client requests one by one. Note that mitmproxy serializes the requests, waiting for a response from the server before starting the next request. This might differ from the recorded conversation, where requests may have been made concurrently.</p> <p>You may want to use client-side replay in conjunction with the <a class="reference internal" href="index.html#anticache"><span class="std std-ref">Anticache</span></a> option, to make sure the server responds with complete data.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">-c</span> <span class="pre">path</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">R</span></code> then <code class="kbd docutils literal"><span class="pre">c</span></code></td> </tr> </tbody> </table> </div> <span id="document-features/serverreplay"></span><div class="section" id="server-side-replay"> <span id="serverreplay"></span><h2>Server-side replay<a class="headerlink" href="#server-side-replay" title="Permalink to this headline">露</a></h2> <p>Server-side replay lets us replay server responses from a saved HTTP conversation.</p> <div class="section" id="matching-requests-with-responses"> <h3>Matching requests with responses<a class="headerlink" href="#matching-requests-with-responses" title="Permalink to this headline">露</a></h3> <p>By default, <strong class="program">mitmproxy</strong> excludes request headers when matching incoming requests with responses from the replay file. This works in most circumstances, and makes it possible to replay server responses in situations where request headers would naturally vary, e.g. using a different user agent. The <code class="docutils literal"><span class="pre">--rheader</span> <span class="pre">headername</span></code> command-line option allows you to override this behaviour by specifying individual headers that should be included in matching.</p> </div> <div class="section" id="response-refreshing"> <h3>Response refreshing<a class="headerlink" href="#response-refreshing" title="Permalink to this headline">露</a></h3> <p>Simply replaying server responses without modification will often result in unexpected behaviour. For example cookie timeouts that were in the future at the time a conversation was recorded might be in the past at the time it is replayed. By default, <strong class="program">mitmproxy</strong> refreshes server responses before sending them to the client. The <strong>date</strong>, <strong>expires</strong> and <strong>last-modified</strong> headers are all updated to have the same relative time offset as they had at the time of recording. So, if they were in the past at the time of recording, they will be in the past at the time of replay, and vice versa. Cookie expiry times are updated in a similar way.</p> <p>You can turn off response refreshing using the <code class="docutils literal"><span class="pre">--norefresh</span></code> argument, or using the <code class="kbd docutils literal"><span class="pre">o</span></code> options shortcut within <strong class="program">mitmproxy</strong>.</p> </div> <div class="section" id="replaying-a-session-recorded-in-reverse-proxy-mode"> <h3>Replaying a session recorded in Reverse-proxy Mode<a class="headerlink" href="#replaying-a-session-recorded-in-reverse-proxy-mode" title="Permalink to this headline">露</a></h3> <p>If you have captured the session in reverse proxy mode, in order to replay it you still have to specify the server URL, otherwise you may get the error: ‘HTTP protocol error in client request: Invalid HTTP request form (expected authority or absolute...)’.</p> <p>During replay, when the client’s requests match previously recorded requests, then the respective recorded responses are simply replayed by mitmproxy. Otherwise, the unmatched requests is forwarded to the upstream server. If forwarding is not desired, you can use the –kill (-k) switch to prevent that.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">-S</span> <span class="pre">path</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">R</span></code> then <code class="kbd docutils literal"><span class="pre">s</span></code></td> </tr> </tbody> </table> </div> </div> <span id="document-features/setheaders"></span><div class="section" id="set-headers"> <span id="setheaders"></span><h2>Set Headers<a class="headerlink" href="#set-headers" title="Permalink to this headline">露</a></h2> <p>This feature lets you specify a set of headers to be added to requests or responses, based on a filter pattern. You can specify these either on the command-line, or through an interactive editor in mitmproxy.</p> <p>Example: Set the <strong>Host</strong> header to “example.com” for all requests.</p> <div class="highlight-none"><div class="highlight"><pre><span></span>mitmdump -R http://example.com --setheader :~q:Host:example.com </pre></div> </div> <table border="1" class="docutils"> <colgroup> <col width="44%" /> <col width="56%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--setheader</span> <span class="pre">PATTERN</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">H</span></code></td> </tr> </tbody> </table> </div> <span id="document-features/passthrough"></span><div class="section" id="ignore-domains"> <span id="passthrough"></span><h2>Ignore Domains<a class="headerlink" href="#ignore-domains" title="Permalink to this headline">露</a></h2> <p>There are two main reasons why you may want to exempt some traffic from mitmproxy’s interception mechanism:</p> <ul class="simple"> <li><strong>Certificate pinning:</strong> Some traffic is is protected using <a class="reference external" href="https://security.stackexchange.com/questions/29988/what-is-certificate-pinning">Certificate Pinning</a> and mitmproxy’s interception leads to errors. For example, the Twitter app, Windows Update or the Apple App Store fail to work if mitmproxy is active.</li> <li><strong>Convenience:</strong> You really don’t care about some parts of the traffic and just want them to go away. Note that mitmproxy’s “Limit” option is often the better alternative here, as it is not affected by the limitations listed below.</li> </ul> <p>If you want to peek into (SSL-protected) non-HTTP connections, check out the <a class="reference internal" href="index.html#tcpproxy"><span class="std std-ref">TCP Proxy</span></a> feature. If you want to ignore traffic from mitmproxy’s processing because of large response bodies, take a look at the <a class="reference internal" href="index.html#responsestreaming"><span class="std std-ref">Response Streaming</span></a> feature.</p> <div class="section" id="how-it-works"> <h3>How it works<a class="headerlink" href="#how-it-works" title="Permalink to this headline">露</a></h3> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--ignore</span> <span class="pre">regex</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">I</span></code></td> </tr> </tbody> </table> <p>mitmproxy allows you to specify a regex which is matched against a <code class="docutils literal"><span class="pre">host:port</span></code> string (e.g. “example.com:443”) to determine hosts that should be excluded.</p> </div> <div class="section" id="limitations"> <h3>Limitations<a class="headerlink" href="#limitations" title="Permalink to this headline">露</a></h3> <p>There are two important quirks to consider:</p> <ul class="simple"> <li><strong>In transparent mode, the ignore pattern is matched against the IP and ClientHello SNI host.</strong> While we usually infer the hostname from the Host header if the <code class="docutils literal"><span class="pre">--host</span></code> argument is passed to mitmproxy, we do not have access to this information before the SSL handshake. If the client uses SNI however, then we treat the SNI host as an ignore target.</li> <li><strong>In regular mode, explicit HTTP requests are never ignored.</strong> <a class="footnote-reference" href="#explicithttp" id="id1">[1]</a> The ignore pattern is applied on CONNECT requests, which initiate HTTPS or clear-text WebSocket connections.</li> </ul> </div> <div class="section" id="tutorial"> <h3>Tutorial<a class="headerlink" href="#tutorial" title="Permalink to this headline">露</a></h3> <p>If you just want to ignore one specific domain, there’s usually a bulletproof method to do so:</p> <ol class="arabic simple"> <li>Run mitmproxy or mitmdump in verbose mode (<code class="docutils literal"><span class="pre">-v</span></code>) and observe the <code class="docutils literal"><span class="pre">host:port</span></code> information in the serverconnect messages. mitmproxy will filter on these.</li> <li>Take the <code class="docutils literal"><span class="pre">host:port</span></code> string, surround it with ^ and $, escape all dots (. becomes \.) and use this as your ignore pattern:</li> </ol> <div class="highlight-none"><div class="highlight"><pre><span></span>>>> mitmdump -v 127.0.0.1:50588: clientconnect 127.0.0.1:50588: request -> CONNECT example.com:443 HTTP/1.1 127.0.0.1:50588: Set new server address: example.com:443 <span class="hll">127.0.0.1:50588: serverconnect </span><span class="hll"> -> example.com:443 </span>^C <span class="hll">>>> mitmproxy --ignore ^example\.com:443$ </span></pre></div> </div> <p>Here are some other examples for ignore patterns:</p> <div class="highlight-none"><div class="highlight"><pre><span></span># Exempt traffic from the iOS App Store (the regex is lax, but usually just works): --ignore apple.com:443 # "Correct" version without false-positives: --ignore '^(.+\.)?apple\.com:443$' # Ignore example.com, but not its subdomains: --ignore '^example.com:' # Ignore everything but example.com and mitmproxy.org: --ignore '^(?!example\.com)(?!mitmproxy\.org)' # Transparent mode: --ignore 17\.178\.96\.59:443 # IP address range: --ignore 17\.178\.\d+\.\d+:443 </pre></div> </div> <div class="admonition seealso"> <p class="first admonition-title">See also</p> <ul class="last simple"> <li><a class="reference internal" href="index.html#tcpproxy"><span class="std std-ref">TCP Proxy</span></a></li> <li><a class="reference internal" href="index.html#responsestreaming"><span class="std std-ref">Response Streaming</span></a></li> <li>mitmproxy’s “Limit” feature</li> </ul> </div> <p class="rubric">Footnotes</p> <table class="docutils footnote" frame="void" id="explicithttp" rules="none"> <colgroup><col class="label" /><col /></colgroup> <tbody valign="top"> <tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td>This stems from an limitation of explicit HTTP proxying: A single connection can be re-used for multiple target domains - a <code class="docutils literal"><span class="pre">GET</span> <span class="pre">http://example.com/</span></code> request may be followed by a <code class="docutils literal"><span class="pre">GET</span> <span class="pre">http://evil.com/</span></code> request on the same connection. If we start to ignore the connection after the first request, we would miss the relevant second one.</td></tr> </tbody> </table> </div> </div> <span id="document-features/proxyauth"></span><div class="section" id="proxy-authentication"> <span id="proxyauth"></span><h2>Proxy Authentication<a class="headerlink" href="#proxy-authentication" title="Permalink to this headline">露</a></h2> <p>Asks the user for authentication before they are permitted to use the proxy. Authentication headers are stripped from the flows, so they are not passed to upstream servers. For now, only HTTP Basic authentication is supported. The proxy auth options are not compatible with the transparent, socks or reverse proxy mode.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--nonanonymous</span></code>, <code class="docutils literal"><span class="pre">--singleuser</span> <span class="pre">USER</span></code>, <code class="docutils literal"><span class="pre">--htpasswd</span> <span class="pre">PATH</span></code></td> </tr> </tbody> </table> </div> <span id="document-features/reverseproxy"></span><div class="section" id="reverse-proxy"> <span id="reverseproxy"></span><h2>Reverse Proxy<a class="headerlink" href="#reverse-proxy" title="Permalink to this headline">露</a></h2> <p>In reverse proxy mode, mitmproxy accepts standard HTTP(S) requests and forwards them to the specified upstream server. This is in contrast to <a class="reference internal" href="index.html#upstreamproxy"><span class="std std-ref">Upstream proxy mode</span></a>, in which mitmproxy forwards HTTP(S) proxy requests to an upstream proxy server.</p> <table border="1" class="docutils"> <colgroup> <col width="36%" /> <col width="64%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">-R</span> <span class="pre">http[s]://hostname[:port]</span></code></td> </tr> </tbody> </table> <p>Here, <strong>http[s]</strong> signifies if the proxy should use TLS to connect to the server. mitmproxy always accepts both encrypted and unencrypted requests and transforms them to what the server expects.</p> <div class="highlight-none"><div class="highlight"><pre><span></span>>>> mitmdump -R https://httpbin.org -p 80 >>> curl http://localhost/ # requests will be transparently upgraded to TLS by mitmproxy >>> mitmdump -R https://httpbin.org -p 443 >>> curl https://localhost/ # mitmproxy will use TLS on both ends. </pre></div> </div> <div class="section" id="host-header"> <h3>Host Header<a class="headerlink" href="#host-header" title="Permalink to this headline">露</a></h3> <p>In reverse proxy mode, mitmproxy automatically rewrites the Host header to match the upstream server. This allows mitmproxy to easily connect to existing endpoints on the open web (e.g. <code class="docutils literal"><span class="pre">mitmproxy</span> <span class="pre">-R</span> <span class="pre">https://example.com</span></code>).</p> <p>However, keep in mind that absolute URLs within the returned document or HTTP redirects will NOT be rewritten by mitmproxy. This means that if you click on a link for “<a class="reference external" href="http://example.com">http://example.com</a>” in the returned web page, you will be taken directly to that URL, bypassing mitmproxy.</p> <p>One possible way to address this is to modify the hosts file of your OS so that “example.com” resolves to your proxy’s IP, and then access the proxy by going directly to example.com. Make sure that your proxy can still resolve the original IP, or specify an IP in mitmproxy.</p> </div> </div> <span id="document-features/responsestreaming"></span><div class="section" id="response-streaming"> <span id="responsestreaming"></span><h2>Response Streaming<a class="headerlink" href="#response-streaming" title="Permalink to this headline">露</a></h2> <p>By using mitmproxy’s streaming feature, response contents can be passed to the client incrementally before they have been fully received by the proxy. This is especially useful for large binary files such as videos, where buffering the whole file slows down the client’s browser.</p> <p>By default, mitmproxy will read the entire response, perform any indicated manipulations on it and then send the (possibly modified) response to the client. In some cases this is undesirable and you may wish to “stream” the response back to the client. When streaming is enabled, the response is not buffered on the proxy but directly sent back to the client instead.</p> <div class="section" id="on-the-command-line"> <h3>On the command-line<a class="headerlink" href="#on-the-command-line" title="Permalink to this headline">露</a></h3> <p>Streaming can be enabled on the command line for all response bodies exceeding a certain size. The SIZE argument understands k/m/g suffixes, e.g. 3m for 3 megabytes.</p> <table border="1" class="docutils"> <colgroup> <col width="51%" /> <col width="49%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--stream</span> <span class="pre">SIZE</span></code></td> </tr> </tbody> </table> <div class="admonition warning"> <p class="first admonition-title">Warning</p> <p class="last">When response streaming is enabled, <strong>streamed response contents will not be recorded or preserved in any way.</strong></p> </div> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">When response streaming is enabled, the response body cannot be modified by the usual means.</p> </div> </div> <div class="section" id="customizing-response-streaming"> <h3>Customizing Response Streaming<a class="headerlink" href="#customizing-response-streaming" title="Permalink to this headline">露</a></h3> <p>You can also use a script to customize exactly which responses are streamed.</p> <p>Responses that should be tagged for streaming by setting their <code class="docutils literal"><span class="pre">.stream</span></code> attribute to <code class="docutils literal"><span class="pre">True</span></code>:</p> <div class="literal-block-wrapper docutils container" id="id1"> <div class="code-block-caption"><span class="caption-text">examples/complex/stream.py</span><a class="headerlink" href="#id1" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">responseheaders</span><span class="p">(</span><span class="n">flow</span><span class="p">):</span> <span class="sd">"""</span> <span class="sd"> Enables streaming for all responses.</span> <span class="sd"> This is equivalent to passing `--stream 0` to mitmproxy.</span> <span class="sd"> """</span> <span class="n">flow</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">stream</span> <span class="o">=</span> <span class="bp">True</span> </pre></div> </div> </div> </div> <div class="section" id="implementation-details"> <h3>Implementation Details<a class="headerlink" href="#implementation-details" title="Permalink to this headline">露</a></h3> <p>When response streaming is enabled, portions of the code which would have otherwise performed changes on the response body will see an empty response body. Any modifications will be ignored.</p> <p>Streamed responses are usually sent in chunks of 4096 bytes. If the response is sent with a <code class="docutils literal"><span class="pre">Transfer-Encoding:</span> <span class="pre">chunked</span></code> header, the response will be streamed one chunk at a time.</p> </div> <div class="section" id="modifying-streamed-data"> <h3>Modifying streamed data<a class="headerlink" href="#modifying-streamed-data" title="Permalink to this headline">露</a></h3> <p>If the <code class="docutils literal"><span class="pre">.stream</span></code> attribute is callable, <code class="docutils literal"><span class="pre">.stream</span></code> will wrap the generator that yields all chunks.</p> <div class="literal-block-wrapper docutils container" id="id2"> <div class="code-block-caption"><span class="caption-text">examples/complex/stream_modify.py</span><a class="headerlink" href="#id2" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="sd">"""</span> <span class="sd">This inline script modifies a streamed response.</span> <span class="sd">If you do not need streaming, see the modify_response_body example.</span> <span class="sd">Be aware that content replacement isn't trivial:</span> <span class="sd"> - If the transfer encoding isn't chunked, you cannot simply change the content length.</span> <span class="sd"> - If you want to replace all occurences of "foobar", make sure to catch the cases</span> <span class="sd"> where one chunk ends with [...]foo" and the next starts with "bar[...].</span> <span class="sd">"""</span> <span class="k">def</span> <span class="nf">modify</span><span class="p">(</span><span class="n">chunks</span><span class="p">):</span> <span class="sd">"""</span> <span class="sd"> chunks is a generator that can be used to iterate over all chunks.</span> <span class="sd"> """</span> <span class="k">for</span> <span class="n">chunk</span> <span class="ow">in</span> <span class="n">chunks</span><span class="p">:</span> <span class="k">yield</span> <span class="n">chunk</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"foo"</span><span class="p">,</span> <span class="s2">"bar"</span><span class="p">)</span> <span class="k">def</span> <span class="nf">responseheaders</span><span class="p">(</span><span class="n">flow</span><span class="p">):</span> <span class="n">flow</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">stream</span> <span class="o">=</span> <span class="n">modify</span> </pre></div> </div> </div> <div class="admonition seealso"> <p class="first admonition-title">See also</p> <ul class="last simple"> <li><a class="reference internal" href="index.html#passthrough"><span class="std std-ref">Ignore Domains</span></a></li> </ul> </div> </div> </div> <span id="document-features/socksproxy"></span><div class="section" id="socks-mode"> <span id="socksproxy"></span><h2>SOCKS Mode<a class="headerlink" href="#socks-mode" title="Permalink to this headline">露</a></h2> <p>In this mode, mitmproxy acts as a SOCKS5 proxy server.</p> <table border="1" class="docutils"> <colgroup> <col width="62%" /> <col width="38%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--socks</span></code></td> </tr> </tbody> </table> </div> <span id="document-features/sticky"></span><div class="section" id="sticky-cookies-and-auth"> <span id="sticky"></span><h2>Sticky cookies and auth<a class="headerlink" href="#sticky-cookies-and-auth" title="Permalink to this headline">露</a></h2> <div class="section" id="sticky-cookies"> <h3>Sticky cookies<a class="headerlink" href="#sticky-cookies" title="Permalink to this headline">露</a></h3> <p>When the sticky cookie option is set, __mitmproxy__ will add the cookie most recently set by the server to any cookie-less request. Consider a service that sets a cookie to track the session after authentication. Using sticky cookies, you can fire up mitmproxy, and authenticate to a service as you usually would using a browser. After authentication, you can request authenticated resources through mitmproxy as if they were unauthenticated, because mitmproxy will automatically add the session tracking cookie to requests. Among other things, this lets you script interactions with authenticated resources (using tools like wget or curl) without having to worry about authentication.</p> <p>Sticky cookies are especially powerful when used in conjunction with <a class="reference internal" href="index.html#clientreplay"><span class="std std-ref">Client-side replay</span></a> - you can record the authentication process once, and simply replay it on startup every time you need to interact with the secured resources.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">-t</span> <span class="pre">FILTER</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">t</span></code></td> </tr> </tbody> </table> </div> <div class="section" id="sticky-auth"> <h3>Sticky auth<a class="headerlink" href="#sticky-auth" title="Permalink to this headline">露</a></h3> <p>The sticky auth option is analogous to the sticky cookie option, in that HTTP <strong>Authorization</strong> headers are simply replayed to the server once they have been seen. This is enough to allow you to access a server resource using HTTP Basic authentication through the proxy. Note that <strong class="program">mitmproxy</strong> doesn’t (yet) support replay of HTTP Digest authentication.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">-u</span> <span class="pre">FILTER</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">A</span></code></td> </tr> </tbody> </table> </div> </div> <span id="document-features/tcpproxy"></span><div class="section" id="tcp-proxy"> <span id="tcpproxy"></span><h2>TCP Proxy<a class="headerlink" href="#tcp-proxy" title="Permalink to this headline">露</a></h2> <p>In case mitmproxy does not handle a specific protocol, you can exempt hostnames from processing, so that mitmproxy acts as a generic TCP forwarder. This feature is closely related to the <a class="reference internal" href="index.html#passthrough"><span class="std std-ref">Ignore Domains</span></a> functionality, but differs in two important aspects:</p> <ul class="simple"> <li>The raw TCP messages are printed to the event log.</li> <li>SSL connections will be intercepted.</li> </ul> <p>Please note that message interception or modification are not possible yet. If you are not interested in the raw TCP messages, you should use the ignore domains feature.</p> <div class="section" id="how-it-works"> <h3>How it works<a class="headerlink" href="#how-it-works" title="Permalink to this headline">露</a></h3> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--tcp</span> <span class="pre">HOST</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">T</span></code></td> </tr> </tbody> </table> <p>For a detailed description how the hostname pattern works, please look at the <a class="reference internal" href="index.html#passthrough"><span class="std std-ref">Ignore Domains</span></a> feature.</p> <div class="admonition seealso"> <p class="first admonition-title">See also</p> <ul class="last simple"> <li><a class="reference internal" href="index.html#passthrough"><span class="std std-ref">Ignore Domains</span></a></li> <li><a class="reference internal" href="index.html#responsestreaming"><span class="std std-ref">Response Streaming</span></a></li> </ul> </div> </div> </div> <span id="document-features/upstreamproxy"></span><div class="section" id="upstream-proxy-mode"> <span id="upstreamproxy"></span><h2>Upstream proxy mode<a class="headerlink" href="#upstream-proxy-mode" title="Permalink to this headline">露</a></h2> <p>In this mode, mitmproxy accepts proxy requests and unconditionally forwards all requests to a specified upstream proxy server. This is in contrast to <a class="reference internal" href="index.html#reverseproxy"><span class="std std-ref">Reverse Proxy</span></a>, in which mitmproxy forwards ordinary HTTP requests to an upstream server.</p> <table border="1" class="docutils"> <colgroup> <col width="38%" /> <col width="62%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">-U</span> <span class="pre">http://hostname[:port]</span></code></td> </tr> </tbody> </table> </div> <span id="document-features/upstreamcerts"></span><div class="section" id="upstream-certificates"> <span id="upstreamcerts"></span><h2>Upstream Certificates<a class="headerlink" href="#upstream-certificates" title="Permalink to this headline">露</a></h2> <p>When mitmproxy receives a connection destined for an SSL-protected service, it freezes the connection before reading its request data, and makes a connection to the upstream server to “sniff” the contents of its SSL certificate. The information gained - the <strong>Common Name</strong> and <strong>Subject Alternative Names</strong> - is then used to generate the interception certificate, which is sent to the client so the connection can continue.</p> <p>This rather intricate little dance lets us seamlessly generate correct certificates even if the client has specified only an IP address rather than the hostname. It also means that we don’t need to sniff additional data to generate certs in transparent mode.</p> <p>Upstream cert sniffing is on by default, and can optionally be turned off.</p> <table border="1" class="docutils"> <colgroup> <col width="45%" /> <col width="55%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>command-line</td> <td><code class="docutils literal"><span class="pre">--no-upstream-cert</span></code></td> </tr> <tr class="row-even"><td>mitmproxy shortcut</td> <td><code class="kbd docutils literal"><span class="pre">o</span></code> then <code class="kbd docutils literal"><span class="pre">U</span></code></td> </tr> </tbody> </table> </div> </div> <div class="toctree-wrapper compound"> <span id="document-transparent"></span><div class="section" id="transparent-proxying"> <span id="transparent"></span><h2>Transparent Proxying<a class="headerlink" href="#transparent-proxying" title="Permalink to this headline">露</a></h2> <p>When a transparent proxy is used, traffic is redirected into a proxy at the network layer, without any client configuration being required. This makes transparent proxying ideal for those situations where you can’t change client behaviour - proxy-oblivious Android applications being a common example.</p> <p>To set up transparent proxying, we need two new components. The first is a redirection mechanism that transparently reroutes a TCP connection destined for a server on the Internet to a listening proxy server. This usually takes the form of a firewall on the same host as the proxy server - <a class="reference external" href="http://www.netfilter.org/">iptables</a> on Linux or <a class="reference external" href="https://en.wikipedia.org/wiki/PF_(firewall)">pf</a> on OSX. When the proxy receives a redirected connection, it sees a vanilla HTTP request, without a host specification. This is where the second new component comes in - a host module that allows us to query the redirector for the original destination of the TCP connection.</p> <p>At the moment, mitmproxy supports transparent proxying on OSX Lion and above, and all current flavors of Linux.</p> <div class="section" id="fully-transparent-mode"> <h3>Fully transparent mode<a class="headerlink" href="#fully-transparent-mode" title="Permalink to this headline">露</a></h3> <p>By default mitmproxy will use its own local ip address for its server-side connections. In case this isn’t desired, the –spoof-source-address argument can be used to use the client’s ip address for server-side connections. The following config is required for this mode to work:</p> <div class="highlight-default"><div class="highlight"><pre><span></span>CLIENT_NET=192.168.1.0/24 TABLE_ID=100 MARK=1 echo "$TABLE_ID mitmproxy" >> /etc/iproute2/rt_tables iptables -t mangle -A PREROUTING -d $CLIENT_NET -j MARK --set-mark $MARK iptables -t nat -A PREROUTING -p tcp -s $CLIENT_NET --match multiport --dports 80,443 -j REDIRECT --to-port 8080 ip rule add fwmark $MARK lookup $TABLE_ID ip route add local $CLIENT_NET dev lo table $TABLE_ID </pre></div> </div> <p>This mode does require root privileges though. There’s a wrapper in the examples directory called ‘mitmproxy_shim.c’, which will enable you to use this mode with dropped priviliges. It can be used as follows:</p> <div class="highlight-default"><div class="highlight"><pre><span></span>gcc examples/complex/full_transparency_shim.c -o mitmproxy_shim -lcap sudo chown root:root mitmproxy_shim sudo chmod u+s mitmproxy_shim ./mitmproxy_shim $(which mitmproxy) -T --spoof-source-address </pre></div> </div> </div> </div> <span id="document-transparent/linux"></span><div class="section" id="linux"> <span id="id1"></span><h2>Linux<a class="headerlink" href="#linux" title="Permalink to this headline">露</a></h2> <p>On Linux, mitmproxy integrates with the iptables redirection mechanism to achieve transparent mode.</p> <blockquote> <div><ol class="arabic"> <li><p class="first"><a class="reference internal" href="index.html#certinstall"><span class="std std-ref">Install the mitmproxy certificate on the test device</span></a></p> </li> <li><p class="first">Enable IP forwarding:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sysctl</span> <span class="o">-</span><span class="n">w</span> <span class="n">net</span><span class="o">.</span><span class="n">ipv4</span><span class="o">.</span><span class="n">ip_forward</span><span class="o">=</span><span class="mi">1</span> </pre></div> </div> <p>You may also want to consider enabling this permanently in <code class="docutils literal"><span class="pre">/etc/sysctl.conf</span></code>.</p> </li> <li><p class="first">If your target machine is on the same physical network and you configured it to use a custom gateway, disable ICMP redirects:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">echo</span> <span class="mi">0</span> <span class="o">|</span> <span class="n">sudo</span> <span class="n">tee</span> <span class="o">/</span><span class="n">proc</span><span class="o">/</span><span class="n">sys</span><span class="o">/</span><span class="n">net</span><span class="o">/</span><span class="n">ipv4</span><span class="o">/</span><span class="n">conf</span><span class="o">/*/</span><span class="n">send_redirects</span> </pre></div> </div> <p>You may also want to consider enabling this permanently in <code class="docutils literal"><span class="pre">/etc/sysctl.conf</span></code> as demonstrated <a class="reference external" href="https://unix.stackexchange.com/a/58081">here</a>.</p> </li> <li><p class="first">Create an iptables ruleset that redirects the desired traffic to the mitmproxy port. Details will differ according to your setup, but the ruleset should look something like this:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8080 </pre></div> </div> </li> <li><p class="first">Fire up mitmproxy. You probably want a command like this:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmproxy</span> <span class="o">-</span><span class="n">T</span> <span class="o">--</span><span class="n">host</span> </pre></div> </div> <p>The <code class="docutils literal"><span class="pre">-T</span></code> flag turns on transparent mode, and the <code class="docutils literal"><span class="pre">--host</span></code> argument tells mitmproxy to use the value of the Host header for URL display.</p> </li> <li><p class="first">Finally, configure your test device to use the host on which mitmproxy is running as the default gateway.</p> </li> </ol> </div></blockquote> <p>For a detailed walkthrough, have a look at the <a class="reference internal" href="index.html#transparent-dhcp"><span class="std std-ref">Transparently proxify virtual machines</span></a> tutorial.</p> </div> <span id="document-transparent/osx"></span><div class="section" id="osx"> <span id="id1"></span><h2>OSX<a class="headerlink" href="#osx" title="Permalink to this headline">露</a></h2> <p>OSX Lion integrated the <a class="reference external" href="https://en.wikipedia.org/wiki/PF_(firewall)">pf</a> packet filter from the OpenBSD project, which mitmproxy uses to implement transparent mode on OSX. Note that this means we don’t support transparent mode for earlier versions of OSX.</p> <blockquote> <div><ol class="arabic"> <li><p class="first"><a class="reference internal" href="index.html#certinstall"><span class="std std-ref">Install the mitmproxy certificate on the test device</span></a></p> </li> <li><p class="first">Enable IP forwarding:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">sysctl</span> <span class="o">-</span><span class="n">w</span> <span class="n">net</span><span class="o">.</span><span class="n">inet</span><span class="o">.</span><span class="n">ip</span><span class="o">.</span><span class="n">forwarding</span><span class="o">=</span><span class="mi">1</span> </pre></div> </div> </li> <li><p class="first">Place the following two lines in a file called, say, <strong>pf.conf</strong>:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>rdr on en2 inet proto tcp to any port 80 -> 127.0.0.1 port 8080 rdr on en2 inet proto tcp to any port 443 -> 127.0.0.1 port 8080 </pre></div> </div> <p>These rules tell pf to redirect all traffic destined for port 80 or 443 to the local mitmproxy instance running on port 8080. You should replace <code class="docutils literal"><span class="pre">en2</span></code> with the interface on which your test device will appear.</p> </li> <li><p class="first">Configure pf with the rules:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">pfctl</span> <span class="o">-</span><span class="n">f</span> <span class="n">pf</span><span class="o">.</span><span class="n">conf</span> </pre></div> </div> </li> <li><p class="first">And now enable it:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">pfctl</span> <span class="o">-</span><span class="n">e</span> </pre></div> </div> </li> <li><p class="first">Configure sudoers to allow mitmproxy to access pfctl. Edit the file <strong>/etc/sudoers</strong> on your system as root. Add the following line to the end of the file:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>ALL ALL=NOPASSWD: /sbin/pfctl -s state </pre></div> </div> <p>Note that this allows any user on the system to run the command <code class="docutils literal"><span class="pre">/sbin/pfctl</span> <span class="pre">-s</span> <span class="pre">state</span></code> as root without a password. This only allows inspection of the state table, so should not be an undue security risk. If you’re special feel free to tighten the restriction up to the user running mitmproxy.</p> </li> <li><p class="first">Fire up mitmproxy. You probably want a command like this:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmproxy</span> <span class="o">-</span><span class="n">T</span> <span class="o">--</span><span class="n">host</span> </pre></div> </div> <p>The <code class="docutils literal"><span class="pre">-T</span></code> flag turns on transparent mode, and the <code class="docutils literal"><span class="pre">--host</span></code> argument tells mitmproxy to use the value of the Host header for URL display.</p> </li> <li><p class="first">Finally, configure your test device to use the host on which mitmproxy is running as the default gateway.</p> </li> </ol> </div></blockquote> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">Note that the <strong>rdr</strong> rules in the pf.conf given above only apply to inbound traffic. <strong>This means that they will NOT redirect traffic coming from the box running pf itself.</strong> We can’t distinguish between an outbound connection from a non-mitmproxy app, and an outbound connection from mitmproxy itself - if you want to intercept your OSX traffic, you should use an external host to run mitmproxy. Nonetheless, pf is flexible to cater for a range of creative possibilities, like intercepting traffic emanating from VMs. See the <strong>pf.conf</strong> man page for more.</p> </div> </div> <span id="document-transparent/openbsd"></span><div class="section" id="openbsd"> <span id="id1"></span><h2>OpenBSD<a class="headerlink" href="#openbsd" title="Permalink to this headline">露</a></h2> <blockquote> <div><ol class="arabic"> <li><p class="first"><a class="reference internal" href="index.html#certinstall"><span class="std std-ref">Install the mitmproxy certificate on the test device</span></a></p> </li> <li><p class="first">Enable IP forwarding:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">sysctl</span> <span class="o">-</span><span class="n">w</span> <span class="n">net</span><span class="o">.</span><span class="n">inet</span><span class="o">.</span><span class="n">ip</span><span class="o">.</span><span class="n">forwarding</span><span class="o">=</span><span class="mi">1</span> </pre></div> </div> </li> <li><p class="first">Place the following two lines in <strong>/etc/pf.conf</strong>:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>mitm_if = "re2" pass in quick proto tcp from $mitm_if to port { 80, 443 } divert-to 127.0.0.1 port 8080 </pre></div> </div> <p>These rules tell pf to divert all traffic from <code class="docutils literal"><span class="pre">$mitm_if</span></code> destined for port 80 or 443 to the local mitmproxy instance running on port 8080. You should replace <code class="docutils literal"><span class="pre">$mitm_if</span></code> value with the interface on which your test device will appear.</p> </li> <li><p class="first">Configure pf with the rules:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">doas</span> <span class="n">pfctl</span> <span class="o">-</span><span class="n">f</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">pf</span><span class="o">.</span><span class="n">conf</span> </pre></div> </div> </li> <li><p class="first">And now enable it:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">doas</span> <span class="n">pfctl</span> <span class="o">-</span><span class="n">e</span> </pre></div> </div> </li> <li><p class="first">Fire up mitmproxy. You probably want a command like this:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmproxy</span> <span class="o">-</span><span class="n">T</span> <span class="o">--</span><span class="n">host</span> </pre></div> </div> <p>The <code class="docutils literal"><span class="pre">-T</span></code> flag turns on transparent mode, and the <code class="docutils literal"><span class="pre">--host</span></code> argument tells mitmproxy to use the value of the Host header for URL display.</p> </li> <li><p class="first">Finally, configure your test device to use the host on which mitmproxy is running as the default gateway.</p> </li> </ol> </div></blockquote> <div class="admonition note"> <p class="first admonition-title">Note</p> <p class="last">Note that the <strong>divert-to</strong> rules in the pf.conf given above only apply to inbound traffic. <strong>This means that they will NOT redirect traffic coming from the box running pf itself.</strong> We can’t distinguish between an outbound connection from a non-mitmproxy app, and an outbound connection from mitmproxy itself - if you want to intercept your traffic, you should use an external host to run mitmproxy. Nonetheless, pf is flexible to cater for a range of creative possibilities, like intercepting traffic emanating from VMs. See the <strong>pf.conf</strong> man page for more.</p> </div> </div> </div> <div class="toctree-wrapper compound"> <span id="document-scripting/overview"></span><div class="section" id="overview"> <span id="id1"></span><h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">露</a></h2> <p>Mitmproxy has a powerful scripting API that allows you to control almost any aspect of traffic being proxied. In fact, much of mitmproxy’s own core functionality is implemented using the exact same API exposed to scripters (see <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/addons">mitmproxy/addons</a>).</p> <div class="section" id="a-simple-example"> <h3>A simple example<a class="headerlink" href="#a-simple-example" title="Permalink to this headline">露</a></h3> <p>Scripting is event driven, with named handlers on the script object called at appropriate points of mitmproxy’s operation. Here’s a complete mitmproxy script that adds a new header to every HTTP response before it is returned to the client:</p> <div class="literal-block-wrapper docutils container" id="id3"> <div class="code-block-caption"><span class="caption-text"><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/examples/simple/add_header.py">examples/simple/add_header.py</a></span><a class="headerlink" href="#id3" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">response</span><span class="p">(</span><span class="n">flow</span><span class="p">):</span> <span class="n">flow</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">headers</span><span class="p">[</span><span class="s2">"newheader"</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"foo"</span> </pre></div> </div> </div> <p>All events that deal with an HTTP request get an instance of <a class="reference external" href="api.html#mitmproxy.models.http.HTTPFlow">HTTPFlow</a>, which we can use to manipulate the response itself. We can now run this script using mitmdump, and the new header will be added to all responses passing through the proxy:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">s</span> <span class="n">add_header</span><span class="o">.</span><span class="n">py</span> </pre></div> </div> </div> <div class="section" id="using-classes"> <h3>Using classes<a class="headerlink" href="#using-classes" title="Permalink to this headline">露</a></h3> <p>In the example above, the script object is the <code class="docutils literal"><span class="pre">add_header</span></code> module itself. That is, the handlers are declared at the global level of the script. This is great for quick hacks, but soon becomes limiting as scripts become more sophisticated.</p> <p>When a script first starts up, the <a class="reference external" href="events.html#start">start</a>, event is called before anything else happens. You can replace the current script object by returning it from this handler. Here’s how this looks when applied to the example above:</p> <div class="literal-block-wrapper docutils container" id="id4"> <div class="code-block-caption"><span class="caption-text"><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/examples/simple/add_header_class.py">examples/simple/add_header_class.py</a></span><a class="headerlink" href="#id4" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">AddHeader</span><span class="p">:</span> <span class="k">def</span> <span class="nf">response</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">flow</span><span class="p">):</span> <span class="n">flow</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">headers</span><span class="p">[</span><span class="s2">"newheader"</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"foo"</span> <span class="k">def</span> <span class="nf">start</span><span class="p">():</span> <span class="k">return</span> <span class="n">AddHeader</span><span class="p">()</span> </pre></div> </div> </div> <p>So here, we’re using a module-level script to “boot up” into a class instance. From this point on, the module-level script is removed from the handler chain, and is replaced by the class instance.</p> </div> <div class="section" id="handling-arguments"> <h3>Handling arguments<a class="headerlink" href="#handling-arguments" title="Permalink to this headline">露</a></h3> <p>Scripts can handle their own command-line arguments, just like any other Python program. Let’s build on the example above to do something slightly more sophisticated - replace one value with another in all responses. Mitmproxy’s <a class="reference external" href="api.html#mitmproxy.models.http.HTTPRequest">HTTPRequest</a> and <a class="reference external" href="api.html#mitmproxy.models.http.HTTPResponse">HTTPResponse</a> objects have a handy <a class="reference external" href="api.html#mitmproxy.models.http.HTTPResponse.replace">replace</a> method that takes care of all the details for us.</p> <div class="literal-block-wrapper docutils container" id="id5"> <div class="code-block-caption"><span class="caption-text"><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/examples/simple/script_arguments.py">examples/simple/script_arguments.py</a></span><a class="headerlink" href="#id5" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">argparse</span> <span class="k">class</span> <span class="nc">Replacer</span><span class="p">:</span> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">src</span><span class="p">,</span> <span class="n">dst</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">src</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">dst</span> <span class="o">=</span> <span class="n">src</span><span class="p">,</span> <span class="n">dst</span> <span class="k">def</span> <span class="nf">response</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">flow</span><span class="p">):</span> <span class="n">flow</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">src</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">dst</span><span class="p">)</span> <span class="k">def</span> <span class="nf">start</span><span class="p">():</span> <span class="n">parser</span> <span class="o">=</span> <span class="n">argparse</span><span class="o">.</span><span class="n">ArgumentParser</span><span class="p">()</span> <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"src"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span> <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"dst"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span> <span class="n">args</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">parse_args</span><span class="p">()</span> <span class="k">return</span> <span class="n">Replacer</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">src</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">dst</span><span class="p">)</span> </pre></div> </div> </div> <p>We can now call this script on the command-line like this:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">dd</span> <span class="o">-</span><span class="n">s</span> <span class="s2">"./script_arguments.py html faketml"</span> </pre></div> </div> <p>Whenever a handler is called, mitpmroxy rewrites the script environment so that it sees its own arguments as if it was invoked from the command-line.</p> </div> <div class="section" id="logging-and-the-context"> <h3>Logging and the context<a class="headerlink" href="#logging-and-the-context" title="Permalink to this headline">露</a></h3> <p>Scripts should not output straight to stderr or stdout. Instead, the <a class="reference external" href="api.html#mitmproxy.controller.Log">log</a> object on the <code class="docutils literal"><span class="pre">ctx</span></code> context module should be used, so that the mitmproxy host program can handle output appropriately. So, mitmdump can print colorised script output to the terminal, and mitmproxy console can place script output in the event buffer.</p> <p>Here’s how this looks:</p> <div class="literal-block-wrapper docutils container" id="id6"> <div class="code-block-caption"><span class="caption-text"><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/examples/simple/log_events.py">examples/simple/log_events.py</a></span><a class="headerlink" href="#id6" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="sd">"""</span> <span class="sd">It is recommended to use `ctx.log` for logging within a script.</span> <span class="sd">This goes to the event log in mitmproxy and to stdout in mitmdump.</span> <span class="sd">If you want to help us out: https://github.com/mitmproxy/mitmproxy/issues/1530 :-)</span> <span class="sd">"""</span> <span class="kn">from</span> <span class="nn">mitmproxy</span> <span class="kn">import</span> <span class="n">ctx</span> <span class="k">def</span> <span class="nf">start</span><span class="p">():</span> <span class="n">ctx</span><span class="o">.</span><span class="n">log</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"This is some informative text."</span><span class="p">)</span> <span class="n">ctx</span><span class="o">.</span><span class="n">log</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s2">"This is an error."</span><span class="p">)</span> </pre></div> </div> </div> <p>The <code class="docutils literal"><span class="pre">ctx</span></code> module also exposes the mitmproxy master object at <code class="docutils literal"><span class="pre">ctx.master</span></code> for advanced usage.</p> </div> <div class="section" id="running-scripts-on-saved-flows"> <h3>Running scripts on saved flows<a class="headerlink" href="#running-scripts-on-saved-flows" title="Permalink to this headline">露</a></h3> <p>When a flow is loaded from disk, the sequence of events that the flow would have gone through on the wire is partially replayed. So, for instance, an HTTP flow loaded from disk will trigger <a class="reference external" href="events.html#requestheaders">requestheaders</a>, <a class="reference external" href="events.html#request">request</a>, <a class="reference external" href="events.html#responseheaders">responseheaders</a> and <a class="reference external" href="events.html#response">response</a> in order. We can use this behaviour to transform saved traffic using scripts. For example, we can invoke the replacer script from above on saved traffic as follows:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">dd</span> <span class="o">-</span><span class="n">s</span> <span class="s2">"./arguments.py html fakehtml"</span> <span class="o">-</span><span class="n">r</span> <span class="n">saved</span> <span class="o">-</span><span class="n">w</span> <span class="n">changed</span> </pre></div> </div> <p>This command starts the <code class="docutils literal"><span class="pre">arguments</span></code> script, reads all the flows from <code class="docutils literal"><span class="pre">saved</span></code> transforming them in the process, then writes them all to <code class="docutils literal"><span class="pre">changed</span></code>.</p> <p>The mitmproxy console tool provides interactive ways to run transforming scripts on flows - for instance, you can run a one-shot script on a single flow through the <code class="docutils literal"><span class="pre">|</span></code> (pipe) shortcut.</p> </div> <div class="section" id="concurrency"> <h3>Concurrency<a class="headerlink" href="#concurrency" title="Permalink to this headline">露</a></h3> <p>The mitmproxy script mechanism is single threaded, and the proxy blocks while script handlers execute. This hugely simplifies the most common case, where handlers are light-weight and the blocking doesn’t have a performance impact. It’s possible to implement a concurrent mechanism on top of the blocking framework, and mitmproxy includes a handy example of this that is fit for most purposes. You can use it as follows:</p> <div class="literal-block-wrapper docutils container" id="id7"> <div class="code-block-caption"><span class="caption-text"><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/examples/complex/nonblocking.py">examples/complex/nonblocking.py</a></span><a class="headerlink" href="#id7" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">time</span> <span class="kn">from</span> <span class="nn">mitmproxy.script</span> <span class="kn">import</span> <span class="n">concurrent</span> <span class="nd">@concurrent</span> <span class="c1"># Remove this and see what happens</span> <span class="k">def</span> <span class="nf">request</span><span class="p">(</span><span class="n">flow</span><span class="p">):</span> <span class="c1"># You don't want to use mitmproxy.ctx from a different thread</span> <span class="k">print</span><span class="p">(</span><span class="s2">"handle request: </span><span class="si">%s%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">flow</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">host</span><span class="p">,</span> <span class="n">flow</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">path</span><span class="p">))</span> <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span> <span class="k">print</span><span class="p">(</span><span class="s2">"start request: </span><span class="si">%s%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">flow</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">host</span><span class="p">,</span> <span class="n">flow</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">path</span><span class="p">))</span> </pre></div> </div> </div> </div> <div class="section" id="testing"> <h3>Testing<a class="headerlink" href="#testing" title="Permalink to this headline">露</a></h3> <p>Mitmproxy includes a number of helpers for testing addons. The <code class="docutils literal"><span class="pre">mitmproxy.test.taddons</span></code> module contains a context helper that takes care of setting up and tearing down the addon event context. The <code class="docutils literal"><span class="pre">mitmproxy.test.tflow</span></code> module contains helpers for quickly creating test flows. Pydoc is the canonical reference for these modules, and mitmproxy’s own test suite is an excellent source of examples of usage. Here, for instance, is the mitmproxy unit tests for the <cite>anticache</cite> option, demonstrating a good cross-section of the test helpers:</p> <div class="literal-block-wrapper docutils container" id="id8"> <div class="code-block-caption"><span class="caption-text"><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/test/mitmproxy/addons/test_anticache.py">test/mitmproxy/addons/test_anticache.py</a></span><a class="headerlink" href="#id8" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">mitmproxy.test</span> <span class="kn">import</span> <span class="n">tflow</span> <span class="kn">from</span> <span class="nn">mitmproxy.addons</span> <span class="kn">import</span> <span class="n">anticache</span> <span class="kn">from</span> <span class="nn">mitmproxy.test</span> <span class="kn">import</span> <span class="n">taddons</span> <span class="k">class</span> <span class="nc">TestAntiCache</span><span class="p">:</span> <span class="k">def</span> <span class="nf">test_simple</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="n">sa</span> <span class="o">=</span> <span class="n">anticache</span><span class="o">.</span><span class="n">AntiCache</span><span class="p">()</span> <span class="k">with</span> <span class="n">taddons</span><span class="o">.</span><span class="n">context</span><span class="p">()</span> <span class="k">as</span> <span class="n">tctx</span><span class="p">:</span> <span class="n">f</span> <span class="o">=</span> <span class="n">tflow</span><span class="o">.</span><span class="n">tflow</span><span class="p">(</span><span class="n">resp</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="n">f</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="p">[</span><span class="s2">"if-modified-since"</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"test"</span> <span class="n">f</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="p">[</span><span class="s2">"if-none-match"</span><span class="p">]</span> <span class="o">=</span> <span class="s2">"test"</span> <span class="n">sa</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="k">assert</span> <span class="s2">"if-modified-since"</span> <span class="ow">in</span> <span class="n">f</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span> <span class="k">assert</span> <span class="s2">"if-none-match"</span> <span class="ow">in</span> <span class="n">f</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span> <span class="n">tctx</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span><span class="n">sa</span><span class="p">,</span> <span class="n">anticache</span> <span class="o">=</span> <span class="bp">True</span><span class="p">)</span> <span class="n">sa</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="k">assert</span> <span class="s2">"if-modified-since"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">f</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span> <span class="k">assert</span> <span class="s2">"if-none-match"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">f</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">headers</span> </pre></div> </div> </div> </div> <div class="section" id="developing-scripts"> <h3>Developing scripts<a class="headerlink" href="#developing-scripts" title="Permalink to this headline">露</a></h3> <p>Mitmproxy monitors scripts for modifications, and reloads them on change. When this happens, the script is shut down (the <a class="reference external" href="events.html#done">done</a> event is called), and the new instance is started up as if the script had just been loaded (the <a class="reference external" href="events.html#start">start</a> and <a class="reference external" href="events.html#configure">configure</a> events are called).</p> </div> </div> <span id="document-scripting/events"></span><div class="section" id="events"> <span id="id1"></span><h2>Events<a class="headerlink" href="#events" title="Permalink to this headline">露</a></h2> <div class="section" id="general"> <h3>General<a class="headerlink" href="#general" title="Permalink to this headline">露</a></h3> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="40%" /> <col width="60%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td><dl class="first last function"> <dt id="configure"> <code class="descname">configure</code><span class="sig-paren">(</span><em>options</em>, <em>updated</em><span class="sig-paren">)</span><a class="headerlink" href="#configure" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called once on startup, and whenever options change.</p> <dl class="last docutils"> <dt><em>options</em></dt> <dd>An <code class="docutils literal"><span class="pre">options.Options</span></code> object with the total current configuration state of mitmproxy.</dd> <dt><em>updated</em></dt> <dd>A set of strings indicating which configuration options have been updated. This contains all options when <em>configure</em> is called on startup, and only changed options subsequently.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="done"> <code class="descname">done</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#done" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td>Called once when the script shuts down, either because it’s been unloaded, or because the proxy itself is shutting down.</td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="log"> <code class="descname">log</code><span class="sig-paren">(</span><em>entry</em><span class="sig-paren">)</span><a class="headerlink" href="#log" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called whenever an event log is added.</p> <dl class="last docutils"> <dt><em>entry</em></dt> <dd>An <code class="docutils literal"><span class="pre">controller.LogEntry</span></code> object - <code class="docutils literal"><span class="pre">entry.msg</span></code> is the log text, and <code class="docutils literal"><span class="pre">entry.level</span></code> is the urgency level (“debug”, “info”, “warn”, “error”).</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="start"> <code class="descname">start</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#start" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td>Called once on startup, before any other events. If you return a value from this event, it will replace the current addon. This allows you to, “boot into” an addon implemented as a class instance from the module level.</td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="tick"> <code class="descname">tick</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#tick" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td>Called at a regular sub-second interval as long as the addon is executing.</td> </tr> </tbody> </table> </div> <div class="section" id="connection"> <h3>Connection<a class="headerlink" href="#connection" title="Permalink to this headline">露</a></h3> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="40%" /> <col width="60%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td><dl class="first last function"> <dt id="clientconnect"> <code class="descname">clientconnect</code><span class="sig-paren">(</span><em>root_layer</em><span class="sig-paren">)</span><a class="headerlink" href="#clientconnect" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a client initiates a connection to the proxy. Note that a connection can correspond to multiple HTTP requests.</p> <dl class="last docutils"> <dt><em>root_layer</em></dt> <dd>The root layer (see <cite>mitmproxy.proxy.protocol</cite> for an explanation what the root layer is), provides transparent access to all attributes of the <code class="xref py py-class docutils literal"><span class="pre">RootContext</span></code>. For example, <code class="docutils literal"><span class="pre">root_layer.client_conn.address</span></code> gives the remote address of the connecting client.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="clientdisconnect"> <code class="descname">clientdisconnect</code><span class="sig-paren">(</span><em>root_layer</em><span class="sig-paren">)</span><a class="headerlink" href="#clientdisconnect" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a client disconnects from the proxy.</p> <dl class="last docutils"> <dt><em>root_layer</em></dt> <dd>The root layer object.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="next_layer"> <code class="descname">next_layer</code><span class="sig-paren">(</span><em>layer</em><span class="sig-paren">)</span><a class="headerlink" href="#next_layer" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called whenever layers are switched. You may change which layer will be used by returning a new layer object from this event.</p> <dl class="last docutils"> <dt><em>layer</em></dt> <dd>The next layer, as determined by mitmpmroxy.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="serverconnect"> <code class="descname">serverconnect</code><span class="sig-paren">(</span><em>server_conn</em><span class="sig-paren">)</span><a class="headerlink" href="#serverconnect" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called before the proxy initiates a connection to the target server. Note that a connection can correspond to multiple HTTP requests.</p> <dl class="last docutils"> <dt><em>server_conn</em></dt> <dd>A <code class="docutils literal"><span class="pre">ServerConnection</span></code> object. It is guaranteed to have a non-None <code class="docutils literal"><span class="pre">address</span></code> attribute.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="serverdisconnect"> <code class="descname">serverdisconnect</code><span class="sig-paren">(</span><em>server_conn</em><span class="sig-paren">)</span><a class="headerlink" href="#serverdisconnect" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when the proxy has closed the server connection.</p> <dl class="last docutils"> <dt><em>server_conn</em></dt> <dd>A <code class="docutils literal"><span class="pre">ServerConnection</span></code> object.</dd> </dl> </td> </tr> </tbody> </table> </div> <div class="section" id="http-events"> <h3>HTTP Events<a class="headerlink" href="#http-events" title="Permalink to this headline">露</a></h3> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="40%" /> <col width="60%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td><dl class="first last function"> <dt id="http_connect"> <code class="descname">http_connect</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#http_connect" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when we receive an HTTP CONNECT request. Setting a non 2xx response on the flow will return the response to the client abort the connection. CONNECT requests and responses do not generate the usual HTTP handler events. CONNECT requests are only valid in regular and upstream proxy modes.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.HTTPFlow</span></code> object. The flow is guaranteed to have non-None <code class="docutils literal"><span class="pre">request</span></code> and <code class="docutils literal"><span class="pre">requestheaders</span></code> attributes.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="request"> <code class="descname">request</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#request" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a client request has been received.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.HTTPFlow</span></code> object. At this point, the flow is guaranteed to have a non-None <code class="docutils literal"><span class="pre">request</span></code> attribute.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="requestheaders"> <code class="descname">requestheaders</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#requestheaders" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when the headers of a client request have been received, but before the request body is read.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.HTTPFlow</span></code> object. At this point, the flow is guaranteed to have a non-None <code class="docutils literal"><span class="pre">request</span></code> attribute.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="responseheaders"> <code class="descname">responseheaders</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#responseheaders" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when the headers of a server response have been received, but before the response body is read.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.HTTPFlow</span></code> object. At this point, the flow is guaranteed to have a non-none <code class="docutils literal"><span class="pre">request</span></code> and <code class="docutils literal"><span class="pre">response</span></code> attributes, however the response will have no content.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="response"> <code class="descname">response</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#response" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a server response has been received.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.HTTPFlow</span></code> object. At this point, the flow is guaranteed to have a non-none <code class="docutils literal"><span class="pre">request</span></code> and <code class="docutils literal"><span class="pre">response</span></code> attributes. The raw response body will be in <code class="docutils literal"><span class="pre">response.body</span></code>, unless response streaming has been enabled.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="error"> <code class="descname">error</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#error" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a flow error has occurred, e.g. invalid server responses, or interrupted connections. This is distinct from a valid server HTTP error response, which is simply a response with an HTTP error code.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>The flow containing the error. It is guaranteed to have non-None <code class="docutils literal"><span class="pre">error</span></code> attribute.</dd> </dl> </td> </tr> </tbody> </table> </div> <div class="section" id="websocket-events"> <h3>WebSocket Events<a class="headerlink" href="#websocket-events" title="Permalink to this headline">露</a></h3> <p>These events are called only after a connection made an HTTP upgrade with “101 Switching Protocols”. No further HTTP-related events after the handshake are issued, only new WebSocket messages are called.</p> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="40%" /> <col width="60%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td><dl class="first last function"> <dt id="websocket_handshake"> <code class="descname">websocket_handshake</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#websocket_handshake" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a client wants to establish a WebSocket connection. The WebSocket-specific headers can be manipulated to alter the handshake. The <code class="docutils literal"><span class="pre">flow</span></code> object is guaranteed to have a non-None <code class="docutils literal"><span class="pre">request</span></code> attribute.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>The flow containing the HTTP WebSocket handshake request. The object is guaranteed to have a non-None <code class="docutils literal"><span class="pre">request</span></code> attribute.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="websocket_start"> <code class="descname">websocket_start</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#websocket_start" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when WebSocket connection is established after a successful handshake.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.WebSocketFlow</span></code> object.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="websocket_message"> <code class="descname">websocket_message</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#websocket_message" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a WebSocket message is received from the client or server. The sender and receiver are identifiable. The most recent message will be <code class="docutils literal"><span class="pre">flow.messages[-1]</span></code>. The message is user-modifiable. Currently there are two types of messages, corresponding to the BINARY and TEXT frame types.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.WebSocketFlow</span></code> object.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="websocket_end"> <code class="descname">websocket_end</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#websocket_end" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when WebSocket connection ends.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.WebSocketFlow</span></code> object.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="websocket_error"> <code class="descname">websocket_error</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#websocket_error" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a WebSocket error occurs - e.g. the connection closing unexpectedly.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.WebSocketFlow</span></code> object.</dd> </dl> </td> </tr> </tbody> </table> </div> <div class="section" id="tcp-events"> <h3>TCP Events<a class="headerlink" href="#tcp-events" title="Permalink to this headline">露</a></h3> <p>These events are called only if the connection is in <a class="reference internal" href="index.html#tcpproxy"><span class="std std-ref">TCP mode</span></a>. So, for instance, TCP events are not called for ordinary HTTP/S connections.</p> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="40%" /> <col width="60%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td><dl class="first last function"> <dt id="tcp_start"> <code class="descname">tcp_start</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#tcp_start" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when TCP streaming starts.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.TCPFlow</span></code> object.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="tcp_message"> <code class="descname">tcp_message</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#tcp_message" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a TCP payload is received from the client or server. The sender and receiver are identifiable. The most recent message will be <code class="docutils literal"><span class="pre">flow.messages[-1]</span></code>. The message is user-modifiable.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.TCPFlow</span></code> object.</dd> </dl> </td> </tr> <tr class="row-odd"><td><dl class="first last function"> <dt id="tcp_end"> <code class="descname">tcp_end</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#tcp_end" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when TCP streaming ends.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.TCPFlow</span></code> object.</dd> </dl> </td> </tr> <tr class="row-even"><td><dl class="first last function"> <dt id="tcp_error"> <code class="descname">tcp_error</code><span class="sig-paren">(</span><em>flow</em><span class="sig-paren">)</span><a class="headerlink" href="#tcp_error" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </td> <td><p class="first">Called when a TCP error occurs - e.g. the connection closing unexpectedly.</p> <dl class="last docutils"> <dt><em>flow</em></dt> <dd>A <code class="docutils literal"><span class="pre">models.TCPFlow</span></code> object.</dd> </dl> </td> </tr> </tbody> </table> </div> </div> <span id="document-scripting/api"></span><div class="section" id="api"> <span id="id1"></span><h2>API<a class="headerlink" href="#api" title="Permalink to this headline">露</a></h2> <ul class="simple"> <li><dl class="first docutils"> <dt>Errors</dt> <dd><ul class="first last"> <li><a class="reference external" href="#mitmproxy.flow.Error">mitmproxy.flow.Error</a></li> </ul> </dd> </dl> </li> <li><dl class="first docutils"> <dt>HTTP</dt> <dd><ul class="first last"> <li><a class="reference external" href="#mitmproxy.http.HTTPRequest">mitmproxy.http.HTTPRequest</a></li> <li><a class="reference external" href="#mitmproxy.http.HTTPResponse">mitmproxy.http.HTTPResponse</a></li> <li><a class="reference external" href="#mitmproxy.http.HTTPFlow">mitmproxy.http.HTTPFlow</a></li> </ul> </dd> </dl> </li> <li><dl class="first docutils"> <dt>Logging</dt> <dd><ul class="first last"> <li><a class="reference external" href="#mitmproxy.controller.Log">mitmproxy.log.Log</a></li> <li><a class="reference external" href="#mitmproxy.controller.LogEntry">mitmproxy.log.LogEntry</a></li> </ul> </dd> </dl> </li> </ul> <div class="section" id="errors"> <h3>Errors<a class="headerlink" href="#errors" title="Permalink to this headline">露</a></h3> <dl class="class"> <dt id="mitmproxy.flow.Error"> <em class="property">class </em><code class="descclassname">mitmproxy.flow.</code><code class="descname">Error</code><span class="sig-paren">(</span><em>msg: str</em>, <em>timestamp=None</em><span class="sig-paren">)</span> → None<a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L12"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/flow.html#Error"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.flow.Error" title="Permalink to this definition">露</a></dt> <dd><p>An Error.</p> <p>This is distinct from an protocol error response (say, a HTTP code 500), which is represented by a normal HTTPResponse object. This class is responsible for indicating errors that fall outside of normal protocol communications, like interrupted connections, timeouts, protocol errors.</p> <p>Exposes the following attributes:</p> <blockquote> <div>msg: Message describing the error timestamp: Seconds since the epoch</div></blockquote> <dl class="method"> <dt id="mitmproxy.flow.Error.get_state"> <code class="descname">get_state</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/stateobject.py#L29"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.flow.Error.get_state" title="Permalink to this definition">露</a></dt> <dd><p>Retrieve object state.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.flow.Error.set_state"> <code class="descname">set_state</code><span class="sig-paren">(</span><em>state</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/stateobject.py#L46"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.flow.Error.set_state" title="Permalink to this definition">露</a></dt> <dd><p>Load object state from data returned by a get_state call.</p> </dd></dl> </dd></dl> </div> <div class="section" id="http"> <h3>HTTP<a class="headerlink" href="#http" title="Permalink to this headline">露</a></h3> <dl class="class"> <dt id="mitmproxy.http.HTTPRequest"> <em class="property">class </em><code class="descclassname">mitmproxy.http.</code><code class="descname">HTTPRequest</code><span class="sig-paren">(</span><em>first_line_format</em>, <em>method</em>, <em>scheme</em>, <em>host</em>, <em>port</em>, <em>path</em>, <em>http_version</em>, <em>headers</em>, <em>content</em>, <em>timestamp_start=None</em>, <em>timestamp_end=None</em>, <em>is_replay=False</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/http.py#L12"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/http.html#HTTPRequest"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest" title="Permalink to this definition">露</a></dt> <dd><p>A mitmproxy HTTP request.</p> <dl class="classmethod"> <dt id="mitmproxy.http.HTTPRequest.wrap"> <em class="property">classmethod </em><code class="descname">wrap</code><span class="sig-paren">(</span><em>request</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/http.py#L65"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/http.html#HTTPRequest.wrap"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.wrap" title="Permalink to this definition">露</a></dt> <dd><p>Wraps an existing <code class="xref py py-class docutils literal"><span class="pre">mitmproxy.net.http.Request</span></code>.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.anticache"> <code class="descname">anticache</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/request.py#L348"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.anticache" title="Permalink to this definition">露</a></dt> <dd><p>Modifies this request to remove headers that might produce a cached response. That is, we remove ETags and If-Modified-Since headers.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.anticomp"> <code class="descname">anticomp</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/request.py#L360"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.anticomp" title="Permalink to this definition">露</a></dt> <dd><p>Modifies this request to remove headers that will compress the resource’s data.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.constrain_encoding"> <code class="descname">constrain_encoding</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/request.py#L367"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.constrain_encoding" title="Permalink to this definition">露</a></dt> <dd><p>Limits the permissible Accept-Encoding values, based on what we can decode appropriately.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.content"> <code class="descname">content</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.content" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with the content-encoding header (e.g. gzip)</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the content-encoding is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code>, <code class="xref py py-attr docutils literal"><span class="pre">text</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.cookies"> <code class="descname">cookies</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.cookies" title="Permalink to this definition">露</a></dt> <dd><p>The request cookies.</p> <p>An empty <code class="xref py py-class docutils literal"><span class="pre">MultiDictView</span></code> object if the cookie monster ate them all.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.decode"> <code class="descname">decode</code><span class="sig-paren">(</span><em>strict=True</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L221"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.decode" title="Permalink to this definition">露</a></dt> <dd><p>Decodes body based on the current Content-Encoding header, then removes the header. If there is no Content-Encoding header, no action is taken.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the content-encoding is invalid and strict is True.</td> </tr> </tbody> </table> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.encode"> <code class="descname">encode</code><span class="sig-paren">(</span><em>e</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L233"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.encode" title="Permalink to this definition">露</a></dt> <dd><p>Encodes body with the encoding e, where e is “gzip”, “deflate”, “identity”, or “br”. Any existing content-encodings are overwritten, the content is not decoded beforehand.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the specified content-encoding is invalid.</td> </tr> </tbody> </table> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.first_line_format"> <code class="descname">first_line_format</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.first_line_format" title="Permalink to this definition">露</a></dt> <dd><p>HTTP request form as defined in <a class="reference external" href="https--//tools.ietf.org/html/rfc7230#section-5.3">RFC7230</a>.</p> <p>origin-form and asterisk-form are subsumed as “relative”.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.get_content"> <code class="descname">get_content</code><span class="sig-paren">(</span><em>strict: bool = True</em><span class="sig-paren">)</span> → bytes<a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L83"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.get_content" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with the content-encoding header (e.g. gzip)</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the content-encoding is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code>, <code class="xref py py-attr docutils literal"><span class="pre">text</span></code></p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.get_text"> <code class="descname">get_text</code><span class="sig-paren">(</span><em>strict: bool = True</em><span class="sig-paren">)</span> → typing.Union[str, NoneType]<a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L181"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.get_text" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with both content-encoding header (e.g. gzip) and content-type header charset.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when either content-encoding or charset is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-attr docutils literal"><span class="pre">content</span></code>, <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.headers"> <code class="descname">headers</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.headers" title="Permalink to this definition">露</a></dt> <dd><p>Message headers object</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body">mitmproxy.net.http.Headers</td> </tr> </tbody> </table> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.host"> <code class="descname">host</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.host" title="Permalink to this definition">露</a></dt> <dd><p>Target host. This may be parsed from the raw request (e.g. from a <code class="docutils literal"><span class="pre">GET</span> <span class="pre">http://example.com/</span> <span class="pre">HTTP/1.1</span></code> request line) or inferred from the proxy mode (e.g. an IP in transparent mode).</p> <p>Setting the host attribute also updates the host header, if present.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.host_header"> <code class="descname">host_header</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.host_header" title="Permalink to this definition">露</a></dt> <dd><p>The request’s host/authority header.</p> <p>This property maps to either <code class="docutils literal"><span class="pre">request.headers["Host"]</span></code> or <code class="docutils literal"><span class="pre">request.headers[":authority"]</span></code>, depending on whether it’s HTTP/1.x or HTTP/2.0.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.http_version"> <code class="descname">http_version</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.http_version" title="Permalink to this definition">露</a></dt> <dd><p>Version string, e.g. “HTTP/1.1”</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.method"> <code class="descname">method</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.method" title="Permalink to this definition">露</a></dt> <dd><p>HTTP request method, e.g. “GET”.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.multipart_form"> <code class="descname">multipart_form</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.multipart_form" title="Permalink to this definition">露</a></dt> <dd><p>The multipart form data as an <code class="xref py py-class docutils literal"><span class="pre">MultiDictView</span></code> object. An empty multidict.MultiDictView if the content-type indicates non-form data or the content could not be parsed.</p> <p>Key and value are bytes.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.path"> <code class="descname">path</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.path" title="Permalink to this definition">露</a></dt> <dd><p>HTTP request path, e.g. “/index.html”. Guaranteed to start with a slash, except for OPTIONS requests, which may just be “*”.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.path_components"> <code class="descname">path_components</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.path_components" title="Permalink to this definition">露</a></dt> <dd><p>The URL’s path components as a tuple of strings. Components are unquoted.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.port"> <code class="descname">port</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.port" title="Permalink to this definition">露</a></dt> <dd><p>Target port</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.pretty_host"> <code class="descname">pretty_host</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.pretty_host" title="Permalink to this definition">露</a></dt> <dd><p>Similar to <code class="xref py py-attr docutils literal"><span class="pre">host</span></code>, but using the Host headers as an additional preferred data source. This is useful in transparent mode where <code class="xref py py-attr docutils literal"><span class="pre">host</span></code> is only an IP address, but may not reflect the actual destination as the Host header could be spoofed.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.pretty_url"> <code class="descname">pretty_url</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.pretty_url" title="Permalink to this definition">露</a></dt> <dd><p>Like <code class="xref py py-attr docutils literal"><span class="pre">url</span></code>, but using <code class="xref py py-attr docutils literal"><span class="pre">pretty_host</span></code> instead of <code class="xref py py-attr docutils literal"><span class="pre">host</span></code>.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.query"> <code class="descname">query</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.query" title="Permalink to this definition">露</a></dt> <dd><p>The request query string as an <code class="xref py py-class docutils literal"><span class="pre">MultiDictView</span></code> object.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.raw_content"> <code class="descname">raw_content</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.raw_content" title="Permalink to this definition">露</a></dt> <dd><p>The raw (encoded) HTTP message body</p> <p>See also: <code class="xref py py-attr docutils literal"><span class="pre">content</span></code>, <code class="xref py py-class docutils literal"><span class="pre">text</span></code></p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPRequest.replace"> <code class="descname">replace</code><span class="sig-paren">(</span><em>pattern</em>, <em>repl</em>, <em>flags=0</em>, <em>count=0</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/request.py#L80"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPRequest.replace" title="Permalink to this definition">露</a></dt> <dd><p>Replaces a regular expression pattern with repl in the headers, the request path and the body of the request. Encoded content will be decoded before replacement, and re-encoded afterwards.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body">The number of replacements made.</td> </tr> </tbody> </table> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.scheme"> <code class="descname">scheme</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.scheme" title="Permalink to this definition">露</a></dt> <dd><p>HTTP request scheme, which should be “http” or “https”.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.text"> <code class="descname">text</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.text" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with both content-encoding header (e.g. gzip) and content-type header charset.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when either content-encoding or charset is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-attr docutils literal"><span class="pre">content</span></code>, <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.timestamp_end"> <code class="descname">timestamp_end</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.timestamp_end" title="Permalink to this definition">露</a></dt> <dd><p>Last byte timestamp</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.timestamp_start"> <code class="descname">timestamp_start</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.timestamp_start" title="Permalink to this definition">露</a></dt> <dd><p>First byte timestamp</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.url"> <code class="descname">url</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.url" title="Permalink to this definition">露</a></dt> <dd><p>The URL string, constructed from the request’s URL components</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPRequest.urlencoded_form"> <code class="descname">urlencoded_form</code><a class="headerlink" href="#mitmproxy.http.HTTPRequest.urlencoded_form" title="Permalink to this definition">露</a></dt> <dd><p>The URL-encoded form data as an <code class="xref py py-class docutils literal"><span class="pre">MultiDictView</span></code> object. An empty multidict.MultiDictView if the content-type indicates non-form data or the content could not be parsed.</p> <p>Starting with mitmproxy 1.0, key and value are strings.</p> </dd></dl> </dd></dl> <dl class="class"> <dt id="mitmproxy.http.HTTPResponse"> <em class="property">class </em><code class="descclassname">mitmproxy.http.</code><code class="descname">HTTPResponse</code><span class="sig-paren">(</span><em>http_version</em>, <em>status_code</em>, <em>reason</em>, <em>headers</em>, <em>content</em>, <em>timestamp_start=None</em>, <em>timestamp_end=None</em>, <em>is_replay=False</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/http.py#L89"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/http.html#HTTPResponse"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse" title="Permalink to this definition">露</a></dt> <dd><p>A mitmproxy HTTP response.</p> <dl class="classmethod"> <dt id="mitmproxy.http.HTTPResponse.wrap"> <em class="property">classmethod </em><code class="descname">wrap</code><span class="sig-paren">(</span><em>response</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/http.py#L123"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/http.html#HTTPResponse.wrap"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.wrap" title="Permalink to this definition">露</a></dt> <dd><p>Wraps an existing <code class="xref py py-class docutils literal"><span class="pre">mitmproxy.net.http.Response</span></code>.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.content"> <code class="descname">content</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.content" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with the content-encoding header (e.g. gzip)</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the content-encoding is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code>, <code class="xref py py-attr docutils literal"><span class="pre">text</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.cookies"> <code class="descname">cookies</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.cookies" title="Permalink to this definition">露</a></dt> <dd><p>The response cookies. A possibly empty <code class="xref py py-class docutils literal"><span class="pre">MultiDictView</span></code>, where the keys are cookie name strings, and values are (value, attr) tuples. Value is a string, and attr is an MultiDictView containing cookie attributes. Within attrs, unary attributes (e.g. HTTPOnly) are indicated by a Null value.</p> <dl class="docutils"> <dt>Caveats:</dt> <dd>Updating the attr</dd> </dl> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.decode"> <code class="descname">decode</code><span class="sig-paren">(</span><em>strict=True</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L221"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.decode" title="Permalink to this definition">露</a></dt> <dd><p>Decodes body based on the current Content-Encoding header, then removes the header. If there is no Content-Encoding header, no action is taken.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the content-encoding is invalid and strict is True.</td> </tr> </tbody> </table> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.encode"> <code class="descname">encode</code><span class="sig-paren">(</span><em>e</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L233"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.encode" title="Permalink to this definition">露</a></dt> <dd><p>Encodes body with the encoding e, where e is “gzip”, “deflate”, “identity”, or “br”. Any existing content-encodings are overwritten, the content is not decoded beforehand.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the specified content-encoding is invalid.</td> </tr> </tbody> </table> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.get_content"> <code class="descname">get_content</code><span class="sig-paren">(</span><em>strict: bool = True</em><span class="sig-paren">)</span> → bytes<a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L83"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.get_content" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with the content-encoding header (e.g. gzip)</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when the content-encoding is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code>, <code class="xref py py-attr docutils literal"><span class="pre">text</span></code></p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.get_text"> <code class="descname">get_text</code><span class="sig-paren">(</span><em>strict: bool = True</em><span class="sig-paren">)</span> → typing.Union[str, NoneType]<a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L181"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.get_text" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with both content-encoding header (e.g. gzip) and content-type header charset.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when either content-encoding or charset is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-attr docutils literal"><span class="pre">content</span></code>, <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.headers"> <code class="descname">headers</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.headers" title="Permalink to this definition">露</a></dt> <dd><p>Message headers object</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body">mitmproxy.net.http.Headers</td> </tr> </tbody> </table> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.http_version"> <code class="descname">http_version</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.http_version" title="Permalink to this definition">露</a></dt> <dd><p>Version string, e.g. “HTTP/1.1”</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.make"> <code class="descname">make</code><span class="sig-paren">(</span><em>status_code: int = 200, content: AnyStr = b'', headers: typing.Union[typing.Dict[AnyStr, AnyStr], typing.Iterable[typing.Tuple[bytes, bytes]]] = ()</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/response.py#L68"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.make" title="Permalink to this definition">露</a></dt> <dd><p>Simplified API for creating response objects.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.raw_content"> <code class="descname">raw_content</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.raw_content" title="Permalink to this definition">露</a></dt> <dd><p>The raw (encoded) HTTP message body</p> <p>See also: <code class="xref py py-attr docutils literal"><span class="pre">content</span></code>, <code class="xref py py-class docutils literal"><span class="pre">text</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.reason"> <code class="descname">reason</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.reason" title="Permalink to this definition">露</a></dt> <dd><p>HTTP Reason Phrase, e.g. “Not Found”. This is always <code class="xref py py-obj docutils literal"><span class="pre">None</span></code> for HTTP2 requests, because HTTP2 responses do not contain a reason phrase.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.refresh"> <code class="descname">refresh</code><span class="sig-paren">(</span><em>now=None</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/response.py#L164"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.refresh" title="Permalink to this definition">露</a></dt> <dd><p>This fairly complex and heuristic function refreshes a server response for replay.</p> <blockquote> <div><ul class="simple"> <li>It adjusts date, expires and last-modified headers.</li> <li>It adjusts cookie expiration.</li> </ul> </div></blockquote> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPResponse.replace"> <code class="descname">replace</code><span class="sig-paren">(</span><em>pattern</em>, <em>repl</em>, <em>flags=0</em>, <em>count=0</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/net/http/message.py#L247"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPResponse.replace" title="Permalink to this definition">露</a></dt> <dd><p>Replaces a regular expression pattern with repl in both the headers and the body of the message. Encoded body will be decoded before replacement, and re-encoded afterwards.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body">The number of replacements made.</td> </tr> </tbody> </table> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.status_code"> <code class="descname">status_code</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.status_code" title="Permalink to this definition">露</a></dt> <dd><p>HTTP Status Code, e.g. <code class="docutils literal"><span class="pre">200</span></code>.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.text"> <code class="descname">text</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.text" title="Permalink to this definition">露</a></dt> <dd><p>The HTTP message body decoded with both content-encoding header (e.g. gzip) and content-type header charset.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field-odd field"><th class="field-name">Raises:</th><td class="field-body">ValueError, when either content-encoding or charset is invalid and strict is True.</td> </tr> </tbody> </table> <p>See also: <code class="xref py py-attr docutils literal"><span class="pre">content</span></code>, <code class="xref py py-class docutils literal"><span class="pre">raw_content</span></code></p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.timestamp_end"> <code class="descname">timestamp_end</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.timestamp_end" title="Permalink to this definition">露</a></dt> <dd><p>Last byte timestamp</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPResponse.timestamp_start"> <code class="descname">timestamp_start</code><a class="headerlink" href="#mitmproxy.http.HTTPResponse.timestamp_start" title="Permalink to this definition">露</a></dt> <dd><p>First byte timestamp</p> </dd></dl> </dd></dl> <dl class="class"> <dt id="mitmproxy.http.HTTPFlow"> <em class="property">class </em><code class="descclassname">mitmproxy.http.</code><code class="descname">HTTPFlow</code><span class="sig-paren">(</span><em>client_conn</em>, <em>server_conn</em>, <em>live=None</em>, <em>mode='regular'</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/http.py#L140"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/http.html#HTTPFlow"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow" title="Permalink to this definition">露</a></dt> <dd><p>An HTTPFlow is a collection of objects representing a single HTTP transaction.</p> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.request"> <code class="descname">request</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.request" title="Permalink to this definition">露</a></dt> <dd><p><code class="xref py py-class docutils literal"><span class="pre">HTTPRequest</span></code> object</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.response"> <code class="descname">response</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.response" title="Permalink to this definition">露</a></dt> <dd><p><code class="xref py py-class docutils literal"><span class="pre">HTTPResponse</span></code> object</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.error"> <code class="descname">error</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.error" title="Permalink to this definition">露</a></dt> <dd><p><code class="xref py py-class docutils literal"><span class="pre">Error</span></code> object</p> <p>Note that it’s possible for a Flow to have both a response and an error object. This might happen, for instance, when a response was received from the server, but there was an error sending it back to the client.</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.server_conn"> <code class="descname">server_conn</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.server_conn" title="Permalink to this definition">露</a></dt> <dd><p><code class="xref py py-class docutils literal"><span class="pre">ServerConnection</span></code> object</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.client_conn"> <code class="descname">client_conn</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.client_conn" title="Permalink to this definition">露</a></dt> <dd><p><code class="xref py py-class docutils literal"><span class="pre">ClientConnection</span></code> object</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.intercepted"> <code class="descname">intercepted</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.intercepted" title="Permalink to this definition">露</a></dt> <dd><p>Is this flow currently being intercepted?</p> </dd></dl> <dl class="attribute"> <dt id="mitmproxy.http.HTTPFlow.mode"> <code class="descname">mode</code><em class="property"> = None</em><a class="headerlink" href="#mitmproxy.http.HTTPFlow.mode" title="Permalink to this definition">露</a></dt> <dd><p>What mode was the proxy layer in when receiving this request?</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.backup"> <code class="descname">backup</code><span class="sig-paren">(</span><em>force=False</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L128"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.backup" title="Permalink to this definition">露</a></dt> <dd><p>Save a backup of this Flow, which can be reverted to using a call to .revert().</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.intercept"> <code class="descname">intercept</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L162"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.intercept" title="Permalink to this definition">露</a></dt> <dd><p>Intercept this Flow. Processing will stop until resume is called.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.kill"> <code class="descname">kill</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L148"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.kill" title="Permalink to this definition">露</a></dt> <dd><p>Kill this request.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.modified"> <code class="descname">modified</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L119"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.modified" title="Permalink to this definition">露</a></dt> <dd><p>Has this Flow been modified?</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.resume"> <code class="descname">resume</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L172"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.resume" title="Permalink to this definition">露</a></dt> <dd><p>Continue with the flow - called after an intercept().</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.revert"> <code class="descname">revert</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/flow.py#L136"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.revert" title="Permalink to this definition">露</a></dt> <dd><p>Revert to the last backed up state.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.http.HTTPFlow.replace"> <code class="descname">replace</code><span class="sig-paren">(</span><em>pattern</em>, <em>repl</em>, <em>*args</em>, <em>**kwargs</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/http.py#L193"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/http.html#HTTPFlow.replace"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.http.HTTPFlow.replace" title="Permalink to this definition">露</a></dt> <dd><p>Replaces a regular expression pattern with repl in both request and response of the flow. Encoded content will be decoded before replacement, and re-encoded afterwards.</p> <p>Returns the number of replacements made.</p> </dd></dl> </dd></dl> </div> <div class="section" id="logging"> <h3>Logging<a class="headerlink" href="#logging" title="Permalink to this headline">露</a></h3> <dl class="class"> <dt id="mitmproxy.log.Log"> <em class="property">class </em><code class="descclassname">mitmproxy.log.</code><code class="descname">Log</code><span class="sig-paren">(</span><em>master</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/log.py#L8"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/log.html#Log"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.log.Log" title="Permalink to this definition">露</a></dt> <dd><p>The central logger, exposed to scripts as mitmproxy.ctx.log.</p> <dl class="method"> <dt id="mitmproxy.log.Log.debug"> <code class="descname">debug</code><span class="sig-paren">(</span><em>txt</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/log.py#L15"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/log.html#Log.debug"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.log.Log.debug" title="Permalink to this definition">露</a></dt> <dd><p>Log with level debug.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.log.Log.info"> <code class="descname">info</code><span class="sig-paren">(</span><em>txt</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/log.py#L21"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/log.html#Log.info"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.log.Log.info" title="Permalink to this definition">露</a></dt> <dd><p>Log with level info.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.log.Log.warn"> <code class="descname">warn</code><span class="sig-paren">(</span><em>txt</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/log.py#L27"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/log.html#Log.warn"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.log.Log.warn" title="Permalink to this definition">露</a></dt> <dd><p>Log with level warn.</p> </dd></dl> <dl class="method"> <dt id="mitmproxy.log.Log.error"> <code class="descname">error</code><span class="sig-paren">(</span><em>txt</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/log.py#L33"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/log.html#Log.error"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.log.Log.error" title="Permalink to this definition">露</a></dt> <dd><p>Log with level error.</p> </dd></dl> </dd></dl> <dl class="class"> <dt id="mitmproxy.log.LogEntry"> <em class="property">class </em><code class="descclassname">mitmproxy.log.</code><code class="descname">LogEntry</code><span class="sig-paren">(</span><em>msg</em>, <em>level</em><span class="sig-paren">)</span><a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/v2.0.2/mitmproxy/log.py#L2"><span class="viewcode-link">[source]</span></a><a class="reference internal" href="_modules/mitmproxy/log.html#LogEntry"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mitmproxy.log.LogEntry" title="Permalink to this definition">露</a></dt> <dd></dd></dl> </div> </div> </div> <div class="toctree-wrapper compound"> <span id="document-tutorials/30second"></span><div class="section" id="client-playback-a-30-second-example"> <span id="second"></span><h2>Client playback: a 30 second example<a class="headerlink" href="#client-playback-a-30-second-example" title="Permalink to this headline">露</a></h2> <p>My local cafe is serviced by a rickety and unreliable wireless network, generously sponsored with ratepayers’ money by our city council. After connecting, you are redirected to an SSL-protected page that prompts you for a username and password. Once you’ve entered your details, you are free to enjoy the intermittent dropouts, treacle-like speeds and incorrectly configured transparent proxy.</p> <p>I tend to automate this kind of thing at the first opportunity, on the theory that time spent now will be more than made up in the long run. In this case, I might use <a class="reference external" href="https://getfirebug.com/">Firebug</a> to ferret out the form post parameters and target URL, then fire up an editor to write a little script using Python’s <a class="reference external" href="https://docs.python.org/library/urllib.html">urllib</a> to simulate a submission. That’s a lot of futzing about. With mitmproxy we can do the job in literally 30 seconds, without having to worry about any of the details. Here’s how.</p> <div class="section" id="run-mitmdump-to-record-our-http-conversation-to-a-file"> <h3>1. Run mitmdump to record our HTTP conversation to a file.<a class="headerlink" href="#run-mitmdump-to-record-our-http-conversation-to-a-file" title="Permalink to this headline">露</a></h3> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">w</span> <span class="n">wireless</span><span class="o">-</span><span class="n">login</span> </pre></div> </div> </div> <div class="section" id="point-your-browser-at-the-mitmdump-instance"> <h3>2. Point your browser at the mitmdump instance.<a class="headerlink" href="#point-your-browser-at-the-mitmdump-instance" title="Permalink to this headline">露</a></h3> <p>I use a tiny Firefox addon called <a class="reference external" href="https://addons.mozilla.org/en-us/firefox/addon/toggle-proxy-51740/">Toggle Proxy</a> to switch quickly to and from mitmproxy. I’m assuming you’ve already <a class="reference internal" href="index.html#certinstall"><span class="std std-ref">configured your browser with mitmproxy’s SSL certificate authority</span></a>.</p> </div> <div class="section" id="log-in-as-usual"> <h3>3. Log in as usual.<a class="headerlink" href="#log-in-as-usual" title="Permalink to this headline">露</a></h3> <p>And that’s it! You now have a serialized version of the login process in the file wireless-login, and you can replay it at any time like this:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmdump</span> <span class="o">-</span><span class="n">c</span> <span class="n">wireless</span><span class="o">-</span><span class="n">login</span> </pre></div> </div> </div> <div class="section" id="embellishments"> <h3>Embellishments<a class="headerlink" href="#embellishments" title="Permalink to this headline">露</a></h3> <p>We’re really done at this point, but there are a couple of embellishments we could make if we wanted. I use <a class="reference external" href="https://launchpad.net/wicd">wicd</a> to automatically join wireless networks I frequent, and it lets me specify a command to run after connecting. I used the client replay command above and voila! - totally hands-free wireless network startup.</p> <p>We might also want to prune requests that download CSS, JS, images and so forth. These add only a few moments to the time it takes to replay, but they’re not really needed and I somehow feel compelled to trim them anyway. So, we fire up the mitmproxy console tool on our serialized conversation, like so:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmproxy</span> <span class="o">-</span><span class="n">r</span> <span class="n">wireless</span><span class="o">-</span><span class="n">login</span> </pre></div> </div> <p>We can now go through and manually delete (using the <code class="kbd docutils literal"><span class="pre">d</span></code> keyboard shortcut) everything we want to trim. When we’re done, we use <code class="kbd docutils literal"><span class="pre">w</span></code> to save the conversation back to the file.</p> </div> </div> <span id="document-tutorials/gamecenter"></span><div class="section" id="setting-highscores-on-apple-s-gamecenter"> <span id="gamecenter"></span><h2>Setting highscores on Apple’s GameCenter<a class="headerlink" href="#setting-highscores-on-apple-s-gamecenter" title="Permalink to this headline">露</a></h2> <div class="section" id="the-setup"> <h3>The setup<a class="headerlink" href="#the-setup" title="Permalink to this headline">露</a></h3> <p>In this tutorial, I’m going to show you how simple it is to creatively interfere with Apple Game Center traffic using mitmproxy. To set things up, <a class="reference internal" href="index.html#certinstall"><span class="std std-ref">install the mitmproxy root certificate</span></a>. Then start mitmproxy on your desktop, and configure the iPhone to use it as a proxy.</p> </div> <div class="section" id="taking-a-look-at-the-game-center-traffic"> <h3>Taking a look at the Game Center traffic<a class="headerlink" href="#taking-a-look-at-the-game-center-traffic" title="Permalink to this headline">露</a></h3> <p>Lets take a first look at the Game Center traffic. The game I’ll use in this tutorial is <a class="reference external" href="https://itunes.apple.com/us/app/super-mega-worm/id388541990?mt=8">Super Mega Worm</a> - a great little retro-apocalyptic sidescroller for the iPhone:</p> <img alt="_images/supermega.png" class="align-center" src="_images/supermega.png" /> <p>After finishing a game (take your time), watch the traffic flowing through mitmproxy:</p> <img alt="_images/one.png" class="align-center" src="_images/one.png" /> <p>We see a bunch of things we might expect - initialisation, the retrieval of leaderboards and so forth. Then, right at the end, there’s a POST to this tantalising URL:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>https://service.gc.apple.com/WebObjects/GKGameStatsService.woa/wa/submitScore </pre></div> </div> <p>The contents of the submission are particularly interesting:</p> <div class="highlight-xml"><div class="highlight"><pre><span></span><span class="c"></span> <span class="nt"><plist</span> <span class="na">version=</span><span class="s">"1.0"</span><span class="nt">></span> <span class="nt"><dict></span> <span class="nt"><key></span>scores<span class="nt"></key></span> <span class="nt"><array></span> <span class="nt"><dict></span> <span class="nt"><key></span>category<span class="nt"></key></span> <span class="nt"><string></span>SMW_Adv_USA1<span class="nt"></string></span> <span class="nt"><key></span>context<span class="nt"></key></span> <span class="nt"><integer></span>0<span class="nt"></integer></span> <span class="nt"><key></span>score-value<span class="nt"></key></span> <span class="nt"><integer></span>55<span class="nt"></integer></span> <span class="nt"><key></span>timestamp<span class="nt"></key></span> <span class="nt"><integer></span>1363515361321<span class="nt"></integer></span> <span class="nt"></dict></span> <span class="nt"></array></span> <span class="nt"></dict></span> <span class="nt"></plist></span> <span class="c"></span> </pre></div> </div> <p>This is a <a class="reference external" href="https://en.wikipedia.org/wiki/Property_list">property list</a>, containing an identifier for the game, a score (55, in this case), and a timestamp. Looks pretty simple to mess with.</p> </div> <div class="section" id="modifying-and-replaying-the-score-submission"> <h3>Modifying and replaying the score submission<a class="headerlink" href="#modifying-and-replaying-the-score-submission" title="Permalink to this headline">露</a></h3> <p>Lets edit the score submission. First, select it in mitmproxy, then press <code class="kbd docutils literal"><span class="pre">enter</span></code> to view it. Make sure you’re viewing the request, not the response - you can use <code class="kbd docutils literal"><span class="pre">tab</span></code> to flick between the two. Now press <code class="kbd docutils literal"><span class="pre">e</span></code> for edit. You’ll be prompted for the part of the request you want to change - press <code class="kbd docutils literal"><span class="pre">r</span></code> for raw body. Your preferred editor (taken from the EDITOR environment variable) will now fire up. Lets bump the score up to something a bit more ambitious:</p> <div class="highlight-xml"><div class="highlight"><pre><span></span><span class="c"></span> <span class="nt"><plist</span> <span class="na">version=</span><span class="s">"1.0"</span><span class="nt">></span> <span class="nt"><dict></span> <span class="nt"><key></span>scores<span class="nt"></key></span> <span class="nt"><array></span> <span class="nt"><dict></span> <span class="nt"><key></span>category<span class="nt"></key></span> <span class="nt"><string></span>SMW_Adv_USA1<span class="nt"></string></span> <span class="nt"><key></span>context<span class="nt"></key></span> <span class="nt"><integer></span>0<span class="nt"></integer></span> <span class="nt"><key></span>score-value<span class="nt"></key></span> <span class="nt"><integer></span>2200272667<span class="nt"></integer></span> <span class="nt"><key></span>timestamp<span class="nt"></key></span> <span class="nt"><integer></span>1363515361321<span class="nt"></integer></span> <span class="nt"></dict></span> <span class="nt"></array></span> <span class="nt"></dict></span> <span class="nt"></plist></span> <span class="c"></span> </pre></div> </div> <p>Save the file and exit your editor.</p> <p>The final step is to replay this modified request. Simply press <code class="kbd docutils literal"><span class="pre">r</span></code> for replay.</p> </div> <div class="section" id="the-glorious-result-and-some-intrigue"> <h3>The glorious result and some intrigue<a class="headerlink" href="#the-glorious-result-and-some-intrigue" title="Permalink to this headline">露</a></h3> <img alt="_images/leaderboard.png" class="align-center" src="_images/leaderboard.png" /> <p>And that’s it - according to the records, I am the greatest Super Mega Worm player of all time.</p> <p>There’s a curious addendum to this tale. When I first wrote this tutorial, all the top competitors’ scores were the same: 2,147,483,647 (this is no longer the case, because there are now so many fellow cheaters using this tutorial). If you think that number seems familiar, you’re right: it’s 2^31-1, the maximum value you can fit into a signed 32-bit int. Now let me tell you another peculiar thing about Super Mega Worm - at the end of every game, it submits your highest previous score to the Game Center, not your current score. This means that it stores your highscore somewhere, and I’m guessing that it reads that stored score back into a signed integer. So, if you _were_ to cheat by the relatively pedestrian means of modifying the saved score on your jailbroken phone, then 2^31-1 might well be the maximum score you could get. Then again, if the game itself stores its score in a signed 32-bit int, you could get the same score through perfect play, effectively beating the game. So, which is it in this case? I’ll leave that for you to decide.</p> </div> </div> <span id="document-tutorials/transparent-dhcp"></span><div class="section" id="transparently-proxify-virtual-machines"> <span id="transparent-dhcp"></span><h2>Transparently proxify virtual machines<a class="headerlink" href="#transparently-proxify-virtual-machines" title="Permalink to this headline">露</a></h2> <p>This walkthrough illustrates how to set up transparent proxying with mitmproxy. We use VirtualBox VMs with an Ubuntu proxy machine in this example, but the general <em>Internet <–> Proxy VM <–> (Virtual) Internal Network</em> setup can be applied to other setups.</p> <div class="section" id="configure-proxy-vm"> <h3>1. Configure Proxy VM<a class="headerlink" href="#configure-proxy-vm" title="Permalink to this headline">露</a></h3> <p>On the proxy machine, <strong>eth0</strong> is connected to the internet. <strong>eth1</strong> is connected to the internal network that will be proxified and configured to use a static ip (192.168.3.1).</p> <div class="section" id="virtualbox-configuration"> <h4>VirtualBox configuration<a class="headerlink" href="#virtualbox-configuration" title="Permalink to this headline">露</a></h4> <img alt="_images/step1_vbox_eth0.png" src="_images/step1_vbox_eth0.png" /> <img alt="_images/step1_vbox_eth1.png" src="_images/step1_vbox_eth1.png" /> </div> <div class="section" id="vm-network-configuration"> <h4>VM Network Configuration<a class="headerlink" href="#vm-network-configuration" title="Permalink to this headline">露</a></h4> <img alt="_images/step1_proxy.png" class="align-center" src="_images/step1_proxy.png" /> </div> </div> <div class="section" id="configure-dhcp-and-dns"> <h3>2. Configure DHCP and DNS<a class="headerlink" href="#configure-dhcp-and-dns" title="Permalink to this headline">露</a></h3> <p>We use dnsmasq to provide DHCP and DNS in our internal network. Dnsmasq is a lightweight server designed to provide DNS (and optionally DHCP and TFTP) services to a small-scale network.</p> <ul> <li><p class="first">Before we get to that, we need to fix some Ubuntu quirks: <strong>Ubuntu >12.04</strong> runs an internal dnsmasq instance (listening on loopback only) by default <a class="reference external" href="https://www.stgraber.org/2012/02/24/dns-in-ubuntu-12-04/">[1]</a>. For our use case, this needs to be disabled by changing <code class="docutils literal"><span class="pre">dns=dnsmasq</span></code> to <code class="docutils literal"><span class="pre">#dns=dnsmasq</span></code> in <strong>/etc/NetworkManager/NetworkManager.conf</strong> and</p> <p>if on Ubuntu 16.04 or newer running:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">systemctl</span> <span class="n">restart</span> <span class="n">NetworkManager</span> </pre></div> </div> <p>if on Ubuntu 12.04 or 14.04 running:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">restart</span> <span class="n">network</span><span class="o">-</span><span class="n">manager</span> </pre></div> </div> <p>afterwards.</p> </li> <li><p class="first">Now, dnsmasq can be be installed and configured:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">dnsmasq</span> </pre></div> </div> <p>Replace <strong>/etc/dnsmasq.conf</strong> with the following configuration:</p> <div class="highlight-none"><div class="highlight"><pre><span></span># Listen for DNS requests on the internal network interface=eth1 # Act as a DHCP server, assign IP addresses to clients dhcp-range=192.168.3.10,192.168.3.100,96h # Broadcast gateway and dns server information dhcp-option=option:router,192.168.3.1 dhcp-option=option:dns-server,192.168.3.1 </pre></div> </div> <p>Apply changes:</p> <p>if on Ubuntu 16.04 or newer:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">systemctl</span> <span class="n">restart</span> <span class="n">dnsmasq</span> </pre></div> </div> <p>if on Ubuntu 12.04 or 14.04:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">sudo</span> <span class="n">service</span> <span class="n">dnsmasq</span> <span class="n">restart</span> </pre></div> </div> <p>Your <strong>proxied machine</strong> in the internal virtual network should now receive an IP address via DHCP:</p> <img alt="_images/step2_proxied_vm.png" src="_images/step2_proxied_vm.png" /> </li> </ul> </div> <div class="section" id="redirect-traffic-to-mitmproxy"> <h3>3. Redirect traffic to mitmproxy<a class="headerlink" href="#redirect-traffic-to-mitmproxy" title="Permalink to this headline">露</a></h3> <p>To redirect traffic to mitmproxy, we need to add two iptables rules:</p> <div class="highlight-none"><div class="highlight"><pre><span></span>sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-port 8080 sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j REDIRECT --to-port 8080 </pre></div> </div> </div> <div class="section" id="run-mitmproxy"> <h3>4. Run mitmproxy<a class="headerlink" href="#run-mitmproxy" title="Permalink to this headline">露</a></h3> <p>Finally, we can run mitmproxy in transparent mode with</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">mitmproxy</span> <span class="o">-</span><span class="n">T</span> </pre></div> </div> <p>The proxied machine cannot to leak any data outside of HTTP or DNS requests. If required, you can now <a class="reference internal" href="index.html#certinstall"><span class="std std-ref">install the mitmproxy certificates on the proxied machine</span></a>.</p> </div> </div> </div> <div class="toctree-wrapper compound"> <span id="document-pathod/intro"></span><div class="section" id="pathology-101"> <span id="intro"></span><h2>Pathology 101<a class="headerlink" href="#pathology-101" title="Permalink to this headline">露</a></h2> <div class="section" id="pathod"> <h3>pathod<a class="headerlink" href="#pathod" title="Permalink to this headline">露</a></h3> <p>Pathod is a pathological HTTP daemon designed to let you craft almost any conceivable HTTP response, including ones that creatively violate the standards. HTTP responses are specified using a <a class="reference internal" href="index.html#language"><span class="std std-ref">small, terse language</span></a> which pathod shares with its evil twin <a class="reference internal" href="#pathoc"><span class="std std-ref">pathoc</span></a>. To start playing with pathod, fire up the daemon:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathod</span> </pre></div> </div> <p>By default, the service listens on port 9999 of localhost, and the default crafting anchor point is the path <strong>/p/</strong>. Anything after this URL prefix is treated as a response specifier. So, hitting the following URL will generate an HTTP 200 response with 100 bytes of random data:</p> <blockquote> <div><a class="reference external" href="http://localhost:9999/p/200:b@100">http://localhost:9999/p/200:b@100</a></div></blockquote> <p>See the <a class="reference internal" href="index.html#language"><span class="std std-ref">language documentation</span></a> to get (much) fancier. The pathod daemon also takes a range of configuration options. To view those, use the command-line help:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathod</span> <span class="o">--</span><span class="n">help</span> </pre></div> </div> <div class="section" id="mimicing-a-proxy"> <h4>Mimicing a proxy<a class="headerlink" href="#mimicing-a-proxy" title="Permalink to this headline">露</a></h4> <p>Pathod automatically responds to both straight HTTP and proxy requests. For proxy requests, the upstream host is ignored, and the path portion of the URL is used to match anchors. This lets you test software that supports a proxy configuration by spoofing responses from upstream servers.</p> <p>By default, we treat all proxy CONNECT requests as HTTPS traffic, serving the response using either pathod’s built-in certificates, or the cert/key pair specified by the user. You can over-ride this behaviour if you’re testing a client that makes a non-SSL CONNECT request using the <strong>-C</strong> command-line option.</p> </div> <div class="section" id="anchors"> <h4>Anchors<a class="headerlink" href="#anchors" title="Permalink to this headline">露</a></h4> <p>Anchors provide an alternative to specifying the response in the URL. Instead, you attach a response to a pre-configured anchor point, specified with a regex. When a URL matching the regex is requested, the specified response is served.</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathod</span> <span class="o">-</span><span class="n">a</span> <span class="s2">"/foo=200"</span> </pre></div> </div> <p>Here, “/foo” is the regex specifying the anchor path, and the part after the “=” is a response specifier.</p> </div> <div class="section" id="file-access"> <h4>File Access<a class="headerlink" href="#file-access" title="Permalink to this headline">露</a></h4> <p>There are two operators in the <a class="reference internal" href="index.html#language"><span class="std std-ref">language</span></a> that load contents from file - the <strong>+</strong> operator to load an entire request specification from file, and the <strong>></strong> value specifier. In pathod, both of these operators are restricted to a directory specified at startup, or disabled if no directory is specified:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathod</span> <span class="o">-</span><span class="n">d</span> <span class="o">~/</span><span class="n">staticdir</span><span class="s2">"</span> </pre></div> </div> </div> <div class="section" id="internal-error-responses"> <h4>Internal Error Responses<a class="headerlink" href="#internal-error-responses" title="Permalink to this headline">露</a></h4> <p>Pathod uses the non-standard 800 response code to indicate internal errors, to distinguish them from crafted responses. For example, a request to:</p> <blockquote> <div><a class="reference external" href="http://localhost:9999/p/foo">http://localhost:9999/p/foo</a></div></blockquote> <p>... will return an 800 response because “foo” is not a valid page specifier.</p> </div> </div> <div class="section" id="pathoc"> <span id="id1"></span><h3>pathoc<a class="headerlink" href="#pathoc" title="Permalink to this headline">露</a></h3> <p>Pathoc is a perverse HTTP daemon designed to let you craft almost any conceivable HTTP request, including ones that creatively violate the standards. HTTP requests are specified using a <a class="reference internal" href="index.html#language"><span class="std std-ref">small, terse language</span></a>, which pathod shares with its server-side twin pathod. To view pathoc’s complete range of options, use the command-line help:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathoc</span> <span class="o">--</span><span class="n">help</span> </pre></div> </div> <div class="section" id="getting-started"> <h4>Getting Started<a class="headerlink" href="#getting-started" title="Permalink to this headline">露</a></h4> <p>The basic pattern for pathoc commands is as follows:</p> <blockquote> <div>pathoc hostname request [request ...]</div></blockquote> <p>That is, we specify the hostname to connect to, followed by one or more requests. Lets start with a simple example:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">></span> <span class="n">pathoc</span> <span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> <span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">13</span><span class="p">:</span><span class="mi">43</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span> <span class="o"><<</span> <span class="mi">302</span> <span class="n">Found</span><span class="p">:</span> <span class="mi">261</span> <span class="nb">bytes</span> </pre></div> </div> <p>Here, we make a GET request to the path / on port 80 of google.com. Pathoc’s output tells us that the server responded with a 302 redirection. We can tell pathoc to connect using SSL, in which case the default port is changed to 443 (you can over-ride the default port with the <strong>-p</strong> command-line option):</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">></span> <span class="n">pathoc</span> <span class="o">-</span><span class="n">s</span> <span class="n">www</span><span class="o">.</span><span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> <span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">14</span><span class="p">:</span><span class="mi">56</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span> <span class="o"><<</span> <span class="mi">302</span> <span class="n">Found</span><span class="p">:</span> <span class="mi">262</span> <span class="nb">bytes</span> </pre></div> </div> </div> <div class="section" id="multiple-requests"> <h4>Multiple Requests<a class="headerlink" href="#multiple-requests" title="Permalink to this headline">露</a></h4> <p>There are two ways to tell pathoc to issue multiple requests. The first is to specify them on the command-line, like so:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">></span> <span class="n">pathoc</span> <span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> <span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">21</span><span class="p">:</span><span class="mi">04</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span> <span class="o"><<</span> <span class="mi">302</span> <span class="n">Found</span><span class="p">:</span> <span class="mi">261</span> <span class="nb">bytes</span> <span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">21</span><span class="p">:</span><span class="mi">04</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span> <span class="o"><<</span> <span class="mi">302</span> <span class="n">Found</span><span class="p">:</span> <span class="mi">261</span> <span class="nb">bytes</span> </pre></div> </div> <p>In this case, pathoc issues the specified requests over the same TCP connection - so in the above example only one connection is made to google.com</p> <p>The other way to issue multiple requests is to use the <strong>-n</strong> flag:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">></span> <span class="n">pathoc</span> <span class="o">-</span><span class="n">n</span> <span class="mi">2</span> <span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> <span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">21</span><span class="p">:</span><span class="mi">04</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span> <span class="o"><<</span> <span class="mi">302</span> <span class="n">Found</span><span class="p">:</span> <span class="mi">261</span> <span class="nb">bytes</span> <span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">21</span><span class="p">:</span><span class="mi">04</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span> <span class="o"><<</span> <span class="mi">302</span> <span class="n">Found</span><span class="p">:</span> <span class="mi">261</span> <span class="nb">bytes</span> </pre></div> </div> <p>The output is identical, but two separate TCP connections are made to the upstream server. These two specification styles can be combined:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pathoc</span> <span class="o">-</span><span class="n">n</span> <span class="mi">2</span> <span class="n">google</span><span class="o">.</span><span class="n">com</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> </pre></div> </div> <p>Here, two distinct TCP connections are made, with two requests issued over each.</p> </div> <div class="section" id="basic-fuzzing"> <h4>Basic Fuzzing<a class="headerlink" href="#basic-fuzzing" title="Permalink to this headline">露</a></h4> <p>The combination of pathoc’s powerful request specification language and a few of its command-line options makes for quite a powerful basic fuzzer. Here’s an example:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pathoc</span> <span class="o">-</span><span class="n">e</span> <span class="o">-</span><span class="n">I</span> <span class="mi">200</span> <span class="o">-</span><span class="n">t</span> <span class="mi">2</span> <span class="o">-</span><span class="n">n</span> <span class="mi">1000</span> <span class="n">localhost</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span><span class="p">:</span><span class="n">b</span><span class="nd">@10</span><span class="p">:</span><span class="n">ir</span><span class="p">,</span><span class="nd">@1</span> </pre></div> </div> <p>The request specified here is a valid GET with a body consisting of 10 random bytes, but with 1 random byte inserted in a random place. This could be in the headers, in the initial request line, or in the body itself. There are a few things to note here:</p> <ul class="simple"> <li>Corrupting the request in this way will often make the server enter a state where it’s awaiting more input from the client. This is where the <strong>-t</strong> option comes in, which sets a timeout that causes pathoc to disconnect after two seconds.</li> <li>The <strong>-n</strong> option tells pathoc to repeat the request 1000 times.</li> <li>The <strong>-I</strong> option tells pathoc to ignore HTTP 200 response codes. You can use this to fine-tune what pathoc considers to be an exceptional condition, and therefore log-worthy.</li> <li>The <strong>-e</strong> option tells pathoc to print an explanation of each logged request, in the form of an expanded pathoc specification with all random portions and automatic header additions resolved. This lets you precisely replay a request that triggered an error.</li> </ul> </div> <div class="section" id="interacting-with-proxies"> <h4>Interacting with Proxies<a class="headerlink" href="#interacting-with-proxies" title="Permalink to this headline">露</a></h4> <p>Pathoc has a reasonably sophisticated suite of features for interacting with proxies. The proxy request syntax very closely mirrors that of straight HTTP, which means that it is possible to make proxy-style requests using pathoc without any additional syntax, by simply specifying a full URL instead of a simple path:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathoc</span> <span class="o">-</span><span class="n">p</span> <span class="mi">8080</span> <span class="n">localhost</span> <span class="s2">"get:'http://google.com'"</span> </pre></div> </div> <p>Another common use case is to use an HTTP CONNECT request to probe remote servers via a proxy. This is done with the <strong>-c</strong> command-line option, which allows you to specify a remote host and port pair:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathoc</span> <span class="o">-</span><span class="n">c</span> <span class="n">google</span><span class="o">.</span><span class="n">com</span><span class="p">:</span><span class="mi">80</span> <span class="o">-</span><span class="n">p</span> <span class="mi">8080</span> <span class="n">localhost</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> </pre></div> </div> <p>Note that pathoc does <strong>not</strong> negotiate SSL without being explictly instructed to do so. If you’re making a CONNECT request to an SSL-protected resource, you must also pass the <strong>-s</strong> flag:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathoc</span> <span class="o">-</span><span class="n">sc</span> <span class="n">google</span><span class="o">.</span><span class="n">com</span><span class="p">:</span><span class="mi">443</span> <span class="o">-</span><span class="n">p</span> <span class="mi">8080</span> <span class="n">localhost</span> <span class="n">get</span><span class="p">:</span><span class="o">/</span> </pre></div> </div> </div> <div class="section" id="embedded-response-specification"> <h4>Embedded response specification<a class="headerlink" href="#embedded-response-specification" title="Permalink to this headline">露</a></h4> <p>One interesting feature of the Request specification language is that you can embed a response specification in it, which is then added to the request path. Here’s an example:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathoc</span> <span class="n">localhost</span><span class="p">:</span><span class="mi">9999</span> <span class="s2">"get:/p/:s'401:ir,@1'"</span> </pre></div> </div> <p>This crafts a request that connects to the pathod server, and which then crafts a response that generates a 401, with one random byte embedded at a random point. The response specification is parsed and expanded by pathoc, so you see syntax errors immediately. This really becomes handy when combined with the <strong>-e</strong> flag to show the expanded request:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="mi">07</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">16</span> <span class="mi">12</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mi">01</span><span class="p">:</span> <span class="o">>></span> <span class="s1">'GET'</span><span class="p">:</span><span class="o">/</span><span class="n">p</span><span class="o">/</span><span class="p">:</span><span class="n">s</span><span class="s1">'401:i35,</span><span class="se">\x27\\</span><span class="s1">x1b</span><span class="se">\x27</span><span class="s1">:h</span><span class="se">\x27</span><span class="s1">Content-Length</span><span class="se">\x27</span><span class="s1">=</span><span class="se">\x27</span><span class="s1">0</span><span class="se">\x27</span><span class="s1">:h</span><span class="se">\x27</span><span class="s1">Content-Length</span><span class="se">\x27</span><span class="s1">=</span><span class="se">\x27</span><span class="s1">0</span><span class="se">\x27</span><span class="s1">'</span><span class="p">:</span><span class="n">h</span><span class="s1">'Host'</span><span class="o">=</span><span class="s1">'localhost'</span> <span class="o"><<</span> <span class="mi">401</span> <span class="n">Unauthorized</span><span class="p">:</span> <span class="mi">0</span> <span class="nb">bytes</span> </pre></div> </div> <p>Note that the embedded response has been resolved <em>before</em> being sent to the server, so that “ir,@1” (embed a random byte at a random location) has become “i15,’o’” (embed the character “o” at offset 15). You now have a pathoc request specification that is precisely reproducible, even with random components. This feature comes in terribly handy when testing a proxy, since you can now drive the server response completely from the client, and have a complete log of reproducible requests to analyze afterwards.</p> </div> </div> <div class="section" id="request-examples"> <h3>Request Examples<a class="headerlink" href="#request-examples" title="Permalink to this headline">露</a></h3> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="50%" /> <col width="50%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>get:/</td> <td>Get path /</td> </tr> <tr class="row-even"><td>get:/:b@100</td> <td>100 random bytes as the body</td> </tr> <tr class="row-odd"><td>get:/:h”Etag”=”&;drop table browsers;”</td> <td>Add a header</td> </tr> <tr class="row-even"><td>get:/:u”&;drop table browsers;”</td> <td>Add a User-Agent header</td> </tr> <tr class="row-odd"><td>get:/:b@100:dr</td> <td>Drop the connection randomly</td> </tr> <tr class="row-even"><td>get:/:b@100,ascii:ir,@1</td> <td>100 ASCII bytes as the body, and randomly inject a random byte</td> </tr> <tr class="row-odd"><td>ws:/</td> <td>Initiate a websocket handshake.</td> </tr> </tbody> </table> </div> <div class="section" id="response-examples"> <h3>Response Examples<a class="headerlink" href="#response-examples" title="Permalink to this headline">露</a></h3> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="50%" /> <col width="50%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>200</td> <td>A basic HTTP 200 response.</td> </tr> <tr class="row-even"><td>200:r</td> <td>A basic HTTP 200 response with no Content-Length header. This will hang.</td> </tr> <tr class="row-odd"><td>200:da</td> <td>Server-side disconnect after all content has been sent.</td> </tr> <tr class="row-even"><td>200:b@100</td> <td>100 random bytes as the body. A Content-Length header is added, so the disconnect is no longer needed.</td> </tr> <tr class="row-odd"><td>200:b@100:h”Etag”=”’;drop table servers;”</td> <td>Add a Server header</td> </tr> <tr class="row-even"><td>200:b@100:dr</td> <td>Drop the connection randomly</td> </tr> <tr class="row-odd"><td>200:b@100,ascii:ir,@1</td> <td>100 ASCII bytes as the body, and randomly inject a random byte</td> </tr> <tr class="row-even"><td>200:b@1k:c”text/json”</td> <td>1k of random bytes, with a text/json content type</td> </tr> <tr class="row-odd"><td>200:b@1k:p50,120</td> <td>1k of random bytes, pause for 120 seconds after 50 bytes</td> </tr> <tr class="row-even"><td>200:b@1k:pr,f</td> <td>1k of random bytes, but hang forever at a random location</td> </tr> <tr class="row-odd"><td>200:b@100:h@1k,ascii_letters=’foo’</td> <td>100 ASCII bytes as the body, randomly generated 100k header name, with the value ‘foo’.</td> </tr> </tbody> </table> </div> </div> <span id="document-pathod/language"></span><div class="section" id="language-spec"> <span id="language"></span><h2>language spec<a class="headerlink" href="#language-spec" title="Permalink to this headline">露</a></h2> <div class="section" id="http-request"> <h3>HTTP Request<a class="headerlink" href="#http-request" title="Permalink to this headline">露</a></h3> <blockquote> <div><strong>method:path:[colon-separated list of features]</strong></div></blockquote> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="20%" /> <col width="80%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>method</td> <td><p class="first">A <a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a> specifying the HTTP method to use. Standard methods do not need to be enclosed in quotes, while non-standard methods can be specified as quoted strings.</p> <p class="last">The special method <strong>ws</strong> creates a valid websocket upgrade GET request, and signals to pathoc to switch to websocket recieve mode if the server responds correctly. Apart from that, websocket requests are just like any other, and all aspects of the request can be over-ridden.</p> </td> </tr> <tr class="row-even"><td>h:<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a>=<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set a header.</td> </tr> <tr class="row-odd"><td>r</td> <td>Set the <strong>raw</strong> flag on this response. Pathod will not calculate a <em>Content-Length</em> header if a body is set.</td> </tr> <tr class="row-even"><td>c<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>A shortcut for setting the Content-Type header. Equivalent to <code class="docutils literal"><span class="pre">h"Content-Type"=VALUE</span></code></td> </tr> <tr class="row-odd"><td>u<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a> uSHORTCUT</td> <td>Set a User-Agent header on this request. You can specify either a complete <a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a>, or a User-Agent shortcut: <strong>android</strong>, <strong>blackberry</strong>, <strong>bingbot</strong>, <strong>chrome</strong>, <strong>firefox</strong>, <strong>googlebot</strong>, <strong>ie9</strong>, <strong>ipad</strong>, <strong>iphone</strong>, <strong>safari</strong>.</td> </tr> <tr class="row-even"><td>b<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set the body. The appropriate Content-Length header is added automatically unless the <strong>r</strong> flag is set.</td> </tr> <tr class="row-odd"><td>s<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>An embedded Response specification, appended to the path of the request.</td> </tr> <tr class="row-even"><td>x<a class="reference internal" href="#integer"><span class="std std-ref">INTEGER</span></a></td> <td>Repeat this message N times.</td> </tr> <tr class="row-odd"><td>d<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a></td> <td>Disconnect after OFFSET bytes (HTTP/1 only).</td> </tr> <tr class="row-even"><td>i<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a>,<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Inject the specified value at the offset (HTTP/1 only)</td> </tr> <tr class="row-odd"><td>p<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a>,SECONDS</td> <td>Pause for SECONDS seconds after OFFSET bytes. SECONDS can be an integer or “f” to pause forever (HTTP/1 only)</td> </tr> </tbody> </table> </div> <div class="section" id="http-response"> <h3>HTTP Response<a class="headerlink" href="#http-response" title="Permalink to this headline">露</a></h3> <blockquote> <div><strong>code:[colon-separated list of features]</strong></div></blockquote> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="20%" /> <col width="80%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>code</td> <td><p class="first">An integer specifying the HTTP response code.</p> <p class="last">The special method <strong>ws</strong> creates a valid websocket upgrade response (code 101), and moves pathod to websocket mode. Apart from that, websocket responses are just like any other, and all aspects of the response can be over-ridden.</p> </td> </tr> <tr class="row-even"><td>m<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>HTTP Reason message. Automatically chosen according to the response code if not specified. (HTTP/1 only)</td> </tr> <tr class="row-odd"><td>h:<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a>=<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set a header.</td> </tr> <tr class="row-even"><td>r</td> <td>Set the <strong>raw</strong> flag on this response. Pathod will not calculate a <em>Content-Length</em> header if a body is set.</td> </tr> <tr class="row-odd"><td>l<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>A shortcut for setting the Location header. Equivalent to <code class="docutils literal"><span class="pre">h"Location"=VALUE</span></code></td> </tr> <tr class="row-even"><td>c<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>A shortcut for setting the Content-Type header. Equivalent to <code class="docutils literal"><span class="pre">h"Content-Type"=VALUE</span></code></td> </tr> <tr class="row-odd"><td>b<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set the body. The appropriate Content-Length header is added automatically unless the <strong>r</strong> flag is set.</td> </tr> <tr class="row-even"><td>d<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a></td> <td>Disconnect after OFFSET bytes (HTTP/1 only).</td> </tr> <tr class="row-odd"><td>i<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a>,<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Inject the specified value at the offset (HTTP/1 only)</td> </tr> <tr class="row-even"><td>p<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a>,SECONDS</td> <td>Pause for SECONDS seconds after OFFSET bytes. SECONDS can be an integer or “f” to pause forever (HTTP/1 only)</td> </tr> </tbody> </table> </div> <div class="section" id="websocket-frame"> <h3>Websocket Frame<a class="headerlink" href="#websocket-frame" title="Permalink to this headline">露</a></h3> <blockquote> <div><strong>wf:[colon-separated list of features]</strong></div></blockquote> <table border="1" class="colwidths-given docutils"> <colgroup> <col width="20%" /> <col width="80%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>b<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set the frame payload. If a masking key is present, the value is encoded automatically.</td> </tr> <tr class="row-even"><td>c<a class="reference internal" href="#integer"><span class="std std-ref">INTEGER</span></a></td> <td>Set the op code. This can either be an integer from 0-15, or be one of the following opcode names: <strong>text</strong> (the default), <strong>continue</strong>, <strong>binary</strong>, <strong>close</strong>, <strong>ping</strong>, <strong>pong</strong>.</td> </tr> <tr class="row-odd"><td>d<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a></td> <td>Disconnect after OFFSET bytes</td> </tr> <tr class="row-even"><td>i<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a>,<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Inject the specified value at the offset</td> </tr> <tr class="row-odd"><td>p<a class="reference internal" href="#offset"><span class="std std-ref">OFFSET</span></a>,SECONDS</td> <td>Pause for SECONDS seconds after OFFSET bytes. SECONDS can be an integer or “f” to pause forever</td> </tr> <tr class="row-even"><td>x<a class="reference internal" href="#integer"><span class="std std-ref">INTEGER</span></a></td> <td>Repeat this message N times.</td> </tr> <tr class="row-odd"><td>[-]fin</td> <td>Set or un-set the <strong>fin</strong> bit.</td> </tr> <tr class="row-even"><td>k<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set the masking key. The resulting value must be exactly 4 bytes long. The special form <strong>knone</strong> specifies that no key should be set, even if the mask bit is on.</td> </tr> <tr class="row-odd"><td>l<a class="reference internal" href="#integer"><span class="std std-ref">INTEGER</span></a></td> <td>Set the payload length in the frame header, regardless of the actual body length.</td> </tr> <tr class="row-even"><td>[-]mask</td> <td>Set or un-set the <b>mask</b> bit.</td> </tr> <tr class="row-odd"><td>r<a class="reference internal" href="#value"><span class="std std-ref">VALUE</span></a></td> <td>Set the raw frame payload. This disables masking, even if the key is present.</td> </tr> <tr class="row-even"><td>[-]rsv1</td> <td>Set or un-set the <strong>rsv1</strong> bit.</td> </tr> <tr class="row-odd"><td>[-]rsv2</td> <td>Set or un-set the <strong>rsv2</strong> bit.</td> </tr> <tr class="row-even"><td>[-]rsv2</td> <td>Set or un-set the <strong>rsv2</strong> bit.</td> </tr> </tbody> </table> </div> <div class="section" id="data-types"> <h3>Data types<a class="headerlink" href="#data-types" title="Permalink to this headline">露</a></h3> <div class="section" id="integer"> <span id="id1"></span><h4>INTEGER<a class="headerlink" href="#integer" title="Permalink to this headline">露</a></h4> </div> <div class="section" id="offset"> <span id="id2"></span><h4>OFFSET<a class="headerlink" href="#offset" title="Permalink to this headline">露</a></h4> <p>Offsets are calculated relative to the base message, before any injections or other transforms are applied. They have 3 flavors:</p> <table border="1" class="docutils"> <colgroup> <col width="21%" /> <col width="79%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>integer</td> <td>An integer byte offset</td> </tr> <tr class="row-even"><td><strong>r</strong></td> <td>A random location</td> </tr> <tr class="row-odd"><td><strong>a</strong></td> <td>The end of the message</td> </tr> </tbody> </table> </div> <div class="section" id="value"> <span id="id3"></span><h4>VALUE<a class="headerlink" href="#value" title="Permalink to this headline">露</a></h4> <div class="section" id="literals"> <h5>Literals<a class="headerlink" href="#literals" title="Permalink to this headline">露</a></h5> <p>Literal values are specified as a quoted strings:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="s2">"foo"</span> </pre></div> </div> <p>Either single or double quotes are accepted, and quotes can be escaped with backslashes within the string:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="s1">'fo</span><span class="se">\'</span><span class="s1">o'</span> </pre></div> </div> <p>Literal values can contain Python-style backslash escape sequences:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="s1">'foo</span><span class="se">\r\n</span><span class="s1">bar'</span> </pre></div> </div> </div> <div class="section" id="generated"> <h5>Generated<a class="headerlink" href="#generated" title="Permalink to this headline">露</a></h5> <p>An @-symbol lead-in specifies that generated data should be used. There are two components to a generator specification - a size, and a data type. By default pathod assumes a data type of “bytes”.</p> <p>Here’s a value specifier for generating 100 bytes:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="nd">@100</span> </pre></div> </div> <p>You can use standard suffixes to indicate larger values. Here, for instance, is a specifier for generating 100 megabytes:</p> <blockquote> <div>@100m</div></blockquote> <p>Data is generated and served efficiently - if you really want to send a terabyte of data to a client, pathod can do it. The supported suffixes are:</p> <table border="1" class="docutils"> <colgroup> <col width="33%" /> <col width="67%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>b</td> <td>1024**0 (bytes)</td> </tr> <tr class="row-even"><td>k</td> <td>1024**1 (kilobytes)</td> </tr> <tr class="row-odd"><td>m</td> <td>1024**2 (megabytes)</td> </tr> <tr class="row-even"><td>g</td> <td>1024**3 (gigabytes)</td> </tr> <tr class="row-odd"><td>t</td> <td>1024**4 (terabytes)</td> </tr> </tbody> </table> <p>Data types are separated from the size specification by a comma. This specification generates 100mb of ASCII:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="nd">@100m</span><span class="p">,</span><span class="n">ascii</span> </pre></div> </div> <p>Supported data types are:</p> <table border="1" class="docutils"> <colgroup> <col width="27%" /> <col width="73%" /> </colgroup> <tbody valign="top"> <tr class="row-odd"><td>ascii</td> <td>All ASCII characters</td> </tr> <tr class="row-even"><td>ascii_letters</td> <td>A-Za-z</td> </tr> <tr class="row-odd"><td>ascii_lowercase</td> <td>a-z</td> </tr> <tr class="row-even"><td>ascii_uppercase</td> <td>A-Z</td> </tr> <tr class="row-odd"><td>bytes</td> <td>All 256 byte values</td> </tr> <tr class="row-even"><td>digits</td> <td>0-9</td> </tr> <tr class="row-odd"><td>hexdigits</td> <td>0-f</td> </tr> <tr class="row-even"><td>octdigits</td> <td>0-7</td> </tr> <tr class="row-odd"><td>punctuation</td> <td>!”#$%&’()*+,-./:;<=>?@[\]^_`{|}~ and space</td> </tr> <tr class="row-even"><td>whitespace</td> <td>\t \n \x0b \x0c \r and space</td> </tr> </tbody> </table> </div> <div class="section" id="files"> <h5>Files<a class="headerlink" href="#files" title="Permalink to this headline">露</a></h5> <p>You can load a value from a specified file path. To do so, you have to specify a _staticdir_ option to pathod on the command-line, like so:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pathod</span> <span class="o">-</span><span class="n">d</span> <span class="o">~/</span><span class="n">myassets</span> </pre></div> </div> <p>All paths are relative paths under this directory. File loads are indicated by starting the value specifier with the left angle bracket:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o"><</span><span class="n">my</span><span class="o">/</span><span class="n">path</span> </pre></div> </div> <p>The path value can also be a quoted string, with the same syntax as literals:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o"><</span><span class="s2">"my/path"</span> </pre></div> </div> </div> </div> </div> </div> <span id="document-pathod/library"></span><div class="section" id="pathod-library"> <span id="library"></span><h2>pathod library<a class="headerlink" href="#pathod-library" title="Permalink to this headline">露</a></h2> <p>Behind the pathod and pathoc command-line tools lurks the <strong>pathod</strong> library, a powerful way to manipulate and serve HTTP requests and responses from code. The canonical documentation for the library is in the code, and can be accessed using pydoc.</p> <div class="literal-block-wrapper docutils container" id="id1"> <div class="code-block-caption"><span class="caption-text">examples/pathod/libpathod_pathoc.py</span><a class="headerlink" href="#id1" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="ch">#!/usr/bin/env python</span> <span class="kn">from</span> <span class="nn">pathod</span> <span class="kn">import</span> <span class="n">pathoc</span> <span class="n">p</span> <span class="o">=</span> <span class="n">pathoc</span><span class="o">.</span><span class="n">Pathoc</span><span class="p">((</span><span class="s2">"google.com"</span><span class="p">,</span> <span class="mi">80</span><span class="p">))</span> <span class="n">p</span><span class="o">.</span><span class="n">connect</span><span class="p">()</span> <span class="k">print</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="s2">"get:/"</span><span class="p">))</span> <span class="k">print</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="s2">"get:/foo"</span><span class="p">))</span> </pre></div> </div> </div> </div> <span id="document-pathod/test"></span><div class="section" id="pathod-test"> <span id="test"></span><h2>pathod.test<a class="headerlink" href="#pathod-test" title="Permalink to this headline">露</a></h2> <p>The <strong>pathod.test</strong> module is a light, flexible testing layer for HTTP clients. It works by firing up a Pathod instance in a separate thread, letting you use Pathod’s full abilities to generate responses, and then query Pathod’s internal logs to establish what happened. All the mechanics of startup, shutdown, finding free ports and so forth are taken care of for you.</p> <p>The canonical docs can be accessed using pydoc:</p> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">pydoc</span> <span class="n">pathod</span><span class="o">.</span><span class="n">test</span> </pre></div> </div> <p>The remainder of this page demonstrates some common interaction patterns using <a class="reference external" href="https://nose.readthedocs.org/en/latest/">Nose</a>. These examples are also applicable with only minor modification to most commonly used Python testing engines.</p> <div class="section" id="context-manager"> <h3>Context Manager<a class="headerlink" href="#context-manager" title="Permalink to this headline">露</a></h3> <div class="literal-block-wrapper docutils container" id="id1"> <div class="code-block-caption"><span class="caption-text">examples/pathod/test_context.py</span><a class="headerlink" href="#id1" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">requests</span> <span class="kn">from</span> <span class="nn">pathod</span> <span class="kn">import</span> <span class="n">test</span> <span class="k">def</span> <span class="nf">test_simple</span><span class="p">():</span> <span class="sd">"""</span> <span class="sd"> Testing the requests module with</span> <span class="sd"> a pathod context manager.</span> <span class="sd"> """</span> <span class="c1"># Start pathod in a separate thread</span> <span class="k">with</span> <span class="n">test</span><span class="o">.</span><span class="n">Daemon</span><span class="p">()</span> <span class="k">as</span> <span class="n">d</span><span class="p">:</span> <span class="c1"># Get a URL for a pathod spec</span> <span class="n">url</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">p</span><span class="p">(</span><span class="s2">"200:b@100"</span><span class="p">)</span> <span class="c1"># ... and request it</span> <span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="c1"># Check the returned data</span> <span class="k">assert</span> <span class="n">r</span><span class="o">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">r</span><span class="o">.</span><span class="n">content</span><span class="p">)</span> <span class="o">==</span> <span class="mi">100</span> <span class="c1"># Check pathod's internal log</span> <span class="n">log</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">last_log</span><span class="p">()[</span><span class="s2">"request"</span><span class="p">]</span> <span class="k">assert</span> <span class="n">log</span><span class="p">[</span><span class="s2">"method"</span><span class="p">]</span> <span class="o">==</span> <span class="s2">"PUT"</span> </pre></div> </div> </div> </div> <div class="section" id="one-instance-per-test"> <h3>One instance per test<a class="headerlink" href="#one-instance-per-test" title="Permalink to this headline">露</a></h3> <div class="literal-block-wrapper docutils container" id="id2"> <div class="code-block-caption"><span class="caption-text">examples/pathod/test_setup.py</span><a class="headerlink" href="#id2" title="Permalink to this code">露</a></div> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">requests</span> <span class="kn">from</span> <span class="nn">pathod</span> <span class="kn">import</span> <span class="n">test</span> <span class="k">class</span> <span class="nc">Test</span><span class="p">:</span> <span class="sd">"""</span> <span class="sd"> Testing the requests module with</span> <span class="sd"> a pathod instance started for</span> <span class="sd"> each test.</span> <span class="sd"> """</span> <span class="k">def</span> <span class="nf">setup</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">d</span> <span class="o">=</span> <span class="n">test</span><span class="o">.</span><span class="n">Daemon</span><span class="p">()</span> <span class="k">def</span> <span class="nf">teardown</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">d</span><span class="o">.</span><span class="n">shutdown</span><span class="p">()</span> <span class="k">def</span> <span class="nf">test_simple</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="c1"># Get a URL for a pathod spec</span> <span class="n">url</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">d</span><span class="o">.</span><span class="n">p</span><span class="p">(</span><span class="s2">"200:b@100"</span><span class="p">)</span> <span class="c1"># ... and request it</span> <span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="c1"># Check the returned data</span> <span class="k">assert</span> <span class="n">r</span><span class="o">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">r</span><span class="o">.</span><span class="n">content</span><span class="p">)</span> <span class="o">==</span> <span class="mi">100</span> <span class="c1"># Check pathod's internal log</span> <span class="n">log</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">d</span><span class="o">.</span><span class="n">last_log</span><span class="p">()[</span><span class="s2">"request"</span><span class="p">]</span> <span class="k">assert</span> <span class="n">log</span><span class="p">[</span><span class="s2">"method"</span><span class="p">]</span> <span class="o">==</span> <span class="s2">"PUT"</span> </pre></div> </div> </div> </div> </div> </div> <div class="toctree-wrapper compound"> <span id="document-dev/contributing"></span><div class="section" id="contributing"> <span id="id1"></span><h2>Contributing<a class="headerlink" href="#contributing" title="Permalink to this headline">露</a></h2> <p>As an open source project, <strong>mitmproxy</strong> welcomes contributions of all forms.</p> <p>Please head over to the <a class="reference external" href="https://github.com/mitmproxy/mitmproxy/blob/master/README.rst">README</a> to get started! 馃槂</p> </div> <span id="document-dev/sslkeylogfile"></span><div class="section" id="tls-master-secrets"> <span id="sslkeylogfile"></span><h2>TLS Master Secrets<a class="headerlink" href="#tls-master-secrets" title="Permalink to this headline">露</a></h2> <p>The SSL master keys can be logged by mitmproxy so that external programs can decrypt TLS connections both from and to the proxy. Key logging is enabled by setting the environment variable <span class="target" id="index-0"></span><code class="xref std std-envvar docutils literal"><span class="pre">SSLKEYLOGFILE</span></code> so that it points to a writable text file. Recent versions of WireShark can use these log files to decrypt packets. You can specify the key file path in WireShark via</p> <p><code class="samp docutils literal"><span class="pre">Edit</span> <span class="pre">-></span> <span class="pre">Preferences</span> <span class="pre">-></span> <span class="pre">Protocols</span> <span class="pre">-></span> <span class="pre">SSL</span> <span class="pre">-></span> <span class="pre">(Pre)-Master-Secret</span> <span class="pre">log</span> <span class="pre">filename</span></code>.</p> <p>Note that <span class="target" id="index-1"></span><code class="xref std std-envvar docutils literal"><span class="pre">SSLKEYLOGFILE</span></code> is respected by other programs as well, e.g. Firefox and Chrome. If this creates any issues, you can set <span class="target" id="index-2"></span><code class="xref std std-envvar docutils literal"><span class="pre">MITMPROXY_SSLKEYLOGFILE</span></code> alternatively.</p> </div> </div> </div> </div> </div> <footer> <hr/> <div role="contentinfo"> <p> © Copyright 2016, the mitmproxy project. <span class="commit"> Revision <code>14b33dca</code>. </span> </p> </div> Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. </footer> </div> </div> </section> </div> <div class="rst-versions" role="note" aria-label="versions"> <span class="rst-current-version"> <span class="fa fa-book"> mitmproxy</span> v2.0.2 </span> </div> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT:'./', VERSION:'2.0.2', COLLAPSE_INDEX:false, FILE_SUFFIX:'.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> </body> </html>