Python implementation of regularized generalized linear models

<!DOCTYPE html>   <html class="no-js" lang="en" >  <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Python implementation of regularized generalized linear models — pyglmnet 1.1 documentation</title> <link rel="stylesheet" href="_static/css/theme.css" type="text/css" /> <link rel="stylesheet" href="_static/gallery.css" type="text/css" /> <link rel="index" title="Index" href="genindex.html"/> <link rel="search" title="Search" href="search.html"/> <link rel="top" title="pyglmnet 1.1 documentation" href="#"/> <link rel="next" title="Installation" href="install.html"/> <script src="_static/js/modernizr.min.js"></script> </head> <body class="wy-body-for-nav" role="document"> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search"> <a href="#" class="icon icon-home"> pyglmnet </a> <div class="version"> 1.1 </div> <div role="search"> <form id="rtd-search-form" class="wy-form" action="search.html" method="get"> <input type="text" name="q" placeholder="Search docs" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> </div> <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> <ul> <li class="toctree-l1"><a class="reference internal" href="install.html">Installation</a></li> <li class="toctree-l1"><a class="reference internal" href="start.html">Getting Started</a></li> <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a></li> <li class="toctree-l1"><a class="reference internal" href="cheatsheet.html">Cheatsheet</a></li> <li class="toctree-l1"><a class="reference internal" href="auto_examples/index.html">Examples Gallery</a></li> <li class="toctree-l1"><a class="reference internal" href="api.html">API Documentation</a></li> <li class="toctree-l1"><a class="reference internal" href="contributing.html">Developer Documentation</a></li> <li class="toctree-l1"><a class="reference internal" href="resources.html">Resources</a></li> <li class="toctree-l1"><a class="reference internal" href="whats_new.html">Release notes</a></li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> <i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="#">pyglmnet</a> </nav> <div class="wy-nav-content"> <div class="rst-content"> <div role="navigation" aria-label="breadcrumbs navigation"> <ul class="wy-breadcrumbs"> <li><a href="#">Docs</a> »</li> <li>Python implementation of regularized generalized linear models</li> <li class="wy-breadcrumbs-aside"> <a href="_sources/index.rst.txt" rel="nofollow"> View page source</a> </li> </ul> <hr/> </div> <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <div class="section" id="python-implementation-of-regularized-generalized-linear-models"> <h1>Python implementation of regularized generalized linear models<a class="headerlink" href="#python-implementation-of-regularized-generalized-linear-models" title="Permalink to this headline">露</a></h1> <p>Pyglmnet is a Python 3.5+ library implementing generalized linear models (GLMs) with advanced regularization options. It provides a wide range of noise models (with paired canonical link functions) including gaussian, binomial, probit, gamma, poisson, and softplus. It supports a wide range of regularizers: ridge, lasso, elastic net, group lasso, and Tikhonov regularization.</p> <p>[<a class="reference external" href="https://github.com/glm-tools/pyglmnet">Repository</a>] [<a class="reference external" href="http://glm-tools.github.io/pyglmnet">Documentation (stable release)</a>] [<a class="reference external" href="https://circleci.com/api/v1.1/project/github/glm-tools/pyglmnet/latest/artifacts/0/html/index.html?branch=master">Documentation (development version)</a>]</p> <div class="section" id="a-brief-introduction-to-glms"> <h2>A brief introduction to GLMs<a class="headerlink" href="#a-brief-introduction-to-glms" title="Permalink to this headline">露</a></h2> <p>For linear models specified as</p> <div class="math notranslate nohighlight"> \[y = \beta_0 + X\beta + \epsilon.\]</div> <p>The parameters <span class="math notranslate nohighlight">\(\beta_0, \beta\)</span> are estimated using ordinary least squares, under the implicit assumption that <span class="math notranslate nohighlight">\(y\)</span> is normally distributed.</p> <p>Generalized linear models allow us to generalize this approach to point-wise nonlinearities <span class="math notranslate nohighlight">\(q(\cdot)\)</span> and corresponding exponential family noise distributions for <span class="math notranslate nohighlight">\(\epsilon\)</span>.</p> <div class="math notranslate nohighlight"> \[y = q(\beta_0 + X\beta) + \epsilon\]</div> <p>Regularized GLMs are estimated by minimizing a loss function specified by the penalized negative log-likelihood. The elastic net penalty interpolates between the L2 and L1 norm. We solve the following optimization problem:</p> <div class="math notranslate nohighlight"> \[\min_{\beta_0, \beta} \frac{1}{N} \sum_{i = 1}^N \mathcal{L} (y_i, \beta_0 + \beta^T x_i) + \lambda [ \frac{1}{2}(1 - \alpha) \mathcal{P}_2 + \alpha \mathcal{P}_1 ]\]</div> <p>where <span class="math notranslate nohighlight">\(\mathcal{P}_2\)</span> and <span class="math notranslate nohighlight">\(\mathcal{P}_1\)</span> are the generalized L2 (Tikhonov) and generalized L1 (Group Lasso) penalties, given by:</p> <div class="math notranslate nohighlight"> \[\begin{split}\mathcal{P}_2 & = & \|\Gamma \beta \|_2^2 \\ \mathcal{P}_1 & = & \sum_g \|\beta_{j,g}\|_2\end{split}\]</div> <p>where <span class="math notranslate nohighlight">\(\Gamma\)</span> is the Tikhonov matrix: a square factorization of the inverse covariance matrix and <span class="math notranslate nohighlight">\(\beta_{j,g}\)</span> is the <span class="math notranslate nohighlight">\(j\)</span> th coefficient of group <span class="math notranslate nohighlight">\(g\)</span>.</p> </div> <div class="section" id="contents"> <h2>Contents<a class="headerlink" href="#contents" title="Permalink to this headline">露</a></h2> <div class="toctree-wrapper compound"> <ul> <li class="toctree-l1"><a class="reference internal" href="install.html">Installation</a></li> <li class="toctree-l1"><a class="reference internal" href="start.html">Getting Started</a></li> <li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a></li> <li class="toctree-l1"><a class="reference internal" href="cheatsheet.html">Cheatsheet</a></li> <li class="toctree-l1"><a class="reference internal" href="auto_examples/index.html">Examples Gallery</a></li> <li class="toctree-l1"><a class="reference internal" href="api.html">API Documentation</a></li> <li class="toctree-l1"><a class="reference internal" href="contributing.html">Developer Documentation</a></li> <li class="toctree-l1"><a class="reference internal" href="resources.html">Resources</a></li> <li class="toctree-l1"><a class="reference internal" href="whats_new.html">Release notes</a></li> </ul> </div> </div> <div class="section" id="questions-errors-bugs"> <h2>Questions / Errors / Bugs<a class="headerlink" href="#questions-errors-bugs" title="Permalink to this headline">露</a></h2> <p>If you have questions about the code or find errors or bugs, please <a class="reference external" href="https://github.com/glm-tools/pyglmnet/issues">report it here</a>. For more specific questions, feel free to email us directly.</p> </div> </div> </div> </div> <footer> <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> <a href="install.html" class="btn btn-neutral float-right" title="Installation" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a> </div> <hr/> <div role="contentinfo"> <p> © Copyright 2016-2019, Pavan Ramkumar. </p> </div> Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. </footer> </div> </div> </section> </div> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT:'./', VERSION:'1.1', COLLAPSE_INDEX:false, FILE_SUFFIX:'.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <script type="text/javascript" src="_static/language_data.js"></script> <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script> <script type="text/javascript" src="_static/js/theme.js"></script> <script type="text/javascript"> jQuery(function () { SphinxRtdTheme.StickyNav.enable(); }); </script> </body> </html>

CINXE.COM

Python implementation of regularized generalized linear models — pyglmnet 1.1 documentation