Apache Zeppelin 0.10.0 Documentation: Python 2 & 3 Interpreter for Apache Zeppelin

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Apache Zeppelin 0.10.0 Documentation: Python 2 & 3 Interpreter for Apache Zeppelin</title> <meta name="description" content="Python is a programming language that lets you work quickly and integrate systems more effectively."> <meta name="author" content="The Apache Software Foundation">  <meta name="viewport" content="width=device-width, initial-scale=1.0">   <link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet">  <link href="/docs/0.10.0/assets/themes/zeppelin/bootstrap/css/bootstrap.css" rel="stylesheet"> <link href="/docs/0.10.0/assets/themes/zeppelin/css/style.css?body=1" rel="stylesheet" type="text/css"> <link href="/docs/0.10.0/assets/themes/zeppelin/css/syntax.css" rel="stylesheet" type="text/css" media="screen" />    <script src="https://code.jquery.com/jquery-1.10.2.min.js"></script> <script src="/docs/0.10.0/assets/themes/zeppelin/bootstrap/js/bootstrap.min.js"></script> <script src="/docs/0.10.0/assets/themes/zeppelin/js/docs.js"></script> <script src="/docs/0.10.0/assets/themes/zeppelin/js/anchor.min.js"></script> <script src="/docs/0.10.0/assets/themes/zeppelin/js/toc.js"></script> <script src="/docs/0.10.0/assets/themes/zeppelin/js/lunr.min.js"></script> <script src="/docs/0.10.0/assets/themes/zeppelin/js/search.js"></script>  <link href="/docs/0.10.0/atom.xml" type="application/atom+xml" rel="alternate" title="Sitewide ATOM Feed"> <link href="/docs/0.10.0/rss.xml" type="application/rss+xml" rel="alternate" title="Sitewide RSS Feed"> </head> <body> <div id="menu" class="navbar navbar-inverse navbar-fixed-top" role="navigation"> <div class="container navbar-container"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <div class="navbar-brand"> <a class="navbar-brand-main" href="http://zeppelin.apache.org"> <img src="/docs/0.10.0/assets/themes/zeppelin/img/zeppelin_logo.png" width="50" style="margin-top: -2px;" alt="I'm zeppelin"> <span style="margin-left: 5px; font-size: 27px;">Zeppelin</span> <a class="navbar-brand-version" href="/docs/0.10.0" style="font-size: 15px; color: white;"> 0.10.0 </a> </a> </div> </div> <nav class="navbar-collapse collapse" role="navigation"> <ul class="nav navbar-nav"> <li> <a href="#" data-toggle="dropdown" class="dropdown-toggle">Quick Start <b class="caret"></b></a> <ul class="dropdown-menu"> <li class="title"><span>Getting Started</span></li> <li><a href="/docs/0.10.0/quickstart/install.html">Install</a></li> <li><a href="/docs/0.10.0/quickstart/explore_ui.html">Explore UI</a></li> <li><a href="/docs/0.10.0/quickstart/tutorial.html">Tutorial</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Run Mode</span></li> <li><a href="/docs/0.10.0/quickstart/kubernetes.html">Kubernetes</a></li> <li><a href="/docs/0.10.0/quickstart/docker.html">Docker</a></li> <li><a href="/docs/0.10.0/quickstart/yarn.html">Yarn</a></li> <li role="separator" class="divider"></li> <li><a href="/docs/0.10.0/quickstart/spark_with_zeppelin.html">Spark with Zeppelin</a></li> <li><a href="/docs/0.10.0/quickstart/flink_with_zeppelin.html">Flink with Zeppelin</a></li> <li><a href="/docs/0.10.0/quickstart/sql_with_zeppelin.html">SQL with Zeppelin</a></li> <li><a href="/docs/0.10.0/quickstart/python_with_zeppelin.html">Python with Zeppelin</a></li> <li><a href="/docs/0.10.0/quickstart/r_with_zeppelin.html">R with Zeppelin</a></li> </ul> </li> <li> <a href="#" data-toggle="dropdown" class="dropdown-toggle">Usage<b class="caret"></b></a> <ul class="dropdown-menu scrollable-menu"> <li class="title"><span>Dynamic Form</span></li> <li><a href="/docs/0.10.0/usage/dynamic_form/intro.html">What is Dynamic Form?</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Display System</span></li> <li><a href="/docs/0.10.0/usage/display_system/basic.html#text">Text Display</a></li> <li><a href="/docs/0.10.0/usage/display_system/basic.html#html">HTML Display</a></li> <li><a href="/docs/0.10.0/usage/display_system/basic.html#table">Table Display</a></li> <li><a href="/docs/0.10.0/usage/display_system/basic.html#network">Network Display</a></li> <li><a href="/docs/0.10.0/usage/display_system/angular_backend.html">Angular Display using Backend API</a></li> <li><a href="/docs/0.10.0/usage/display_system/angular_frontend.html">Angular Display using Frontend API</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Interpreter</span></li> <li><a href="/docs/0.10.0/usage/interpreter/overview.html">Overview</a></li> <li><a href="/docs/0.10.0/usage/interpreter/interpreter_binding_mode.html">Interpreter Binding Mode</a></li> <li><a href="/docs/0.10.0/usage/interpreter/user_impersonation.html">User Impersonation</a></li> <li><a href="/docs/0.10.0/usage/interpreter/dependency_management.html">Dependency Management</a></li> <li><a href="/docs/0.10.0/usage/interpreter/installation.html">Installing Interpreters</a></li>  <li><a href="/docs/0.10.0/usage/interpreter/execution_hooks.html">Execution Hooks (Experimental)</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Other Features</span></li> <li><a href="/docs/0.10.0/usage/other_features/publishing_paragraphs.html">Publishing Paragraphs</a></li> <li><a href="/docs/0.10.0/usage/other_features/personalized_mode.html">Personalized Mode</a></li> <li><a href="/docs/0.10.0/usage/other_features/customizing_homepage.html">Customizing Zeppelin Homepage</a></li> <li><a href="/docs/0.10.0/usage/other_features/notebook_actions.html">Notebook Actions</a></li> <li><a href="/docs/0.10.0/usage/other_features/cron_scheduler.html">Cron Scheduler</a></li> <li><a href="/docs/0.10.0/usage/other_features/zeppelin_context.html">Zeppelin Context</a></li> <li role="separator" class="divider"></li> <li class="title"><span>REST API</span></li> <li><a href="/docs/0.10.0/usage/rest_api/interpreter.html">Interpreter API</a></li> <li><a href="/docs/0.10.0/usage/rest_api/zeppelin_server.html">Zeppelin Server API</a></li> <li><a href="/docs/0.10.0/usage/rest_api/notebook.html">Notebook API</a></li> <li><a href="/docs/0.10.0/usage/rest_api/notebook_repository.html">Notebook Repository API</a></li> <li><a href="/docs/0.10.0/usage/rest_api/configuration.html">Configuration API</a></li> <li><a href="/docs/0.10.0/usage/rest_api/credential.html">Credential API</a></li> <li><a href="/docs/0.10.0/usage/rest_api/helium.html">Helium API</a></li> <li class="title"><span>Zeppelin SDK</span></li> <li><a href="/docs/0.10.0/usage/zeppelin_sdk/client_api.html">Client API</a></li> <li><a href="/docs/0.10.0/usage/zeppelin_sdk/session_api.html">Session API</a></li> </ul> </li> <li> <a href="#" data-toggle="dropdown" class="dropdown-toggle">Setup<b class="caret"></b></a> <ul class="dropdown-menu scrollable-menu"> <li class="title"><span>Basics</span></li> <li><a href="/docs/0.10.0/setup/basics/how_to_build.html">How to Build Zeppelin</a></li> <li><a href="/docs/0.10.0/setup/basics/hadoop_integration.html">Hadoop Integration</a></li> <li><a href="/docs/0.10.0/setup/basics/multi_user_support.html">Multi-user Support</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Deployment</span></li>  <li><a href="/docs/0.10.0/setup/deployment/spark_cluster_mode.html#spark-standalone-mode">Spark Cluster Mode: Standalone</a></li> <li><a href="/docs/0.10.0/setup/deployment/spark_cluster_mode.html#spark-on-yarn-mode">Spark Cluster Mode: YARN</a></li> <li><a href="/docs/0.10.0/setup/deployment/spark_cluster_mode.html#spark-on-mesos-mode">Spark Cluster Mode: Mesos</a></li> <li><a href="/docs/0.10.0/setup/deployment/flink_and_spark_cluster.html">Zeppelin with Flink, Spark Cluster</a></li> <li><a href="/docs/0.10.0/setup/deployment/cdh.html">Zeppelin on CDH</a></li> <li><a href="/docs/0.10.0/setup/deployment/virtual_machine.html">Zeppelin on VM: Vagrant</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Security</span></li> <li><a href="/docs/0.10.0/setup/security/authentication_nginx.html">HTTP Basic Auth using NGINX</a></li> <li><a href="/docs/0.10.0/setup/security/shiro_authentication.html">Shiro Authentication</a></li> <li><a href="/docs/0.10.0/setup/security/notebook_authorization.html">Notebook Authorization</a></li> <li><a href="/docs/0.10.0/setup/security/datasource_authorization.html">Data Source Authorization</a></li> <li><a href="/docs/0.10.0/setup/security/http_security_headers.html">HTTP Security Headers</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Notebook Storage</span></li> <li><a href="/docs/0.10.0/setup/storage/storage.html#notebook-storage-in-local-git-repository">Git Storage</a></li> <li><a href="/docs/0.10.0/setup/storage/storage.html#notebook-storage-in-s3">S3 Storage</a></li> <li><a href="/docs/0.10.0/setup/storage/storage.html#notebook-storage-in-azure">Azure Storage</a></li> <li><a href="/docs/0.10.0/setup/storage/storage.html#notebook-storage-in-oss">OSS Storage</a></li> <li><a href="/docs/0.10.0/setup/storage/storage.html#notebook-storage-in-zeppelinhub">ZeppelinHub Storage</a></li> <li><a href="/docs/0.10.0/setup/storage/storage.html#notebook-storage-in-mongodb">MongoDB Storage</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Operation</span></li> <li><a href="/docs/0.10.0/setup/operation/configuration.html">Configuration</a></li> <li><a href="/docs/0.10.0/setup/operation/proxy_setting.html">Proxy Setting</a></li> <li><a href="/docs/0.10.0/setup/operation/upgrading.html">Upgrading</a></li> <li><a href="/docs/0.10.0/setup/operation/trouble_shooting.html">Trouble Shooting</a></li> </ul> </li> <li> <a href="#" data-toggle="dropdown" class="dropdown-toggle">Interpreter <b class="caret"></b></a> <ul class="dropdown-menu scrollable-menu"> <li class="title"><span>Interpreters</span></li> <li><a href="/docs/0.10.0/usage/interpreter/overview.html">Overview</a></li> <li role="separator" class="divider"></li> <li><a href="/docs/0.10.0/interpreter/spark.html">Spark</a></li> <li><a href="/docs/0.10.0/interpreter/flink.html">Flink</a></li> <li><a href="/docs/0.10.0/interpreter/jdbc.html">JDBC</a></li> <li><a href="/docs/0.10.0/interpreter/python.html">Python</a></li> <li><a href="/docs/0.10.0/interpreter/r.html">R</a></li> <li role="separator" class="divider"></li> <li><a href="/docs/0.10.0/interpreter/alluxio.html">Alluxio</a></li> <li><a href="/docs/0.10.0/interpreter/beam.html">Beam</a></li> <li><a href="/docs/0.10.0/interpreter/bigquery.html">BigQuery</a></li> <li><a href="/docs/0.10.0/interpreter/cassandra.html">Cassandra</a></li> <li><a href="/docs/0.10.0/interpreter/elasticsearch.html">Elasticsearch</a></li> <li><a href="/docs/0.10.0/interpreter/geode.html">Geode</a></li> <li><a href="/docs/0.10.0/interpreter/groovy.html">Groovy</a></li> <li><a href="/docs/0.10.0/interpreter/hazelcastjet.html">Hazelcast Jet</a></li> <li><a href="/docs/0.10.0/interpreter/hbase.html">HBase</a></li> <li><a href="/docs/0.10.0/interpreter/hdfs.html">HDFS</a></li> <li><a href="/docs/0.10.0/interpreter/hive.html">Hive</a></li> <li><a href="/docs/0.10.0/interpreter/ignite.html">Ignite</a></li> <li><a href="/docs/0.10.0/interpreter/influxdb.html">influxDB</a></li> <li><a href="/docs/0.10.0/interpreter/java.html">Java</a></li> <li><a href="/docs/0.10.0/interpreter/jupyter.html">Jupyter</a></li> <li><a href="/docs/0.10.0/interpreter/kotlin.html">Kotlin</a></li> <li><a href="/docs/0.10.0/interpreter/ksql.html">KSQL</a></li> <li><a href="/docs/0.10.0/interpreter/kylin.html">Kylin</a></li> <li><a href="/docs/0.10.0/interpreter/lens.html">Lens</a></li> <li><a href="/docs/0.10.0/interpreter/livy.html">Livy</a></li> <li><a href="/docs/0.10.0/interpreter/mahout.html">Mahout</a></li> <li><a href="/docs/0.10.0/interpreter/markdown.html">Markdown</a></li> <li><a href="/docs/0.10.0/interpreter/mongodb.html">MongoDB</a></li> <li><a href="/docs/0.10.0/interpreter/neo4j.html">Neo4j</a></li> <li><a href="/docs/0.10.0/interpreter/pig.html">Pig</a></li> <li><a href="/docs/0.10.0/interpreter/postgresql.html">Postgresql, HAWQ</a></li> <li><a href="/docs/0.10.0/interpreter/sap.html">SAP</a></li> <li><a href="/docs/0.10.0/interpreter/scalding.html">Scalding</a></li> <li><a href="/docs/0.10.0/interpreter/scio.html">Scio</a></li> <li><a href="/docs/0.10.0/interpreter/shell.html">Shell</a></li> <li><a href="/docs/0.10.0/interpreter/sparql.html">Sparql</a></li> <li><a href="/docs/0.10.0/interpreter/submarine.html">Submarine</a></li> </ul> </li> <li> <a href="#" data-toggle="dropdown" class="dropdown-toggle">More<b class="caret"></b></a> <ul class="dropdown-menu scrollable-menu" style="right: 0; left: auto;"> <li class="title"><span>Extending Zeppelin</span></li> <li><a href="/docs/0.10.0/development/writing_zeppelin_interpreter.html">Writing Zeppelin Interpreter</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Helium (Experimental)</span></li> <li><a href="/docs/0.10.0/development/helium/overview.html">Overview</a></li> <li><a href="/docs/0.10.0/development/helium/writing_application.html">Writing Helium Application</a></li> <li><a href="/docs/0.10.0/development/helium/writing_spell.html">Writing Helium Spell</a></li> <li><a href="/docs/0.10.0/development/helium/writing_visualization_basic.html">Writing Helium Visualization: Basics</a></li> <li><a href="/docs/0.10.0/development/helium/writing_visualization_transformation.html">Writing Helium Visualization: Transformation</a></li> <li role="separator" class="divider"></li> <li class="title"><span>Contributing to Zeppelin</span></li> <li><a href="/docs/0.10.0/setup/basics/how_to_build.html">How to Build Zeppelin</a></li> <li><a href="/docs/0.10.0/development/contribution/useful_developer_tools.html">Useful Developer Tools</a></li> <li><a href="/docs/0.10.0/development/contribution/how_to_contribute_code.html">How to Contribute (code)</a></li> <li><a href="/docs/0.10.0/development/contribution/how_to_contribute_website.html">How to Contribute (website)</a></li> <li role="separator" class="divider"></li> <li class="title"><span>External Resources</span></li> <li><a target="_blank" href="https://zeppelin.apache.org/community.html">Mailing List</a></li> <li><a target="_blank" href="https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Home">Apache Zeppelin Wiki</a></li> <li><a target="_blank" href="http://stackoverflow.com/questions/tagged/apache-zeppelin">Stackoverflow Questions about Zeppelin</a></li> </ul> </li> <li> <a href="/docs/0.10.0/search.html" class="nav-search-link"> <span class="fa fa-search nav-search-icon"></span> </a> </li> </ul> </nav> </div> </div> <div class="content">  <div class="row"> <div class="col-md-12">  <h1>Python 2 & 3 Interpreter for Apache Zeppelin</h1> <div id="toc"></div> <h2>Overview</h2> <p>Zeppelin supports python language which is very popular in data analytics and machine learning.</p> <table class="table-configuration"> <tr> <th>Name</th> <th>Class</th> <th>Description</th> </tr> <tr> <td>%python</td> <td>PythonInterpreter</td> <td>Vanilla python interpreter, with least dependencies, only python environment installed is required, <code>%python</code> will use IPython if its prerequisites are met</td> </tr> <tr> <td>%python.ipython</td> <td>IPythonInterpreter</td> <td>Provide more fancy python runtime via IPython, almost the same experience like Jupyter. It requires more things, but is the recommended interpreter for using python in Zeppelin, see below for more details</td> </tr> <tr> <td>%python.sql</td> <td>PythonInterpreterPandasSql</td> <td>Provide sql capability to query data in Pandas DataFrame via <code>pandasql</code>, it can access dataframes in <code>%python</code></td> </tr> </table> <h2>Main Features</h2> <table class="table-configuration"> <tr> <th>Feature</th> <th>Description</th> </tr> <tr> <td>Support vanilla Python and IPython</td> <td>Vanilla Python only requires python install, IPython provides almost the same user experience like Jupyter, like inline plotting, code completion, magic methods and etc.</td> </tr> <tr> <td>Built-in ZeppelinContext Support</td> <td>You can use ZeppelinContext to visualize pandas dataframe</td> </tr> <tr> <td>Support SQL on Pandas dataframe</td> <td>You can use Sql to query dataframe which is defined in Python</td> </tr> <tr> <td>Run Python in yarn cluster with customized Python runtime</td> <td>You can run Python in yarn cluster with customized Python runtime without affecting each other</td> </tr> </table> <h2>Play Python in Zeppelin docker</h2> <p>For beginner, we would suggest you to play Python in Zeppelin docker first. In the Zeppelin docker image, we have already installed miniconda and lots of <a href="https://github.com/apache/zeppelin/blob/branch-0.10/scripts/docker/zeppelin/bin/env_python_3_with_R.yml">useful python libraries</a> including IPython's prerequisites, so <code>%python</code> would use IPython.</p> <p>Without any extra configuration, you can run most of tutorial notes under folder <code>Python Tutorial</code> directly.</p> <div class="highlight"><pre><code class="bash language-bash" data-lang="bash">docker run -u <span class="k">$(</span>id -u<span class="k">)</span> -p 8080:8080 --rm --name zeppelin apache/zeppelin:0.10.0 </code></pre></div> <p>After running the above command, you can open <code>http://localhost:8080</code> to play Python in Zeppelin. </p> <h2>Configuration</h2> <table class="table-configuration"> <tr> <th>Property</th> <th>Default</th> <th>Description</th> </tr> <tr> <td>zeppelin.python</td> <td>python</td> <td>Path of the installed Python binary (could be python2 or python3). You should set this property explicitly if python is not in your <code>$PATH</code>(example: /usr/bin/python). </td> </tr> <tr> <td>zeppelin.python.maxResult</td> <td>1000</td> <td>Max number of dataframe rows to display.</td> </tr> <tr> <td>zeppelin.python.useIPython</td> <td>true</td> <td>When this property is true, <code>%python</code> would be delegated to <code>%python.ipython</code> if IPython is available, otherwise IPython is only used in <code>%python.ipython</code>. </td> </tr> <tr> <td>zeppelin.yarn.dist.archives</td> <td></td> <td>Used for ipython in yarn mode. It is a general zeppelin interpreter configuration, not python specific. For Python interpreter it is used to specify the conda env archive file which could be on local filesystem or on hadoop compatible file system.</td> </tr> <tr> <td>zeppelin.interpreter.conda.env.name</td> <td></td> <td>Used for ipython in yarn mode. conda environment name, aka the folder name in the working directory of interpreter yarn container.</td> </tr> </table> <h2>Vanilla Python Interpreter (<code>%python</code>)</h2> <p>The vanilla python interpreter provides basic python interpreter feature, only python installed is required.</p> <h3>Matplotlib integration</h3> <p>The vanilla python interpreter can display matplotlib figures inline automatically using the <code>matplotlib</code>:</p> <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span> <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span> <span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span> </code></pre></div> <p>The output of this command will by default be converted to HTML by implicitly making use of the <code>%html</code> magic. Additional configuration can be achieved using the builtin <code>z.configure_mpl()</code> method. For example, </p> <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="n">z</span><span class="o">.</span><span class="n">configure_mpl</span><span class="p">(</span><span class="n">width</span><span class="o">=</span><span class="mi">400</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">,</span> <span class="n">fmt</span><span class="o">=</span><span class="s">'svg'</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span> </code></pre></div> <p>Will produce a 400x300 image in SVG format, which by default are normally 600x400 and PNG respectively. In the future, another option called <code>angular</code> can be used to make it possible to update a plot produced from one paragraph directly from another (the output will be <code>%angular</code> instead of <code>%html</code>). However, this feature is already available in the <code>pyspark</code> interpreter. More details can be found in the included "Zeppelin Tutorial: Python - matplotlib basic" tutorial notebook. </p> <p>If Zeppelin cannot find the matplotlib backend files (which should usually be found in <code>$ZEPPELIN_HOME/interpreter/lib/python</code>) in your <code>PYTHONPATH</code>, then the backend will automatically be set to agg, and the (otherwise deprecated) instructions below can be used for more limited inline plotting.</p> <p>If you are unable to load the inline backend, use <code>z.show(plt)</code>:</p> <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span> <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span> <span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">()</span> <span class="p">(</span><span class="o">..</span> <span class="o">..</span><span class="p">)</span> <span class="n">z</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">plt</span><span class="p">)</span> <span class="n">plt</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> </code></pre></div> <p>The <code>z.show()</code> function can take optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).</p> <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span> <span class="n">z</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">plt</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="s">'50px'</span><span class="p">)</span> <span class="n">z</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">plt</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="s">'150px'</span><span class="p">,</span> <span class="n">fmt</span><span class="o">=</span><span class="s">'svg'</span><span class="p">)</span> </code></pre></div> <p><img class="img-responsive" src="/docs/0.10.0/assets/themes/zeppelin/img/docs-img/pythonMatplotlib.png" /></p> <h2>IPython Interpreter (<code>%python.ipython</code>) (recommended)</h2> <p>IPython is more powerful than the vanilla python interpreter with extra functionality. This is what we recommend you to use instead of vanilla python interpreter. You can use IPython with Python2 or Python3 which depends on which python you set in <code>zeppelin.python</code>.</p> <h3>Prerequisites</h3> <ul> <li>For non-anaconda environment, You need to install the following packages</li> </ul> <div class="highlight"><pre><code class="text language-text" data-lang="text">pip install jupyter pip install grpcio pip install protobuf </code></pre></div> <ul> <li>For anaconda environment (<code>zeppelin.python</code> points to the python under anaconda)</li> </ul> <div class="highlight"><pre><code class="text language-text" data-lang="text">pip install grpcio pip install protobuf </code></pre></div> <p>Zeppelin will check the above prerequisites when using <code>%python</code>, if IPython prerequisites are met, <code>%python</code> would use IPython interpreter, otherwise it would use vanilla Python interpreter in <code>%python</code>.</p> <p>In addition to all the basic functions of the vanilla python interpreter, you can use all the IPython advanced features as you use it in Jupyter Notebook. Take a look at tutorial note <code>Python Tutorial/1. IPython Basic</code> and <code>Python Tutorial/2. IPython Visualization Tutorial</code> for how to use IPython in Zeppelin.</p> <h3>Use IPython magic</h3> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.ipython #python help range? #timeit %timeit range(100) </code></pre></div> <h3>Use matplotlib</h3> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.ipython %matplotlib inline import matplotlib.pyplot as plt print("hello world") data=[1,2,3,4] plt.figure() plt.plot(data) </code></pre></div> <h3>Run shell command</h3> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.ipython !pip install pandas </code></pre></div> <h3>Colored text output</h3> <p><img class="img-responsive" src="/docs/0.10.0/assets/themes/zeppelin/img/docs-img/ipython_error.png" width="80%"/></p> <h3>More types of visualization</h3> <p>e.g. You can use hvplot in the same way as in Jupyter, Take a look at tutorial note <code>Python Tutorial/2. IPython Visualization Tutorial</code> for more visualization examples.</p> <p><img class="img-responsive" src="/docs/0.10.0/assets/themes/zeppelin/img/docs-img/ipython_hvplot.png" /></p> <h3>Better code completion</h3> <p>Type <code>tab</code> can give you all the completion candidates just like in Jupyter.</p> <p><img class="img-responsive" src="/docs/0.10.0/assets/themes/zeppelin/img/docs-img/ipython_code_completion.png" /></p> <h2>Pandas Integration</h2> <p>Apache Zeppelin <a href="../usage/display_system/basic.html#table">Table Display System</a> provides built-in data visualization capabilities. Python interpreter leverages it to visualize Pandas DataFrames via <code>z.show()</code> API.</p> <p>For example:</p> <p><img class="img-responsive" src="/docs/0.10.0/assets/themes/zeppelin/img/docs-img/python_zshow_df.png" /></p> <p>By default, <code>z.show</code> only display 1000 rows, you can configure <code>zeppelin.python.maxResult</code> to adjust the max number of rows.</p> <h2>SQL over Pandas DataFrames</h2> <p>There is a convenience <code>%python.sql</code> interpreter that matches Apache Spark experience in Zeppelin and enables usage of SQL language to query <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html">Pandas DataFrames</a> and visualization of results through built-in <a href="../usage/display_system/basic.html#table">Table Display System</a>. <code>%python.sql</code> can access dataframes defined in <code>%python</code>.</p> <p><strong>Prerequisites</strong></p> <ul> <li>Pandas <code>pip install pandas</code></li> <li>PandaSQL <code>pip install -U pandasql</code></li> </ul> <p>Here's one example:</p> <ul> <li><p>first paragraph</p> <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="kn">as</span> <span class="nn">pd</span> <span class="n">rates</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s">"bank.csv"</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s">";"</span><span class="p">)</span> </code></pre></div></li> <li><p>next paragraph</p> <div class="highlight"><pre><code class="sql language-sql" data-lang="sql"><span class="o">%</span><span class="n">python</span><span class="p">.</span><span class="k">sql</span> <span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">rates</span> <span class="k">WHERE</span> <span class="n">age</span> <span class="o"><</span> <span class="mi">40</span> </code></pre></div></li> </ul> <p><img class="img-responsive" src="/docs/0.10.0/assets/themes/zeppelin/img/docs-img/python_pandas_sql.png" /></p> <h2>Using Zeppelin Dynamic Forms</h2> <p>You can leverage <a href="/docs/0.10.0/usage/dynamic_form/intro.html">Zeppelin Dynamic Form</a> inside your Python code.</p> <p>Example : </p> <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span> <span class="c">### Input form</span> <span class="k">print</span><span class="p">(</span><span class="n">z</span><span class="o">.</span><span class="n">input</span><span class="p">(</span><span class="s">"f1"</span><span class="p">,</span><span class="s">"defaultValue"</span><span class="p">))</span> <span class="c">### Select form</span> <span class="k">print</span><span class="p">(</span><span class="n">z</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s">"f2"</span><span class="p">,[(</span><span class="s">"o1"</span><span class="p">,</span><span class="s">"1"</span><span class="p">),(</span><span class="s">"o2"</span><span class="p">,</span><span class="s">"2"</span><span class="p">)],</span><span class="s">"o1"</span><span class="p">))</span> <span class="c">### Checkbox form</span> <span class="k">print</span><span class="p">(</span><span class="s">""</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">z</span><span class="o">.</span><span class="n">checkbox</span><span class="p">(</span><span class="s">"f3"</span><span class="p">,</span> <span class="p">[(</span><span class="s">"o1"</span><span class="p">,</span><span class="s">"1"</span><span class="p">),</span> <span class="p">(</span><span class="s">"o2"</span><span class="p">,</span><span class="s">"2"</span><span class="p">)],[</span><span class="s">"o1"</span><span class="p">])))</span> </code></pre></div> <h2>ZeppelinContext API</h2> <p>Python interpreter create a variable <code>z</code> which represent <code>ZeppelinContext</code> for you. User can use it to do more fancy and complex things in Zeppelin.</p> <table class="table-configuration"> <tr> <th>API</th> <th>Description</th> </tr> <tr> <td>z.put(key, value)</td> <td>Put object <code>value</code> with identifier <code>key</code> to distributed resource pool of Zeppelin, so that it can be used by other interpreters</td> </tr> <tr> <td>z.get(key)</td> <td>Get object with identifier <code>key</code> from distributed resource pool of Zeppelin</td> </tr> <tr> <td>z.remove(key)</td> <td>Remove object with identifier <code>key</code> from distributed resource pool of Zeppelin</td> </tr> <tr> <td>z.getAsDataFrame(key)</td> <td>Get object with identifier <code>key</code> from distributed resource pool of Zeppelin and converted into pandas dataframe. The object in the distributed resource pool must be table type, e.g. jdbc interpreter result. </td> </tr> <tr> <td>z.angular(name, noteId = None, paragraphId = None)</td> <td>Get the angular object with identifier <code>name</code></td> </tr> <tr> <td>z.angularBind(name, value, noteId = None, paragraphId = None)</td> <td>Bind value to angular object with identifier <code>name</code></td> </tr> <tr> <td>z.angularUnbind(name, noteId = None)</td> <td>Unbind value from angular object with identifier <code>name</code></td> </tr> <tr> <td>z.show(p)</td> <td>Show python object <code>p</code> in Zeppelin, if it is pandas dataframe, it would be displayed in Zeppelin's table format, others will be converted to string</td> </tr> <tr> <td>z.textbox(name, defaultValue="")</td> <td>Create dynamic form Textbox <code>name</code> with defaultValue</td> </tr> <tr> <td>z.select(name, options, defaultValue="")</td> <td>Create dynamic form Select <code>name</code> with options and defaultValue. options should be a list of Tuple(first element is key, the second element is the displayed value) e.g. <code>z.select("f2",[("o1","1"),("o2","2")],"o1")</code></td> </tr> <tr> <td>z.checkbox(name, options, defaultChecked=[])</td> <td>Create dynamic form Checkbox `name` with options and defaultChecked. options should be a list of Tuple(first element is key, the second element is the displayed value) e.g. <code>z.checkbox("f3", [("o1","1"), ("o2","2")],["o1"])</code></td> </tr> <tr> <td>z.noteTextbox(name, defaultValue="")</td> <td>Create note level dynamic form Textbox</td> </tr> <tr> <td>z.noteSelect(name, options, defaultValue="")</td> <td>Create note level dynamic form Select</td> </tr> <tr> <td>z.noteCheckbox(name, options, defaultChecked=[])</td> <td>Create note level dynamic form Checkbox</td> </tr> <tr> <td>z.run(paragraphId)</td> <td>Run paragraph</td> </tr> <tr> <td>z.run(noteId, paragraphId)</td> <td>Run paragraph</td> </tr> <tr> <td>z.runNote(noteId)</td> <td>Run the whole note</td> </tr> </table> <h2>Run Python interpreter in yarn cluster</h2> <p>Zeppelin supports to <a href="../quickstart/yarn.html">run interpreter in yarn cluster</a> which means the python interpreter can run in a yarn container. This can achieve better multi-tenant for python interpreter especially when you already have a hadoop yarn cluster.</p> <p>But there's one critical problem to run python in yarn cluster: how to manage the python environment in yarn container. Because hadoop yarn cluster is a distributed cluster environment which is composed of many nodes, and your python interpreter can start in any node. It is not practical to manage python environment in each node beforehand.</p> <p>So in order to run python in yarn cluster, we would suggest you to use conda to manage your python environment, and Zeppelin can ship your conda environment to yarn container, so that each python interpreter can have its own python environment without affecting each other.</p> <p>Python interpreter in yarn cluster only works for IPython, so make sure IPython's prerequisites are met. So make sure including the following packages in Step 1.</p> <ul> <li>python</li> <li>jupyter</li> <li>grpcio</li> <li>protobuf</li> </ul> <h3>Step 1</h3> <p>We would suggest you to use <a href="https://conda.github.io/conda-pack/">conda-pack</a> to create archive of conda environment, and ship it to yarn container. Otherwise python interpreter will use the python executable file in PATH of yarn container.</p> <p>Here's one example of yaml file which could be used to create a conda environment with python 3 and some useful python libraries.</p> <ul> <li>Create yaml file for conda environment, write the following content into file <code>python_3_env.yml</code></li> </ul> <div class="highlight"><pre><code class="text language-text" data-lang="text">name: python_3_env channels: - conda-forge - defaults dependencies: - python=3.7 - jupyter - grpcio - protobuf - pycodestyle - numpy - pandas - scipy - pandasql - panel - pyyaml - seaborn - plotnine - hvplot - intake - intake-parquet - intake-xarray - altair - vega_datasets - pyarrow </code></pre></div> <ul> <li>Create conda environment via this yml file using either <a href="https://docs.conda.io/en/latest/">conda</a> or <a href="https://github.com/mamba-org/mamba">mamba</a></li> </ul> <div class="highlight"><pre><code class="bash language-bash" data-lang="bash">conda env create -f python_3_env.yml </code></pre></div><div class="highlight"><pre><code class="bash language-bash" data-lang="bash">mamba env create -f python_3_env </code></pre></div> <ul> <li>Pack the conda environment using <code>conda</code></li> </ul> <div class="highlight"><pre><code class="bash language-bash" data-lang="bash">conda pack -n python_3_env </code></pre></div> <h3>Step 2</h3> <p>Specify the following properties to enable yarn mode for python interpreter.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conf zeppelin.interpreter.launcher yarn zeppelin.yarn.dist.archives /home/hadoop/python_3_env.tar.gz#environment zeppelin.interpreter.conda.env.name environment </code></pre></div> <p>Setting <code>zeppelin.interpreter.launcher</code> as <code>yarn</code> will launch python interpreter in yarn cluster.</p> <p><code>zeppelin.yarn.dist.archives</code> is the python conda environment tar which is created in step 1. This tar will be shipped to yarn container and untar in the working directory of yarn container. <code>environment</code> in <code>/home/hadoop/python_3.tar.gz#environment</code> is the folder name after untar.</p> <p>This folder name should be the same as <code>zeppelin.interpreter.conda.env.name</code>. Usually we name it as <code>environment</code> here.</p> <h2>Python environments (used for vanilla python interpreter in non-yarn mode)</h2> <h3>Default</h3> <p>By default, PythonInterpreter will use python command defined in <code>zeppelin.python</code> property to run python process. The interpreter can use all modules already installed (with pip, easy_install...)</p> <h3>Conda</h3> <p><a href="http://conda.pydata.org/">Conda</a> is an package management system and environment management system for python. <code>%python.conda</code> interpreter lets you change between environments.</p> <h4>Usage</h4> <ul> <li><p>get the Conda Information: </p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda info </code></pre></div></li> <li><p>list the Conda environments: </p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda env list </code></pre></div></li> <li><p>create a conda enviornment: </p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda create --name [ENV NAME] </code></pre></div></li> <li><p>activate an environment (python interpreter will be restarted): </p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda activate [ENV NAME] </code></pre></div></li> <li><p>deactivate</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda deactivate </code></pre></div></li> <li><p>get installed package list inside the current environment</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda list </code></pre></div></li> <li><p>install package</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda install [PACKAGE NAME] </code></pre></div></li> <li><p>uninstall package</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.conda uninstall [PACKAGE NAME] </code></pre></div></li> </ul> <h3>Docker</h3> <p><code>%python.docker</code> interpreter allows PythonInterpreter creates python process in a specified docker container.</p> <h4>Usage</h4> <ul> <li><p>activate an environment</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.docker activate [Repository] %python.docker activate [Repository:Tag] %python.docker activate [Image Id] </code></pre></div></li> <li><p>deactivate</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">%python.docker deactivate </code></pre></div></li> </ul> <p><br/> Here is an example</p> <div class="highlight"><pre><code class="text language-text" data-lang="text"># activate latest tensorflow image as a python environment %python.docker activate gcr.io/tensorflow/tensorflow:latest </code></pre></div> <h2>Community</h2> <p><a href="http://zeppelin.apache.org/community.html">Join our community</a> to discuss with others.</p> </div> </div> <hr> <footer>  </footer> </div> <script type="text/javascript"> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-45176241-5', 'zeppelin.apache.org'); ga('require', 'linkid', 'linkid.js'); ga('send', 'pageview'); </script> </body> </html>

CINXE.COM

Apache Zeppelin 0.10.0 Documentation: Python 2 & 3 Interpreter for Apache Zeppelin