CINXE.COM

<!DOCTYPE html>  <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head><script type="text/javascript" src="/_static/js/bundle-playback.js?v=HxkREWBo" charset="utf-8"></script> <script type="text/javascript" src="/_static/js/wombat.js?v=txqj7nKC" charset="utf-8"></script> <script>window.RufflePlayer=window.RufflePlayer||{};window.RufflePlayer.config={"autoplay":"on","unmuteOverlay":"hidden"};</script> <script type="text/javascript" src="/_static/js/ruffle/ruffle.js"></script> <script type="text/javascript"> __wm.init("http://web.archive.org/web"); __wm.wombat("https://knox.apache.org/","20230914024541","http://web.archive.org/","web","/_static/", "1694659541"); </script> <link rel="stylesheet" type="text/css" href="/_static/css/banner-styles.css?v=S1zqJCYt" /> <link rel="stylesheet" type="text/css" href="/_static/css/iconochive.css?v=3PDvdIFv" />  <meta charset="UTF-8"/> <meta name="viewport" content="width=device-width, initial-scale=1.0"/> <meta name="Date-Revision-yyyymmdd" content="20230105"/> <meta http-equiv="Content-Language" content="en"/> <title>Knox Gateway – Announcing Apache Knox 2.0.0!</title> <link rel="stylesheet" href="/web/20230914024541cs_/https://knox.apache.org/css/apache-maven-fluido-1.7.min.css"/> <link rel="stylesheet" href="/web/20230914024541cs_/https://knox.apache.org/css/site.css"/> <link rel="stylesheet" href="/web/20230914024541cs_/https://knox.apache.org/css/print.css" media="print"/> <script type="text/javascript" src="/web/20230914024541js_/https://knox.apache.org/js/apache-maven-fluido-1.7.min.js"></script> <script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-27188762-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'http://web.archive.org/web/20230914024541/https://ssl' : 'http://web.archive.org/web/20230914024541/http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> </head> <body class="topBarDisabled"> <div class="container-fluid"> <div id="banner"> <div class="pull-left"><a href="./" id="bannerLeft"><img src="/web/20230914024541im_/https://knox.apache.org/images/knox-logo.gif" alt="Knox Gateway" width="341" height="156"/></a></div> <div class="pull-right"></div> <div class="clear"><hr/></div> </div> <div id="breadcrumbs"> <ul class="breadcrumb"> <li id="publishDate">Last Published: 2023-01-05</li> </ul> </div> <div class="row-fluid"> <div id="leftColumn" class="span2"> <div class="well sidebar-nav"> <ul class="nav nav-list"> <li class="nav-header">Apache Knox™</li> <li class="active"><a href="#"><span class="none"></span>Home</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/News" class="externalLink" title="News"><span class="none"></span>News</a></li> <li><a href="licenses.html" title="License"><span class="none"></span>License</a></li> <li class="nav-header">Documentation</li> <li><a href="books/knox-2-0-0/user-guide.html" title="User's Guide"><span class="none"></span>User's Guide</a></li> <li><a href="books/knox-2-0-0/knoxshell_user_guide.html" title="KnoxShell User Guide"><span class="none"></span>KnoxShell User Guide</a></li> <li><a href="books/knox-2-0-0/dev-guide.html" title="Developer's Guide"><span class="none"></span>Developer's Guide</a></li> <li><a href="books/knox-2-0-0/user-guide.html#Quick+Start" title="Quick Start"><span class="none"></span>Quick Start</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki"><span class="none"></span>Wiki</a></li> <li class="nav-header">Releases</li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Apache+Knox+Releases" class="externalLink" title="Releases"><span class="none"></span>Releases</a></li> <li class="nav-header">Processes</li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Build+Process" class="externalLink" title="Build"><span class="none"></span>Build</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Release+Process" class="externalLink" title="Release"><span class="none"></span>Release</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Contribution+Process" class="externalLink" title="Contribute"><span class="none"></span>Contribute</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Site+Maintenance" class="externalLink" title="Site Maintenance"><span class="none"></span>Site Maintenance</a></li> <li class="nav-header">Resources</li> <li><a href="http://web.archive.org/web/20230914024541/https://gitbox.apache.org/repos/asf/knox.git" class="externalLink" title="Product Source"><span class="none"></span>Product Source</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://svn.apache.org/repos/asf/knox" class="externalLink" title="Site Source"><span class="none"></span>Site Source</a></li> <li><a href="team.html" title="Project Team"><span class="none"></span>Project Team</a></li> <li><a href="mailing-lists.html" title="Mailing Lists"><span class="none"></span>Mailing Lists</a></li> <li><a href="issue-management.html" title="Issue Tracking"><span class="none"></span>Issue Tracking</a></li> <li class="nav-header">ASF</li> <li><a href="http://web.archive.org/web/20230914024541/https://www.apache.org/foundation/how-it-works.html" class="externalLink" title="How Apache Works"><span class="none"></span>How Apache Works</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://www.apache.org/foundation/" class="externalLink" title="Foundation"><span class="none"></span>Foundation</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://www.apache.org/foundation/sponsorship.html" class="externalLink" title="Sponsoring Apache"><span class="none"></span>Sponsoring Apache</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://www.apache.org/foundation/thanks.html" class="externalLink" title="Thanks"><span class="none"></span>Thanks</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://www.apache.org/security/" class="externalLink" title="Security"><span class="none"></span>Security</a></li> <li><a href="http://web.archive.org/web/20230914024541/https://www.apache.org/licenses/" class="externalLink" title="License"><span class="none"></span>License</a></li> </ul> <hr/> <div id="poweredBy"> <div class="clear"></div> <div class="clear"></div> <div class="clear"></div> <div class="clear"></div> <a href="http://web.archive.org/web/20230914024541/http://maven.apache.org/" title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="/web/20230914024541im_/https://knox.apache.org/images/logos/maven-feather.png"/></a> </div> </div> </div> <div id="bodyColumn" class="span10">  <div class="section"> <h2><a name="Announcing_Apache_Knox_2.0.0"></a>Announcing Apache Knox 2.0.0!</h2></div> <div class="section"> <h2><a name="REST_API_and_Application_Gateway_for_the_Apache_Hadoop_Ecosystem"></a>REST API and Application Gateway for the Apache Hadoop Ecosystem</h2> <p>The Apache Knox™ Gateway is an Application Gateway for interacting with the REST APIs and UIs<br/> of Apache Hadoop deployments.</p> <p>The Knox Gateway provides a single access point for all REST and HTTP interactions with Apache Hadoop<br/> clusters.</p> <p>Knox delivers three groups of user facing services:</p> <img src="http://web.archive.org/web/20230914024541im_/https://knox.apache.org/images/knox-services.png" alt="Services" style="width: 600px;"/> <ul> <li> <p><b>Proxying Services</b><br/> Primary goals of the Apache Knox project is to provide access to Apache Hadoop via proxying of HTTP resources.</p> </li> <li> <p><b>Authentication Services</b><br/> Authentication for REST API access as well as WebSSO flow for UIs. LDAP/AD, Header based PreAuth, Kerberos,<br/> SAML, OAuth are all available options.</p> </li> <li> <p><b>Client Services</b><br/> Client development can be done with scripting through DSL or using the Knox Shell classes directly as SDK. The KnoxShell interactive scripting environment<br/> combines the interactive shell of groovy shell with the Knox Shell SDK classes for a interating with data<br/> from your deployed Hadoop cluster.</p> </li> </ul></div> <div class="section"> <h2><a name="Overview"></a>Overview</h2> <p>The Knox API Gateway is designed as a reverse proxy with consideration for pluggability in the areas of<br/> policy enforcement, through providers and the backend services for which it proxies requests.</p> <p>Policy enforcement ranges from authentication/federation, authorization, audit, dispatch, hostmapping<br/> and content rewrite rules. Policy is enforced through a chain of providers that are defined within the topology<br/> deployment descriptor for each Apache Hadoop cluster gated by Knox. The cluster definition is also defined<br/> within the topology deployment descriptor and provides the Knox Gateway with the layout of the<br/> cluster for purposes of routing and translation between user facing URLs and cluster internals.</p> <p>Each Apache Hadoop cluster that is protected by Knox has its set of REST APIs represented by a single cluster specific<br/> application context path. This allows the Knox Gateway to both protect multiple clusters and present<br/> the REST API consumer with a single endpoint for access to all of the services required, across the<br/> multiple clusters.</p> <p>Simply by writing a topology deployment descriptor to the topologies directory of the Knox installation, a<br/> new Apache Hadoop cluster definition is processed, the policy enforcement providers are configured and the application<br/> context path is made available for use by API consumers.</p> <p>While there are a number of benefits for unsecured Apache Hadoop clusters,<br/> the Knox Gateway also complements the kerberos secured cluster quite nicely.</p> <p>Coupled with proper network isolation of a Kerberos secured Apache Hadoop cluster,<br/> the Knox Gateway provides the enterprise with a solution that:</p> <ul> <li>Integrates well with enterprise identity management solutions</li> <li>Protects the details of the cluster deployment (hosts and ports are hidden from endusers)</li> <li>Simplifies the number of services that clients need to interact with</li> </ul></div> <div class="section"> <h2><a name="Supported_Apache_Hadoop_Services"></a>Supported Apache Hadoop Services</h2> <p>The following Apache Hadoop ecosystem services have integrations with the Knox Gateway:</p> <p>Ambari<br/> Cloudera Manager<br/> WebHDFS (HDFS)<br/> Yarn RM<br/> Stargate (Apache HBase)<br/> Apache Oozie<br/> Apache Hive/JDBC<br/> Apache Hive WebHCat (Templeton)<br/> Apache Storm<br/> Apache Tinkerpop - Gremlin<br/> Apache Avatica/Phoenix<br/> Apache SOLR<br/> Apache Livy (Spark REST Service)<br/> Apache Flink<br/> Kafka REST Proxy<br/></p></div> <div class="section"> <h2><a name="Supported_Apache_Hadoop_ecosystem_UIs"></a>Supported Apache Hadoop ecosystem UIs</h2> <p>Name Node UI<br/> Job History UI<br/> Yarn UI<br/> Apache Oozie UI<br/> Apache HBase UI<br/> Apache Spark UI<br/> Apache Ambari UI<br/> Apache Impala<br/> Apache Ranger Admin Console<br/> Apache Zeppelin<br/> Apache NiFi<br/> Hue<br/> Livy<br/></p></div> <div class="section"> <h2><a name="Configuring_Support_for_new_services_and_UIs"></a>Configuring Support for new services and UIs</h2> <p>Apache Knox provides a configuration driven method of adding new routing services.<br/> This enables for new Apache Hadoop REST APIs to come on board very quickly and easily. It also enables<br/> users and developers to add support for custom REST APIs to the Knox gateway as well.<br/> This capability was added in release 0.6.0 and furthers the Knox commitment to extensibility and integration.</p></div> <div class="section"> <h2><a name="Home_Page"></a>Home Page</h2> <p>Knox provides a conenient Home Page that may be used as the front door to your deployment and<br/> the resources that you have published for access through Apache Knox. This is a nice alternative<br/> to having to distribute a link to the administrative interface in order to get Quick Links.<br/></p></div> <div class="section"> <h2><a name="Authentication"></a>Authentication</h2> <p>Providers with the role of authentication are responsible for collecting credentials presented by the API<br/> consumer, validating them and communicating the successful or failed authentication to the client or the<br/> rest of the provider chain.</p> <p>Out of the box, the Knox Gateway provides the Shiro authentication provider. This is a provider that leverages<br/> the Apache Shiro project for authenticating BASIC credentials against an LDAP user store. There is support for<br/> OpenLDAP, ApacheDS and Microsoft Active Directory.</p></div> <div class="section"> <h2><a name="Federation.2FSSO"></a>Federation/SSO</h2> <p>For customers that require credentials to be presented to a limited set of trusted entities within the enterprise,<br/> the Knox Gateway may be configured to federate the authenticated identity from an external authentication event.<br/> This is done through providers with the role of federation. The set of out-of-the-box federation providers include:<br/></p> <div class="section"> <div class="section"> <h4><a name="KnoxSSO_Default_Form-based_IDP_-"></a>KnoxSSO Default Form-based IDP -</h4> <p>The default configuration of KnoxSSO provides a form-based authentication mechanism that leverages the Shiro authentication<br/> to authenticate against LDAP/AD with credentials collected from a form-based challenge.</p></div> <div class="section"> <h4><a name="Pac4J_-"></a>Pac4J -</h4> <p>The pac4j provider adds numerous authentication and federation capabilities including: SAML, CAS, OpenID Connect, Google,<br/> Twitter, etc.</p></div> <div class="section"> <h4><a name="HeaderPreAuth_-"></a>HeaderPreAuth -</h4> <p>A simple mechanism for propagating the identity through HTTP Headers that specify the username and group for the<br/> authenticated user. This has been built with vendor usecases such as SiteMinder and IBM Tivoli Access Manager.</p></div></div></div> <div class="section"> <h2><a name="KnoxSSO"></a>KnoxSSO</h2> <p>The KnoxSSO service is an integration service that provides a normalized SSO token for representing the authenticated user.<br/> This token is generally used for WebSSO capabilities for participating UIs and their consumption of the Apache Hadoop REST APIs.<br/> KnoxSSO abstracts the actual identity provider integration away from participating applications so that they only need to<br/> be aware of the KnoxSSO cookie. The token is presented by the browser as a cookie and applications that are participating in<br/> the KnoxSSO integration are able to cryptographically validate the presented token and remain agnostic to the underlying<br/> SSO integration.</p></div> <div class="section"> <h2><a name="Authorization"></a>Authorization</h2> <p>The authorization role is used by providers that make access decisions for the requested resources based on the<br/> effective user identity context. This identity context is determined by the authentication provider and the identity<br/> assertion provider mapping rules. Evaluation of the identity context’s user and group principals against a set of<br/> access policies is done by the authorization provider in order to determine whether access should be granted to<br/> the effective user for the requested resource.</p> <p>Out of the box, the Knox Gateway provides an ACL based authorization provider that evaluates rules that comprise<br/> of username, groups and ip addresses. These ACLs are bound to and protect resources at the service level.<br/> That is, they protect access to the Apache Hadoop services themselves based on user, group and remote ip address.</p></div> <div class="section"> <h2><a name="Audit"></a>Audit</h2> <p>The ability to determine what actions were taken by whom during some period of time is provided by the auditing<br/> capabilities of the Knox Gateway. The facility is built on an extension of the Log4j framework and may be extended<br/> by replacing the out of the box implementation with another.</p></div> </div> </div> </div> <hr/> <footer> <div class="container-fluid"> <div class="row-fluid"> <div class="row span12"> Copyright © 2019 <a href="http://web.archive.org/web/20230914024541/https://www.apache.org/">Apache Software Foundation</a>. All Rights Reserved. <br/> Apache Knox Gateway, Apache, the Apache feather logo and the Apache Knox Gateway project logos are trademarks of The Apache Software Foundation. <br/> All other marks mentioned may be trademarks or registered trademarks of their respective owners. <br/> <a href="http://web.archive.org/web/20230914024541/https://cwiki.apache.org/confluence/display/KNOX/Privacy+Policy">Privacy Policy</a> </div> <div align="right" class="row span12"> <img vertical-align="middle" src="/web/20230914024541im_/https://knox.apache.org/images/apache-logo.gif" alt="Generic placeholder image"/> </div> </div> </div> </footer> </body> </html>