Apache Helix

<!DOCTYPE html>  <html xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <title>Apache Helix – Home</title> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="description" content="" /> <meta http-equiv="content-language" content="en" /> <link href="./css/bootstrap.min.css" rel="stylesheet" /> <link href="./css/bootstrap-responsive.min.css" rel="stylesheet" /> <link href="./css/docs.css" rel="stylesheet" /> <link href="./css/reflow-skin.css" rel="stylesheet" /> <link href="./css/lightbox.css" rel="stylesheet" /> <link href="./css/site.css" rel="stylesheet" /> <link href="./css/print.css" rel="stylesheet" media="print" />    <script> var _paq = window._paq = window._paq || []; _paq.push(['disableCookies']); _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="https://analytics.apache.org"; _paq.push(['setTrackerUrl', u+'/matomo.php']); _paq.push(['setSiteId', '15']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); })(); </script>  </head> <body class="page-index project-website" data-spy="scroll" data-offset="60" data-target="#toc-scroll-target"> <div class="navbar navbar-fixed-top"> <div class="navbar-inner"> <div class="container"> <a class="btn btn-navbar" data-toggle="collapse" data-target="#top-nav-collapse"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </a> <a class="brand" href="./">Apache Helix</a> <div class="nav-collapse collapse" id="top-nav-collapse"> <ul class="nav pull-right"> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Learn <b class="caret"></b></a> <ul class="dropdown-menu"> <li ><a href="Concepts.html" title="Core Concepts">Core Concepts</a></li> <li ><a href="Architecture.html" title="Architecture">Architecture</a></li> <li ><a href="Publications.html" title="Publications">Publications</a></li> <li ><a href="ClientLibraries.html" title="Client Libraries">Client Libraries</a></li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> <ul class="dropdown-menu"> <li ><a href="1.4.3-docs/index.html" title="1.4.3">1.4.3</a></li> <li ><a href="1.3.1-docs/index.html" title="1.3.1">1.3.1</a></li> <li ><a href="0.9.9-docs/index.html" title="0.9.10 (0.9.9)">0.9.10 (0.9.9)</a></li> <li ><a href="trunk-docs/index.html" title="trunk">trunk</a></li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Helix 1.4.3 <b class="caret"></b></a> <ul class="dropdown-menu"> <li ><a href="1.4.3-docs/index.html" title="Documentation">Documentation</a></li> <li ><a href="1.4.3-docs/Quickstart.html" title="Quick Start">Quick Start</a></li> <li ><a href="1.4.3-docs/Tutorial.html" title="Tutorial">Tutorial</a></li> <li ><a href="1.4.3-docs/download.html" title="Download">Download</a></li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Helix 0.9.10 (0.9.9) <b class="caret"></b></a> <ul class="dropdown-menu"> <li ><a href="0.9.9-docs/index.html" title="Documentation">Documentation</a></li> <li ><a href="0.9.9-docs/Quickstart.html" title="Quick Start">Quick Start</a></li> <li ><a href="0.9.9-docs/Tutorial.html" title="Tutorial">Tutorial</a></li> <li ><a href="0.9.9-docs/download.html" title="Download">Download</a></li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Get Involved <b class="caret"></b></a> <ul class="dropdown-menu"> <li ><a href="IRC.html" title="IRC">IRC</a></li> <li ><a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a></li> <li ><a href="issue-tracking.html" title="Issues">Issues</a></li> <li ><a href="team-list.html" title="Team">Team</a></li> <li ><a href="sources.html" title="Sources">Sources</a></li> <li ><a href="integration.html" title="Continuous Integration">Continuous Integration</a></li> <li ><a href="involved/building.html" title="Building Guide">Building Guide</a></li> <li ><a href="releasing.html" title="Release Guide">Release Guide</a></li> <li ><a href="involved/contribdocs.html" title="Improve this Website">Improve this Website</a></li> </ul> </li> <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> <ul class="dropdown-menu"> <li ><a href="https://www.apache.org/" title="ASF Home" class="externalLink">ASF Home</a></li> <li ><a href="https://www.apache.org/licenses/" title="License" class="externalLink">License</a></li> <li ><a href="https://www.apache.org/foundation/sponsorship.html" title="Sponsorship" class="externalLink">Sponsorship</a></li> <li ><a href="https://www.apache.org/foundation/thanks.html" title="Thanks" class="externalLink">Thanks</a></li> <li ><a href="https://www.apache.org/security/" title="Security" class="externalLink">Security</a></li> </ul> </li> </ul> </div> </div> </div> </div> <div class="container">  <header> <div class="jumbotron subhead"> <div class="row" id="banner"> <div class="span12"> <div class="pull-left"> <a href="./" id="bannerLeft"><img src="images/helix-logo.jpg" alt='"''"' /></a> <p class="lead">A cluster management framework for partitioned and replicated distributed resources</p> </div> <div class="pull-right"> <a href="https://www.apache.org/" id="bannerRight"><img src="images/feather_small.gif" alt='"''"' /></a> </div> </div> </div> </div> <div> <ul class="breadcrumb"> <li><a href="./" title="Apache Helix">Apache Helix</a></li> <li class="divider">/</li> <li>Home</li> </ul> </div> </header> <div class="main-body"> <div class="row"> <div class="span8"> <div class="body-content"> <section> <div class="page-header"> <h2 id="Apache_Helix">Apache Helix</h2> </div> <p>Apache Helix is a generic <i>cluster management</i> framework used for the automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes. <b>Helix automates reassignment of resources in the face of node failure and recovery, cluster expansion, and reconfiguration.</b></p> </section> <section> <h2 id="What_Is_Cluster_Management.3F">What Is Cluster Management?</h2> <p>To understand Helix, you first need to understand <b>cluster management</b>. A distributed system typically runs on multiple nodes for the following reasons:</p> <ul> <li>scalability</li> <li>fault tolerance</li> <li>load balancing</li> </ul> <p>Each node performs one or more of the primary functions of the cluster, such as storing and serving data, producing and consuming data streams, and so on. <b>Once configured for your system, Helix acts as the global brain for the system</b>. It is designed to make decisions that cannot be made in isolation. Examples of such decisions that require global knowledge and coordination:</p> <ul> <li>scheduling of maintenance tasks, such as backups, garbage collection, file consolidation, index rebuilds</li> <li>repartitioning of data or resources across the cluster</li> <li>informing dependent systems of changes so they can react appropriately to cluster changes</li> <li>throttling system tasks and changes</li> </ul> <p>While it is possible to integrate these functions into the distributed system, it complicates the code. Helix has abstracted common cluster management tasks, enabling the system builder to model the desired behavior with a declarative state model, and let Helix manage the coordination. <b>The result is less new code to write, and a robust, highly operable system</b>.</p> </section> <section> <h2 id="What_does_Helix_provide.3F">What does Helix provide?</h2> <ul> <li>Automatic assignment of resources and partitions to nodes</li> <li>Node failure detection and recovery</li> <li>Dynamic addition of resources</li> <li>Dynamic addition of nodes to the cluster</li> <li>Pluggable distributed state machine to manage the state of a resource via state transitions</li> <li>Automatic load balancing and throttling of transitions</li> <li>Optional pluggable rebalancing for user-defined assignment of resources and partitions</li> </ul> </section> <section> <h2 id="Why_Helix.3F">Why Helix?</h2> <p>Modeling a distributed system as a state machine with constraints on states and transitions has the following benefits:</p> <ul> <li>Separates cluster management from the core functionality of the system.</li> <li>Allows a quick transformation from a single node system to an operable, distributed system.</li> <li>Increases simplicity: system components do not have to manage a global cluster. This division of labor makes it easier to build, debug, and maintain your system.</li> </ul> </section> </div> </div> <div class="span4"> <div class="well sidebar"> <h3 id="Join_the_Conversation">Join the Conversation</h3> <p><a class="externalLink" href="https://www.meetup.com/Building-distributed-systems-using-Apache-Helix-Meetup-group/">Bay Area Meetup Group</a></p> <p><a href="./IRC.html"><code>#apachehelix</code></a></p> <p><a class="externalLink" href="mailto:user@helix.apache.org"><code>user@helix.apache.org</code></a></p> <section> </section> <h3 id="News">News</h3> <p>Apache Helix has a new release:</p> <ul> <li> <p><a href="./1.4.3-docs/index.html">1.4.3</a> - A release containing minor fixes for instance operation and task framework</p> <p><a href="./1.4.3-docs/Quickstart.html">[Quick Start]</a> <a href="./1.4.3-docs/releasenotes/release-1.4.3.html">[Release Notes]</a></p> </li> <li> <p><a href="./0.9.9-docs/index.html">0.9.9</a> - A release with several critical bug fixing for 0.9 series release</p> <p><a href="./0.9.9-docs/Quickstart.html">[Quick Start]</a> <a href="./0.9.9-docs/releasenotes/release-0.9.9.html">[Release Notes]</a></p> </li> </ul> <section> </section> <h3 id="Download">Download</h3> <p><a href="./1.4.3-docs/download.html" class="btn btn-primary btn-small">1.4.3</a></p> <p><a href="./0.9.9-docs/download.html" class="btn btn-primary btn-small">0.9.9</a></p> <section> </section> <h3 id="Maven_Dependency">Maven Dependency</h3> <div class="source"> <pre><code><dependency> <groupId>org.apache.helix</groupId> <artifactId>helix-core</artifactId> <version>1.4.3</version> </dependency> </code></pre> </div> <section> </section> <h3 id="Building">Building</h3> <p>Requirements: JDK 1.8+, Maven 3.6.0+ for 1.1.0 and below</p> <p>Requirements: JDK 11, Maven 3.6.0+ for 1.2.0+</p> <div class="source"> <pre><code>git clone https://github.com/apache/helix.git cd helix git checkout helix-1.4.3 mvn install package -DskipTests </code></pre> </div> </div> </div> </div> </div> </div>  <footer class="well"> <div class="container"> <div class="row"> <div class="span2 bottom-nav"> <ul class="nav nav-list"> <li class="nav-header">Learn</li> <li > <a href="Concepts.html" title="Core Concepts">Core Concepts</a> </li> <li > <a href="Architecture.html" title="Architecture">Architecture</a> </li> <li > <a href="Publications.html" title="Publications">Publications</a> </li> <li > <a href="ClientLibraries.html" title="Client Libraries">Client Libraries</a> </li> </ul> </div> <div class="span2 bottom-nav"> <ul class="nav nav-list"> <li class="nav-header">Documentation</li> <li > <a href="1.4.3-docs/index.html" title="1.4.3">1.4.3</a> </li> <li > <a href="1.3.1-docs/index.html" title="1.3.1">1.3.1</a> </li> <li > <a href="0.9.9-docs/index.html" title="0.9.10 (0.9.9)">0.9.10 (0.9.9)</a> </li> <li > <a href="trunk-docs/index.html" title="trunk">trunk</a> </li> <li class="nav-header">Helix 1.4.3</li> <li > <a href="1.4.3-docs/index.html" title="Documentation">Documentation</a> </li> <li > <a href="1.4.3-docs/Quickstart.html" title="Quick Start">Quick Start</a> </li> <li > <a href="1.4.3-docs/Tutorial.html" title="Tutorial">Tutorial</a> </li> <li > <a href="1.4.3-docs/download.html" title="Download">Download</a> </li> </ul> </div> <div class="span2 bottom-nav"> <ul class="nav nav-list"> <li class="nav-header">Get Involved</li> <li > <a href="IRC.html" title="IRC">IRC</a> </li> <li > <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> </li> <li > <a href="issue-tracking.html" title="Issues">Issues</a> </li> <li > <a href="team-list.html" title="Team">Team</a> </li> <li > <a href="sources.html" title="Sources">Sources</a> </li> <li > <a href="integration.html" title="Continuous Integration">Continuous Integration</a> </li> <li > <a href="involved/building.html" title="Building Guide">Building Guide</a> </li> <li > <a href="releasing.html" title="Release Guide">Release Guide</a> </li> <li > <a href="involved/contribdocs.html" title="Improve this Website">Improve this Website</a> </li> </ul> </div> <div class="span3 bottom-nav"> <ul class="nav nav-list"> <li class="nav-header">ASF</li> <li > <a href="https://www.apache.org/" title="ASF Home" class="externalLink">ASF Home</a> </li> <li > <a href="https://www.apache.org/licenses/" title="License" class="externalLink">License</a> </li> <li > <a href="https://www.apache.org/foundation/sponsorship.html" title="Sponsorship" class="externalLink">Sponsorship</a> </li> <li > <a href="https://www.apache.org/foundation/thanks.html" title="Thanks" class="externalLink">Thanks</a> </li> <li > <a href="https://www.apache.org/security/" title="Security" class="externalLink">Security</a> </li> </ul> </div> </div> </div> </footer> <div class="container subfooter"> <div class="row"> <div class="span12"> <p class="pull-right"><a href="#">Back to top</a></p> <p class="copyright">Copyright ©2025 <a href="https://www.apache.org">Apache Software Foundation</a>. All Rights Reserved.</p> <p><a href="https://github.com/olamy/reflow-maven-skin" title="Reflow Maven skin">Reflow Maven skin</a> maintained by <a href="https://twitter.com/olamy" target="_blank" title="Olivier Lamy">Olivier Lamy</a>.</p> <div class="row span16"><div>Apache Helix, Apache, the Apache feather logo, and the Apache Helix project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</div> <a href="https://helix.apache.org/privacy-policy.html">Privacy Policy</a> </div> </div> </div> </div>   <script src="./js/jquery.min.js"></script> <script src="./js/bootstrap.min.js"></script> <script src="./js/lightbox.min.js"></script> <script src="./js/reflow-scroll.js"></script> <script src="./js/reflow-skin.js"></script> </body> </html>

CINXE.COM

Apache Helix – Home