Samza - Apache Samza 1.1 <a href="/learn/documentation/1.1.0/"> [Docs] </a>

<!DOCTYPE html>  <html lang="en"> <head> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> <meta charset="utf-8"> <title>Samza - Apache Samza 1.1 <a href="/learn/documentation/1.1.0/"> [Docs] </a></title> <link rel="apple-touch-icon-precomposed" sizes="57x57" href="/img/favicon/apple-touch-icon-57x57.png" /> <link rel="apple-touch-icon-precomposed" sizes="114x114" href="/img/favicon/apple-touch-icon-114x114.png" /> <link rel="apple-touch-icon-precomposed" sizes="72x72" href="/img/favicon/apple-touch-icon-72x72.png" /> <link rel="apple-touch-icon-precomposed" sizes="144x144" href="/img/favicon/apple-touch-icon-144x144.png" /> <link rel="apple-touch-icon-precomposed" sizes="60x60" href="/img/favicon/apple-touch-icon-60x60.png" /> <link rel="apple-touch-icon-precomposed" sizes="120x120" href="/img/favicon/apple-touch-icon-120x120.png" /> <link rel="apple-touch-icon-precomposed" sizes="76x76" href="/img/favicon/apple-touch-icon-76x76.png" /> <link rel="apple-touch-icon-precomposed" sizes="152x152" href="/img/favicon/apple-touch-icon-152x152.png" /> <link rel="icon" type="image/png" href="/img/favicon/favicon-196x196.png" sizes="196x196" /> <link rel="icon" type="image/png" href="/img/favicon/favicon-96x96.png" sizes="96x96" /> <link rel="icon" type="image/png" href="/img/favicon/favicon-32x32.png" sizes="32x32" /> <link rel="icon" type="image/png" href="/img/favicon/favicon-16x16.png" sizes="16x16" /> <link rel="icon" type="image/png" href="/img/favicon/favicon-128.png" sizes="128x128" /> <meta name="application-name" content="https://samza.apache.org" /> <meta name="msapplication-TileColor" content="#FFFFFF" /> <meta name="msapplication-TileImage" content="/img/favicon/mstile-144x144.png" /> <meta name="msapplication-square70x70logo" content="/img/favicon/mstile-70x70.png" /> <meta name="msapplication-square150x150logo" content="/img/favicon/mstile-150x150.png" /> <meta name="msapplication-wide310x150logo" content="/img/favicon/mstile-310x150.png" /> <meta name="msapplication-square310x310logo" content="/img/favicon/mstile-310x310.png" /> <link href="/css/ionicons.min.css" rel="stylesheet"> <link href="/css/google-fonts.css" rel="stylesheet"> <link href="/css/syntax.css" rel="stylesheet"/> <link rel="stylesheet" href="/css/main.new.css" /> </head> <body class="page">  <div class="main-navigation" data-plugin="menu"> <div class="main-navigation__toggle" data-menu-closed> <i class="icon ion-md-menu"></i> </div> <div class="main-navigation__toggle main-navigation__toggle--opened" data-menu-opened> <i class="icon ion-md-close"></i> </div> <div class="main-navigation__inner"> <div class="main-navigation__logo"> <a href="/"> <img class="main-navigation__logo-img" src="/img/samza-logo.png" srcset="/img/samza-logo.png 1x, /img/samza-logo@2x.png 2x" alt="Samza Logo" /> </a> </div> <div class="main-navigation__items" data-menu-opened> <a class="main-navigation__item" href="/">Home</a> <a class="main-navigation__item" href="/learn/documentation/latest/core-concepts/core-concepts.html">Docs</a> <a class="main-navigation__item" href="/powered-by/">Powered By</a> <a class="main-navigation__item" href="/startup/download/">Downloads</a> <a class="main-navigation__item" href="/blog/">Blog</a> <div class="main-navigation__item main-navigation__item--group"> <div class="main-navigation__item-group-title"> Community <i class="icon ion-md-arrow-dropdown"></i> </div> <div class="main-navigation__item-group-list"> <a class="main-navigation__item" href="/community/contact-us.html">Contact Us</a> <a class="main-navigation__item" href="/contribute/contributors-corner.html">Contributor's Corner</a> <a class="main-navigation__item" href="/community/committers.html">PMC Members and committers</a> <a class="main-navigation__item" href="/meetups/">Talks and Meetups</a> </div> </div> </div> </div> </div> <div class="container"> <div class="container__toggle"> <i class="icon ion-md-arrow-dropleft-circle container__toggle-icon"></i> <i class="icon ion-md-arrow-dropright-circle container__toggle-icon container__toggle-icon--opened"></i> </div>   <div class="side-navigation">  <div class="side-navigation__group side-navigation__group--has-nested" data-plugin="sub-menu" data-sub-menu-show-class="side-navigation__group--has-nested-visible">  <div class="side-navigation__group-title"> <i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i> Getting Started </div> <div class="side-navigation__group-items " data-sub-menu >  <a class="side-navigation__group-item" data-match-active="" href="/startup/quick-start/latest/">QuickStart</a> <a class="side-navigation__group-item" data-match-active="" href="/startup/code-examples/latest/">Code Examples</a>   </div>  </div>  <div class="side-navigation__group side-navigation__group--has-nested" data-plugin="sub-menu" data-sub-menu-show-class="side-navigation__group--has-nested-visible">  <div class="side-navigation__group-title"> <i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i> Documentation </div> <div class="side-navigation__group-items side-navigation__group-has-submenus" data-sub-menu data-documentation="/learn/documentation/latest/">    </div>  </div>  <div class="side-navigation__group side-navigation__group--has-nested" data-plugin="sub-menu" data-sub-menu-show-class="side-navigation__group--has-nested-visible">  <div class="side-navigation__group-title"> <i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i> Releases </div> <div class="side-navigation__group-items " data-sub-menu >  <a class="side-navigation__group-item" data-match-active="" href="/releases/1.8.0">1.8.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.7.0">1.7.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.6.0">1.6.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.5.1">1.5.1</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.5.0">1.5.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.4.0">1.4.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.3.1">1.3.1</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.3.0">1.3.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.2.0">1.2.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.1.0">1.1.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/1.0.0">1.0.0</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/0.14">0.14</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/0.13">0.13</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/0.12">0.12</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/0.11">0.11</a> <a class="side-navigation__group-item" data-match-active="" href="/releases/0.10">0.10</a>   </div>  </div>  <div class="side-navigation__group">  <a class="side-navigation__group-title" data-plugin="top-menu" data-match-active="" href="/blog/"> Blog </a>     </div>  <div class="side-navigation__group side-navigation__group--has-nested" data-plugin="sub-menu" data-sub-menu-show-class="side-navigation__group--has-nested-visible">  <div class="side-navigation__group-title"> <i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i> Community </div> <div class="side-navigation__group-items " data-sub-menu >  <a class="side-navigation__group-item" data-match-active="" href="/community/contact-us.html">Contact Us</a> <a class="side-navigation__group-item" data-match-active="" href="/contribute/contributors-corner.html">Contributor's Corner</a> <a class="side-navigation__group-item" data-match-active="" href="/contribute/enhancement-proposal.html">Enhancement Proposal</a> <a class="side-navigation__group-item" data-match-active="" href="/community/committers.html">PMC members & Committers</a> <a class="side-navigation__group-item" data-match-active="" href="/meetups/">Talks and Meetups</a>   </div>  </div>  <div class="side-navigation__group side-navigation__group--has-nested" data-plugin="sub-menu" data-sub-menu-show-class="side-navigation__group--has-nested-visible">  <div class="side-navigation__group-title"> <i class="side-navigation__group-title-icon icon ion-md-arrow-dropdown"></i> Case Studies </div> <div class="side-navigation__group-items " data-sub-menu >  <a class="side-navigation__group-item" data-match-active="exact" href="/case-studies/">View All</a> <hr>  <a class="side-navigation__group-item" href="/case-studies/ebay" data-match-active="">eBay</a> <a class="side-navigation__group-item" href="/case-studies/tripadvisor" data-match-active="">TripAdvisor</a> <a class="side-navigation__group-item" href="/case-studies/slack" data-match-active="">Slack</a> <a class="side-navigation__group-item" href="/case-studies/optimizely" data-match-active="">Optimizely</a> <a class="side-navigation__group-item" href="/case-studies/redfin" data-match-active="">Redfin</a> <a class="side-navigation__group-item" href="/case-studies/linkedin" data-match-active="">LinkedIn</a>  </div>  </div> </div> <div class="section"> <div class="content"> <h2>Apache Samza 1.1 <a href="/learn/documentation/1.1.0/"> [Docs] </a></h2>  <p>We’re thrilled to announce to the release of Apache Samza 1.1.0.</p> <p>Today Samza forms the backbone of hundreds of real-time production applications across a multitude of companies, such as LinkedIn, VMWare, Slack, Redfin among many others. Samza provides leading support for large-scale stateful stream processing with:</p> <ul> <li> <p>First class support for local state (with RocksDB store). This allows a stateful application to scale up to 1.1 Million events/sec on a single machine with SSD.</p> </li> <li> <p>Support for incremental checkpointing of state instead of full snapshots. This enables Samza to scale to applications with very large state.</p> </li> <li> <p>A fully asynchronous programming model that makes parallelizing remote calls efficient and effortless.</p> </li> <li> <p>High level API for expressing complex stream processing pipelines in a few lines of code.</p> </li> <li> <p>Beam Samza Runner that marries Beam’s best in class support for EventTime based windowed processing and sophisticated triggering with Samza’s stable and scalable stateful processing model.</p> </li> <li> <p>A fully pluggable model for input sources (e.g. Kafka, Kinesis, DynamoDB streams etc.) and output systems (HDFS, Kafka, ElastiCache etc.).</p> </li> <li> <p>A Table API that provides a common abstraction for accessing remote or local databases and allowing developers are able to “join” an input event stream with such a Table.</p> </li> <li> <p>Flexible deployment model for running the applications in any hosting environment and with cluster managers other than YARN.</p> </li> <li> <p>Features like canaries, upgrades and rollbacks that support extremely large deployments with minimal downtime.</p> </li> </ul> <h3 id="new-features-upgrades-and-bug-fixes"><strong>New Features, Upgrades and Bug Fixes:</strong></h3> <p>This release brings the following features, upgrades, and capabilities:</p> <ul> <li>We have created a new Samza Stream Processing <a href="https://www.youtube.com/playlist?list=PLZDyxA22zzGyNgtBMUIXAgIaO5Ok3PR-x">video series</a> on Youtube</li> <li>New and improved documentation, code snippets, and examples for using the latest version of Samza with Apache Beam (Code samples are here: <a href="https://github.com/apache/samza-beam-examples">https://github.com/apache/samza-beam-examples</a>)</li> </ul> <h3 id="api-enhancements-and-simplifications"><strong>API enhancements and simplifications:</strong></h3> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1981">SAMZA-1981</a>: Consolidate table descriptors to samza-api.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1998">SAMZA-1998</a>: Table API refactoring.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1980">SAMZA-1980</a>: Rename LocalStoreBackedTable to LocalTable.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2043">SAMZA-2043</a>: Consolidate ReadableTable and ReadWriteTable.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2012">SAMZA-2012</a>: Add API for wiring an external context through to application processing.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2026">SAMZA-2026</a>: Refactor remote table API to separate retry policy settings.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2041">SAMZA-2041</a>: Add system descriptors for HDFS and Kinesis.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2081">SAMZA-2081</a>: Samza SQL: Type system for Samza SQL.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2106">SAMZA-2106</a>: Samza App & Job Config Refactor.</p> <h3 id="state-store-restoration"><strong>State Store Restoration:</strong></h3> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2018">SAMZA-2018</a>: State restore improvements using RocksDB writebatch API.</p> <h3 id="standalone-improvements"><strong>Standalone Improvements:</strong></h3> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1973">SAMZA-1973</a>: Unify the TaskNameGrouper interface for yarn and standalone.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1952">SAMZA-1952</a>: StreamPartitionCountMonitor for standalone.</p> <h3 id="other-upgrades-and-bug-fixes"><strong>Other Upgrades and Bug-fixes:</strong></h3> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1638">SAMZA-1638</a>: Recreate SystemProducer on KafkaCheckpointManager.writeCheckpoint failure.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-1946">SAMZA-1946</a>: Problem with Race between TimerListener initialization and timers fired from init().</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2004">SAMZA-2004</a>: Add ability to disable table metrics.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2013">SAMZA-2013</a>: Account for cycles in graph traversal within Execution Planner.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2015">SAMZA-2015</a>: Refactor timer handling in tables to be consistent with stores.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2072">SAMZA-2072</a>: Update guava to 23.0.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2090">SAMZA-2090</a>: Fix flush behavior for remote and hybrid tables.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2108">SAMZA-2108:</a> Check for host affinity config before resolving preferred host matching.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2109">SAMZA-2109</a>: Reduce default-buffer sizes for per-partition queues.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2118">SAMZA-2118</a>: Improve the shutdown sequence of AsyncRunLoop.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2119">SAMZA-2119</a>: Upgrading yarn-client version to 2.7.1.</p> <p><a href="https://issues.apache.org/jira/browse/SAMZA-2122">SAMZA-2122</a>: Fix the task caught-up logic which doesn’t handle no incoming messages</p> <p>The complete list of resolved Jira tickets for this release is found <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20resolution%20%20%3D%20Fixed%20%20AND%20(fixVersion%20%3E%3D%201.1%20)%20ORDER%20BY%20createdDate%20%20DESC">here</a>.</p> <h3 id="upgrading-your-application-to-apache-samza-110"><strong>Upgrading your application to Apache Samza 1.1.0</strong></h3> <p>Thank you on your decision to upgrade to Samza 1.1.0!</p> <h4 id="api-updates"><strong>API Updates</strong></h4> <p>The following imports for Table API have been updated:</p> <ul> <li> <p>Rename the import <em>org.apache.samza.storage.kv.descriptors.BaseLocalStoreBackedTableDescriptor</em> to <em>org.apache.samza.storage.kv.descriptors.BaseLocalTableDescriptor</em></p> </li> <li> <p>Rename the import <em>org.apache.samza.table.remote.descriptors.RemoteTableDescriptor</em> to <em>org.apache.samza.table.descriptors.RemoteTableDescriptor</em></p> </li> <li> <p>Rename the import <em>org.apache.samza.table.caching.descriptors.CachingTableDescriptor</em> to <em>org.apache.samza.table.descriptors.CachingTableDescriptor</em></p> </li> </ul> <h4 id="configurations-updates"><strong>Configurations Updates</strong></h4> <p>The job.name and job.id configs are now deprecated in favor of app.name and app.id configs respectively.</p> <p>A source download of Samza 1.1.0 is available <a href="https://dist.apache.org/repos/dist/release/samza/1.1.0/">here</a>, and is also available in Apache’s Maven repository. Samza’s download <a href="https://samza.apache.org/startup/download/">page</a> for details and Samza’s feature preview for new features.</p> <h2 id="community-developments"><strong>Community Developments</strong></h2> <p>A <a href="https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/">Stream Processing with Apache Kafka & Apache Samza meetup/symposium</a> that was held on March 20th which had following presentation for Samza:</p> <ul> <li>Apache Samza 1.0: Recent Advances and our plans for future in Stream Processing</li> </ul> <p>##<strong>Contribute</strong></p> <p>It’s a great time to get involved. You can start by reviewing the <a href="http://samza.apache.org/startup/preview/#try-it-out">tutorials</a>, signing up for the <a href="http://samza.apache.org/community/mailing-lists.html">mailing list</a>, and grabbing some <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20labels%20%3D%20newbie%20AND%20status%20%3D%20Open">newbie JIRAs</a>.</p> <p>I’d like to close by thanking everyone who’s been involved in the project. It’s been a great experience to be involved in this community, and I look forward to its continued growth.</p> </div> </div> </div>   <footer> <div class="footer-inner"> <div class="side-by-side"> <div> <div class="footer__heading">Learn More</div> <div class="footer__items"> <a class="footer__item" href="/meetups/">Meetups</a> <a class="footer__item" href="/blog/">Blog</a> <a class="footer__item" href="/learn/documentation/latest/introduction/background.html">About</a> </div> </div> <div> <div class="footer__heading">Community</div> <div class="footer__items"> <a class="footer__item" href="/community/contact-us.html">Contact Us</a> <a class="footer__item" href="/contribute/contributors-corner.html">Contributors' Corner</a> <a class="footer__item" href="/community/committers.html">PMC members and committers</a> <a class="footer__item" href="/powered-by/">Powered By</a> </div> </div> <div> <div class="quick-links"> <a class="quick-link" href="/startup/download" target="_blank"> <i class="icon ion-md-download"></i> </a> <a class="quick-link" href="https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tree" target="_blank"> <i class="icon ion-md-code"></i> </a> <a class="quick-link" href="https://twitter.com/samzastream" target="_blank"> <i class="icon ion-logo-twitter"></i> </a> </div> <p> <script>document.write(new Date().getFullYear());</script> © samza.apache.org</p> </div> </div> </div> </footer>  <script> (function (i, s, o, g, r, a, m) { i['GoogleAnalyticsObject'] = r; i[r] = i[r] || function () { (i[r].q = i[r].q || []).push(arguments) }, i[r].l = 1 * new Date(); a = s.createElement(o), m = s.getElementsByTagName(o)[0]; a.async = 1; a.src = g; m.parentNode.insertBefore(a, m) })(window, document, 'script', '//www.google-analytics.com/analytics.js', 'ga'); ga('create', 'UA-43122768-1', 'apache.org'); ga('send', 'pageview'); </script> <script src="/js/main.new.js"></script> </body> </html>

CINXE.COM

Samza - Apache Samza 1.1 [Docs]