CINXE.COM
Google SRE - What is Toil in SRE: Understanding Its Impact
<!DOCTYPE html> <html lang="en"> <head> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-WVF23W3');</script> <meta charset="utf-8"> <meta content="initial-scale=1, minimum-scale=1, width=device-width" name="viewport"> <title>Google SRE - What is Toil in SRE: Understanding Its Impact</title> <meta name="description" content="Learn what toil is in SRE, how it affects operational work, and why minimizing it is crucial for efficiency and morale."> <meta name="referrer" content="no-referrer" /> <link rel="canonical" href="https://sre.google/sre-book/eliminating-toil/"> <link rel="apple-touch-icon-precomposed" sizes="180x180" href="https://lh3.googleusercontent.com/Yf2DCX8RKda6r4Jml9DLMByS2zQCBFs3kQpvBfN8UgIh4YVWIYSYIQOoTxJriyuM26cT5PDjyEb5aynDQ0Xyz46yHKnfg8JlUbDW"> <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Google+Sans:400|Roboto:400,400italic,500,500italic,700,700italic|Roboto+Mono:400,500,700|Material+Icons"> <link rel="icon" type="image/png" sizes="32x32" href="https://lh3.googleusercontent.com/Yf2DCX8RKda6r4Jml9DLMByS2zQCBFs3kQpvBfN8UgIh4YVWIYSYIQOoTxJriyuM26cT5PDjyEb5aynDQ0Xyz46yHKnfg8JlUbDW"> <link rel="icon" type="image/png" sizes="16x16" href="https://lh3.googleusercontent.com/Yf2DCX8RKda6r4Jml9DLMByS2zQCBFs3kQpvBfN8UgIh4YVWIYSYIQOoTxJriyuM26cT5PDjyEb5aynDQ0Xyz46yHKnfg8JlUbDW"> <link rel="shortcut icon" href="https://lh3.googleusercontent.com/Yf2DCX8RKda6r4Jml9DLMByS2zQCBFs3kQpvBfN8UgIh4YVWIYSYIQOoTxJriyuM26cT5PDjyEb5aynDQ0Xyz46yHKnfg8JlUbDW"> <link href="/sre-book/static/css/index.min.css?cache=6c30b59" rel="stylesheet"> <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-75468017-1', 'auto'); ga('send', 'pageview'); </script> <script type="application/ld+json"> { "@context": "https://schema.org", "@type": "Article", "mainEntityOfPage": { "@type": "WebPage", "@id": "/sre-book/eliminating-toil/" }, "headline": "Eliminating Toil", "description": "In SRE, we want to spend time on long-term engineering project work instead of operational work. Because the term operational work may be misinterpreted, we use a specific word: toil.", "author": [ { "@type": "Person", "name": "Vivek Rau" } ], "publisher": { "@type": "Organization", "name": "Google SRE", "logo": { "@type": "ImageObject", "url": "https://lh3.googleusercontent.com/C3_YVnTdc7xzTDekhsGeZ2hEYUnAlp47Au-9C50vi5r44rfpJAgiycs1g6AFKWqIpw6KVPrZWLse1VUqgOqYht-RxV1iowdB0_IABUd966aDsDWW-65m" } } } </script> <script src="/sre-book/static/js/detect.min.js?cache=4cb778b"></script> </head> <body> <noscript><iframe class="no-script-iframe" src="https://www.googletagmanager.com/ns.html?id=GTM-WVF23W3"></iframe></noscript> <main> <div ng-controller= "HeaderCtrl as headerCtrl"> <div id="curtain" class="menu-closed"></div> <div class="header clearfix"> <a id="burger-menu" class="expand"></a> <h2 class="chapter-title"> Chapter 5 - Eliminating Toil </h2> </div> <div id="overlay-element" class="expands"> <div class="logo"> <a href="https://www.google.com"><img src="https://lh3.googleusercontent.com/YoVRtLOHMSRYQZ3OhFL8RIamcjFYbmQXX4oAQx02MRqqY9zlKNvsuZpS73khXiOqTH3qrFW27VrERJJIHTjPk-tAh46q8-Fd4w6qlw" alt="Google"></a> </div> <ol id="drop-down" class="dropdown-content hide"> <li><a class="menu-buttons" href="/sre-book/table-of-contents/">Table of Contents</a></li> <li> <a href="/sre-book/foreword/" class="menu-buttons"> Foreword </a> </li> <li> <a href="/sre-book/preface/" class="menu-buttons"> Preface </a> </li> <li> <a href="/sre-book/part-I-introduction/" class="menu-buttons"> Part I - Introduction </a> </li> <li> <a href="/sre-book/introduction/" class="menu-buttons"> 1. Introduction </a> </li> <li> <a href="/sre-book/production-environment/" class="menu-buttons"> 2. The Production Environment at Google, from the Viewpoint of an SRE </a> </li> <li> <a href="/sre-book/part-II-principles/" class="menu-buttons"> Part II - Principles </a> </li> <li> <a href="/sre-book/embracing-risk/" class="menu-buttons"> 3. Embracing Risk </a> </li> <li> <a href="/sre-book/service-level-objectives/" class="menu-buttons"> 4. Service Level Objectives </a> </li> <li class="active"> <a href="/sre-book/eliminating-toil/" class="menu-buttons"> 5. Eliminating Toil </a> </li> <li> <a href="/sre-book/monitoring-distributed-systems/" class="menu-buttons"> 6. Monitoring Distributed Systems </a> </li> <li> <a href="/sre-book/automation-at-google/" class="menu-buttons"> 7. The Evolution of Automation at Google </a> </li> <li> <a href="/sre-book/release-engineering/" class="menu-buttons"> 8. Release Engineering </a> </li> <li> <a href="/sre-book/simplicity/" class="menu-buttons"> 9. Simplicity </a> </li> <li> <a href="/sre-book/part-III-practices/" class="menu-buttons"> Part III - Practices </a> </li> <li> <a href="/sre-book/practical-alerting/" class="menu-buttons"> 10. Practical Alerting </a> </li> <li> <a href="/sre-book/being-on-call/" class="menu-buttons"> 11. Being On-Call </a> </li> <li> <a href="/sre-book/effective-troubleshooting/" class="menu-buttons"> 12. Effective Troubleshooting </a> </li> <li> <a href="/sre-book/emergency-response/" class="menu-buttons"> 13. Emergency Response </a> </li> <li> <a href="/sre-book/managing-incidents/" class="menu-buttons"> 14. Managing Incidents </a> </li> <li> <a href="/sre-book/postmortem-culture/" class="menu-buttons"> 15. Postmortem Culture: Learning from Failure </a> </li> <li> <a href="/sre-book/tracking-outages/" class="menu-buttons"> 16. Tracking Outages </a> </li> <li> <a href="/sre-book/testing-reliability/" class="menu-buttons"> 17. Testing for Reliability </a> </li> <li> <a href="/sre-book/software-engineering-in-sre/" class="menu-buttons"> 18. Software Engineering in SRE </a> </li> <li> <a href="/sre-book/load-balancing-frontend/" class="menu-buttons"> 19. Load Balancing at the Frontend </a> </li> <li> <a href="/sre-book/load-balancing-datacenter/" class="menu-buttons"> 20. Load Balancing in the Datacenter </a> </li> <li> <a href="/sre-book/handling-overload/" class="menu-buttons"> 21. Handling Overload </a> </li> <li> <a href="/sre-book/addressing-cascading-failures/" class="menu-buttons"> 22. Addressing Cascading Failures </a> </li> <li> <a href="/sre-book/managing-critical-state/" class="menu-buttons"> 23. Managing Critical State: Distributed Consensus for Reliability </a> </li> <li> <a href="/sre-book/distributed-periodic-scheduling/" class="menu-buttons"> 24. Distributed Periodic Scheduling with Cron </a> </li> <li> <a href="/sre-book/data-processing-pipelines/" class="menu-buttons"> 25. Data Processing Pipelines </a> </li> <li> <a href="/sre-book/data-integrity/" class="menu-buttons"> 26. Data Integrity: What You Read Is What You Wrote </a> </li> <li> <a href="/sre-book/reliable-product-launches/" class="menu-buttons"> 27. Reliable Product Launches at Scale </a> </li> <li> <a href="/sre-book/part-IV-management/" class="menu-buttons"> Part IV - Management </a> </li> <li> <a href="/sre-book/accelerating-sre-on-call/" class="menu-buttons"> 28. Accelerating SREs to On-Call and Beyond </a> </li> <li> <a href="/sre-book/dealing-with-interrupts/" class="menu-buttons"> 29. Dealing with Interrupts </a> </li> <li> <a href="/sre-book/operational-overload/" class="menu-buttons"> 30. Embedding an SRE to Recover from Operational Overload </a> </li> <li> <a href="/sre-book/communication-and-collaboration/" class="menu-buttons"> 31. Communication and Collaboration in SRE </a> </li> <li> <a href="/sre-book/evolving-sre-engagement-model/" class="menu-buttons"> 32. The Evolving SRE Engagement Model </a> </li> <li> <a href="/sre-book/part-V-conclusions/" class="menu-buttons"> Part V - Conclusions </a> </li> <li> <a href="/sre-book/lessons-learned/" class="menu-buttons"> 33. Lessons Learned from Other Industries </a> </li> <li> <a href="/sre-book/conclusion/" class="menu-buttons"> 34. Conclusion </a> </li> <li> <a href="/sre-book/availability-table/" class="menu-buttons"> Appendix A. Availability Table </a> </li> <li> <a href="/sre-book/service-best-practices/" class="menu-buttons"> Appendix B. A Collection of Best Practices for Production Services </a> </li> <li> <a href="/sre-book/incident-document/" class="menu-buttons"> Appendix C. Example Incident State Document </a> </li> <li> <a href="/sre-book/example-postmortem/" class="menu-buttons"> Appendix D. Example Postmortem </a> </li> <li> <a href="/sre-book/launch-checklist/" class="menu-buttons"> Appendix E. Launch Coordination Checklist </a> </li> <li> <a href="/sre-book/production-meeting/" class="menu-buttons"> Appendix F. Example Production Meeting Minutes </a> </li> <li> <a href="/sre-book/bibliography/" class="menu-buttons"> Bibliography </a> </li> </ol> </div> </div> <div id="maia-main"> <div class="content" id="content"> <h1 class="heading jumptargets" id="eliminating-toil">Eliminating Toil</h1> <p class="byline author">Written by Vivek Rau<br>Edited by Betsy Beyer</p> <blockquote> <p class="quote">If a human operator needs to touch your system during normal operations, you have a bug. The definition of normal changes as your systems grow.</p> <p class="quote-author">Carla Geisser, Google SRE</p> </blockquote> <p>In SRE, we want to spend time on long-term engineering project work instead of operational work. Because the term <em>operational work</em> may be misinterpreted, we use a specific word: <em>toil</em>.</p> <h2 class="heading jumptargets" id="toil-defined">Toil Defined</h2> <p>Toil is not just "work I don鈥檛 like to do." It鈥檚 also not simply equivalent to administrative chores or grungy work. Preferences as to what types of work are satisfying and enjoyable vary from person to person, and some people even enjoy manual, repetitive work. There are also administrative chores that have to get done, but should not be categorized as toil: this is <em>overhead</em>. Overhead is often work not directly tied to running a production service, and includes tasks like team meetings, setting and grading goals,<sup><a class="jumptarget" data-type="noteref" id="id-dA2uXtyF9Tk-marker" href="#id-dA2uXtyF9Tk">19</a></sup> snippets,<sup><a class="jumptarget" data-type="noteref" id="id-BWDu1hjFGTg-marker" href="#id-BWDu1hjFGTg">20</a></sup> and HR paperwork. Grungy work can sometimes have long-term value, and in that case, it鈥檚 not toil, either. Cleaning up the entire alerting configuration for your service and removing clutter may be grungy, but it鈥檚 not toil.</p> <p>So what <em>is</em> toil? Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows. Not every task deemed toil has all these attributes, but the more closely work matches one or more of the following descriptions, the more likely it is to be toil:</p> <dl class="desc-list"> <dt class="dt-heading jumptargets" id="manual">Manual</dt> <dd>This includes work such as manually running a script that automates some task. Running a script may be quicker than manually executing each step in the script, but the <em>hands-on</em> time a human spends running that script (not the elapsed time) is still toil time.</dd> <dt class="dt-heading jumptargets" id="repetitive">Repetitive</dt> <dd>If you鈥檙e performing a task for the first time ever, or even the second time, this work is not toil. Toil is work you do over and over. If you鈥檙e solving a novel problem or inventing a new solution, this work is not toil.</dd> <dt class="dt-heading jumptargets" id="automatable">Automatable</dt> <dd>If a machine could accomplish the task just as well as a human, or the need for the task could be designed away, that task is toil. If human judgment is essential for the task, there鈥檚 a good chance it鈥檚 not toil.<sup><a class="jumptarget" data-type="noteref" id="id-EYJuMSqSwTgtKTk-marker" href="#id-EYJuMSqSwTgtKTk">21</a></sup></dd> <dt class="dt-heading jumptargets" id="tactical">Tactical</dt> <dd>Toil is interrupt-driven and reactive, rather than strategy-driven and proactive. Handling pager alerts is toil. We may never be able to eliminate this type of work completely, but we have to continually work toward minimizing it.</dd> <dt class="dt-heading jumptargets" id="no-enduring-value">No enduring value</dt> <dd>If your service remains in the same state after you have finished a task, the task was probably toil. If the task produced a permanent improvement in your service, it probably wasn鈥檛 toil, even if some amount of grunt work鈥攕uch as digging into legacy code and configurations and straightening them out鈥攚as involved.</dd> <dt class="dt-heading jumptargets" id="o-n-with-service-growth">O(n) with service growth</dt> <dd>If the work involved in a task scales up linearly with service size, traffic volume, or user count, that task is probably toil. An ideally managed and designed service can grow by at least one order of magnitude with zero additional work, other than some one-time efforts to add resources.</dd> </dl> <h1 class="heading jumptargets" id="why-less-toil-is-better">Why Less Toil Is Better</h1> <p>Our SRE organization has an advertised goal of keeping operational work (i.e., toil) below 50% of each SRE鈥檚 time. At least 50% of each SRE鈥檚 time should be spent on engineering project work that will either reduce future toil or add service features. Feature development typically focuses on improving reliability, performance, or utilization, which often reduces toil as a second-order effect.</p> <p>We share this 50% goal because toil tends to expand if left unchecked and can quickly fill 100% of everyone鈥檚 time. The work of reducing toil and scaling up services is the "Engineering" in Site Reliability Engineering. Engineering work is what enables the SRE organization to scale up sublinearly with service size and to manage services more efficiently than either a pure Dev team or a pure Ops team.</p> <p>Furthermore, when we hire new SREs, we promise them that SRE is not a typical Ops organization, quoting the 50% rule just mentioned. We need to keep that promise by not allowing the SRE organization or any subteam within it to devolve into an Ops team.</p> <h5 class="subheaders jumptargets" id="calculating-toil">Calculating Toil</h5> <p>If we seek to cap the time an SRE spends on toil to 50%, how is that time spent?</p> <p>There鈥檚 a floor on the amount of toil any SRE has to handle if they are on-call. A typical SRE has one week of primary on-call and one week of secondary on-call in each cycle (for discussion of primary versus secondary on-call shifts, see <a href="/sre-book/being-on-call/">Being On-Call</a>). It follows that in a 6-person rotation, at least 2 of every 6 weeks are dedicated to on-call shifts and interrupt handling, which means the lower bound on potential toil is 2/6 = 33% of an SRE鈥檚 time. In an 8-person rotation, the lower bound is 2/8 = 25%.</p> <p>Consistent with this data, SREs report that their top source of toil is interrupts (that is, non-urgent service-related messages and emails). The next leading source is on-call (urgent) response, followed by releases and pushes. Even though our release and push processes are usually handled with a fair amount of automation, there鈥檚 still plenty of room for improvement in this area.</p> <p>Quarterly surveys of Google鈥檚 SREs show that the average time spent toiling is about 33%, so we do much better than our overall target of 50%. However, the average doesn鈥檛 capture outliers: some SREs claim 0% toil (pure development projects with no on-call work) and others claim 80% toil. When individual SREs report excessive toil, it often indicates a need for managers to spread the toil load more evenly across the team and to encourage those SREs to find satisfying engineering projects.</p> <h1 class="heading jumptargets" id="what-qualifies-as-engineering">What Qualifies as Engineering?</h1> <p>Engineering work is novel and intrinsically requires human judgment. It produces a permanent improvement in your service, and is guided by a strategy. It is frequently creative and innovative, taking a design-driven approach to solving a problem鈥攖he more generalized, the better. Engineering work helps your team or the SRE organization handle a larger service, or more services, with the same level of staffing.</p> <p>Typical SRE activities fall into the following approximate categories:</p> <dl class="desc-list"> <dt class="dt-heading jumptargets" id="software-engineering">Software engineering</dt> <dd>Involves writing or modifying code, in addition to any associated design and documentation work. Examples include writing automation scripts, creating tools or frameworks, adding service features for scalability and reliability, or modifying infrastructure code to make it more robust.</dd> <dt class="dt-heading jumptargets" id="systems-engineering">Systems engineering</dt> <dd>Involves configuring production systems, modifying configurations, or documenting systems in a way that produces lasting improvements from a one-time effort. Examples include monitoring setup and updates, load balancing configuration, server configuration, tuning of OS parameters, and load balancer setup. Systems engineering also includes consulting on architecture, design, and productionization for developer teams.</dd> <dt class="dt-heading jumptargets" id="toil">Toil</dt> <dd>Work directly tied to running a service that is repetitive, manual, etc.</dd> <dt class="dt-heading jumptargets" id="overhead">Overhead</dt> <dd>Administrative work not tied directly to running a service. Examples include hiring, HR paperwork, team/company meetings, bug queue hygiene, snippets, peer reviews and self-assessments, and training courses.</dd> </dl> <p>Every SRE needs to spend at least 50% of their time on engineering work, when averaged over a few quarters or a year. Toil tends to be spiky, so a steady 50% of time spent on engineering may not be realistic for some SRE teams, and they may dip below that target in some quarters. But if the fraction of time spent on projects averages significantly below 50% over the long haul, the affected team needs to step back and figure out what鈥檚 wrong.</p> <h1 class="heading jumptargets" id="is-toil-always-bad">Is Toil Always Bad?</h1> <p>Toil doesn鈥檛 make everyone unhappy all the time, especially in small amounts. Predictable and repetitive tasks can be quite calming. They produce a sense of accomplishment and quick wins. They can be low-risk and low-stress activities. Some people gravitate toward tasks involving toil and may even enjoy that type of work.</p> <p>Toil isn鈥檛 always and invariably bad, and everyone needs to be absolutely clear that some amount of toil is unavoidable in the SRE role, and indeed in almost any engineering role. It鈥檚 fine in small doses, and if you鈥檙e happy with those small doses, toil is not a problem. Toil becomes toxic when experienced in large quantities. If you鈥檙e burdened with too much toil, you should be very concerned and complain loudly. Among the many reasons why too much toil is bad, consider the following:</p> <dl class="desc-list"> <dt class="dt-heading jumptargets" id="career-stagnation">Career stagnation</dt> <dd>Your career progress will slow down or grind to a halt if you spend too little time on projects. Google rewards grungy work when it鈥檚 inevitable and has a big positive impact, but you can鈥檛 make a career out of grunge.</dd> <dt class="dt-heading jumptargets" id="low-morale">Low morale</dt> <dd>People have different limits for how much toil they can tolerate, but everyone has a limit. Too much toil leads to burnout, boredom, and discontent.</dd> </dl> <p>Additionally, spending too much time on toil at the expense of time spent engineering hurts an SRE organization in the following ways:</p> <dl class="desc-list"> <dt class="dt-heading jumptargets" id="creates-confusion">Creates confusion</dt> <dd>We work hard to ensure that everyone who works in or with the SRE organization understands that we are an engineering organization. Individuals or teams within SRE that engage in too much toil undermine the clarity of that communication and confuse people about our role.</dd> <dt class="dt-heading jumptargets" id="slows-progress">Slows progress</dt> <dd>Excessive toil makes a team less productive. A product鈥檚 feature velocity will slow if the SRE team is too busy with manual work and firefighting to roll out new features promptly.</dd> <dt class="dt-heading jumptargets" id="sets-precedent">Sets precedent</dt> <dd>If you鈥檙e too willing to take on toil, your Dev counterparts will have incentives to load you down with even more toil, sometimes shifting operational tasks that should rightfully be performed by Devs to SRE. Other teams may also start expecting SREs to take on such work, which is bad for obvious reasons.</dd> <dt class="dt-heading jumptargets" id="promotes-attrition">Promotes attrition</dt> <dd>Even if you鈥檙e not personally unhappy with toil, your current or future teammates might like it much less. If you build too much toil into your team鈥檚 procedures, you motivate the team鈥檚 best engineers to start looking elsewhere for a more rewarding job.</dd> <dt class="dt-heading jumptargets" id="causes-breach-of-faith">Causes breach of faith</dt> <dd>New hires or transfers who joined SRE with the promise of project work will feel cheated, which is bad for morale.</dd> </dl> <h1 class="heading jumptargets" id="conclusion">Conclusion</h1> <p>If we all commit to eliminate a bit of toil each week with some good engineering, we鈥檒l steadily clean up our services, and we can shift our collective efforts to engineering for scale, architecting the next generation of services, and building cross-SRE toolchains. Let鈥檚 invent more, and toil less.</p> <div class="footnotes"><p data-type="footnote" id="id-dA2uXtyF9Tk"><sup><a class="jumptargets" href="#id-dA2uXtyF9Tk-marker">19</a></sup>We use the Objectives and Key Results system, pioneered by Andy Grove at Intel; see <a href="/sre-book/bibliography#Kla12">[Kla12]</a>.</p><p data-type="footnote" id="id-BWDu1hjFGTg"><sup><a class="jumptargets" href="#id-BWDu1hjFGTg-marker">20</a></sup>Googlers record short free-form summaries, or "snippets," of what we鈥檝e worked on each week.</p><p data-type="footnote" id="id-EYJuMSqSwTgtKTk"><sup><a class="jumptargets" href="#id-EYJuMSqSwTgtKTk-marker">21</a></sup>We have to be careful about saying a task is "not toil because it needs human judgment." We need to think carefully about whether the nature of the task intrinsically requires human judgment and cannot be addressed by better design. For example, one could build (and some have built) a service that alerts its SREs several times a day, where each alert requires a complex response involving plenty of human judgment. Such a service is poorly designed, with unnecessary complexity. The system needs to be simplified and rebuilt to either eliminate the underlying failure conditions or deal with these conditions automatically. Until the redesign and reimplementation are finished, and the improved service is rolled out, the work of applying human judgment to respond to each alert is definitely toil.</p></div> </div> </div> <div class="footer"> <div class="maia-aux"> <div class="previous"> <a href="/sre-book/service-level-objectives/"> <p class="footer-caption">Previous</p> <p class="chapter-link"> Chapter 4 - Service Level Objectives </p> </a> </div> <div class="next"> <a href="/sre-book/monitoring-distributed-systems/"> <p class="footer-caption">Next</p> <p class="chapter-link"> Chapter 6 - Monitoring Distributed Systems </p> </a> </div> <p class="footer-link">Copyright 漏 2017 Google, Inc. Published by O'Reilly Media, Inc. Licensed under <a href="https://creativecommons.org/licenses/by-nc-nd/4.0/" rel="noopener noreferrer" target="_blank">CC BY-NC-ND 4.0</a></p> </div> </div> </main> <script src="//ajax.googleapis.com/ajax/libs/angularjs/1.6.6/angular.min.js"></script> <script src="//ajax.googleapis.com/ajax/libs/angularjs/1.6.6/angular-animate.min.js"></script> <script src="//ajax.googleapis.com/ajax/libs/angularjs/1.6.6/angular-touch.min.js"></script> <script src="/sre-book/static/js/index.min.js?cache=5b7f90b"></script> </body> </html>