CINXE.COM
Higher-Order Perl
<html> <head><title>Higher-Order Perl</title> <base href="http://hop.perl.plover.com/"> <link rel='stylesheet' type='text/css' href='common.css' /> <link rel="SHORTCUT ICON" href="/favicon.ico" /> </head> <body bgcolor="#ffffff"> <a href="cover.html"><img src="cover-med.jpg" align=right border=0></a> <h1>Higher-Order Perl</h1> <h2>by Mark Jason Dominus</h2> <!-- <IMG SRC="http://service.bfast.com/bfast/serve?bfmid=2181&sourceid=41259285&bfpid=1558607013&bfmtype=book" BORDER="0" WIDTH="1" HEIGHT="1" NOSAVE > --> <p><a href="ORDER.html">Order <cite>Higher-Order Perl</cite> online from Powell's Books</a></p> <p><a href="book/">Download full text</a></p> <p><b>Current Status:</b> The book was published on 8 March 2005. Copies are now available in bookstores in the U.S. and Europe, and from online booksellers. A second printing became available in August 2005.</p> <br clear=all> <a href="hopcn/"><img src="hopcn/hopcn_cover_medsm.jpg" align=right border=0></a> <p> <img src="../pics/new.gif" border=0 ALT="**NEW**"> A <a href="hopcn/">Chinese-language edition of HOP</a>, translated by Mr. Teng Jiahai, was published in June 2013 by China Machine Press and is available from Chinese-language online booksellers. </p> <ul> <li>ISBN 1558607013 <li><a href="#idea">Idea of the Book</a> <li><a href="#toc">Outline</a> <li><a href="#free">HOP is available for free</a> <li><img src="../pics/new.gif" border=0 ALT="**NEW**"> <a href="hopcn/">HOP in Chinese (高阶 Perl)</a> <li><a href="#title">About the Title</a> <li><a href="#mlist"><!--<img src="../pics/updated.gif" border=0 ALT="**UPDATED**"> -->Mailing Lists</a> <li><a href="cover.html">About the cover</a> <li><a href="preface.html">Preface</a> <li><a href="acknowledgments.html">Acknowledgments</a> <li><a href="reviews.html"><!-- <img src="../pics/updated.gif" border=0 ALT="**UPDATED**"> -->Reviews</a> <li><a href="index/index.cgi">Index</a> <li><a href="grep.cgi">Full-text search</a> <li><a href="Examples/">Code Examples</a> <li>Errata list for <a href="errata/errata-d-1.html">first printing</a> <a href="errata/errata-d-2.html">second printing</a> <a href="errata/huh.html">huh?</a> <li><a href="bookplates.html">Obtaining a signed copy</a> <li><a href="http://www.theperlreview.com/Interviews/mjd-hop-20050407.html"> Interview with <cite>The Perl Review</cite></a> <li><a href="http://interglacial.com/hoj/">Higher-Order Javascript</a> <li><a href="http://graysoftinc.com/higher-order-ruby">Higher-Order Ruby</a> <li><a href="http://perl.plover.com/classes/hop/"><cite>Higher-Order Perl</cite> Class</a> </ul> <a name="idea"><h2>Idea of the Book</h2></a> <p><cite>Higher-Order Perl</cite> is about functional programming techniques in Perl. It's about how to write functions that can modify and manufacture other functions.</p> <!-- It's difficult to explain briefly exactly what this means---it's as if someone was writing a book about object-oriented programming in 1982 when hardly anyone had heard of object-oriented programming. Obviously, functional programming isn't simply programming with functions, because everyone does that already. Perhaps the best summary is that it's about writing functions that can manufacture other functions.</p> --> <p>Why would you want to do that? Because that way your code is more flexible and more reusable. Instead of writing ten similar functions, you write a general pattern or framework that can generate the functions you want; then you generate just the functions you need according to the pattern. The program doesn't need to know in advance which functions are necessary; it can generate them as needed. Instead of writing the complete program yourself, you get the computer to write it for you. </p> <!-- <hr> <a name="manifesto"><h2>Manifesto</h2></a> <p>There are many useful, powerful programming techniques that are well-known in academia but which are not in common use among industrial and commercial programmers.</p> <p>The position of these techniques is similar to that of recursion in the 1970's. Twenty-five years ago, it was common to meet programmers who believed the following:</p> <ul> <li><p> That recursion techniques were unimportant;</p> <li><p> That recursion was inefficient;</p> <li><p>That recursion was unnecessary, because anything that can be accomplished with recursion can also be accomplished with iterative techniques;</p> <li> <p>That recursion was an ivory-tower concept, not useful in day-to-day industrial programming;</p> </ul> <p>or simply that they had not heard of recursion and did not know what it was for.</p> <p>It is difficult now to believe that such views were ever common. The change in attitude since 1970 has been complete. If anyone tried today to promulgate a general-purpose programming language that did not support recursion, they would be laughed at. A programmer who doesn't understand recursion is a pitiful anachronism.</p> <p>It is important to notice that some the beliefs above are <em>correct</em>. Recursion <em>is</em> inefficient. Recursion <em>is</em> unnecessary because its ends can always be accomplished with iterative techniques. But these criticisms miss the point, which is that recursion is nevertheless an important and powerful programming technique that can make many programs simpler. Many problems naturally have recursive structure, and recursive programs to solve them are more straightforward and direct than the equivalent iterative programs. -- A similar argument once raged over the use of high-level languages, which are inefficient compared with assembly language, and the high-level languages won out in spite of being inefficient. -- <p>There was a vicious cycle in the acceptance of recursion. Popular languages of the 1960's, Fortran and Cobol, did not support recursion. Programmers never saw recursion used, and could not use it even if they did understand it, because the languages did not support it. This engendered attitudes like the ones I listed above. Because programmers saw no need for recursion, there was no tendency to develop languages that were capable of recursion, and the cycle continued.</p> <p>What broke the cycle was the introduction of new languages, most notably C and Pascal, which did support recursion. These languages came from outside the industrial and commercial programming communities. C was developed at Bell Laboratories, and Pascal at ETH Zürich, both research institutions. When languages like C and Pascal became popular, the usefulness of recursion was recognized, and it became a standard technique among programmers.</p> <p>I believe that there are a number of programming techniques whose position today is similar to that of recursion in 1975. They are little-known and poorly understood outside of the academic community, because most commercial and industrial programming languages do not support them. As a result, few new languages are developed to support these techniques, because programmers generally don't appreciate their usefulness.</p> <p>Perl, however, is unusual. It contains the essential features necessary to apply these techniques. Among these features are: first-class function values; dynamic construction of anonymous functions, variables with lexically closed scope, and automatic garbage collection. Perl is already in use in industry, so the vicious cycle can be broken.</p> <p>HOP will explore powerful programming techniques, formerly only available in academic languages, that are possible in Perl because of the presence of language features such as anonymous first-class functions.</p> <p>HOP will be aimed at professional programmers who already know Perl and who work in commerce and industry. It will demonstrate real-world applications of the advanced techniques, such as database lookups, parsing, and physical simulations.</p> --> <hr> <a name="toc"><h2>Table of Contents</h2></a> <ol> <li><a href="chap01.html">Recursion and Callbacks</a> <li><a href="chap02.html">Dispatch Tables</a> <li><a href="chap03.html">Caching and Memoization</a> <li><a href="chap04.html">Iterators</a> <li><a href="chap05.html">From Recursion to Iterators</a> <li><a href="chap06.html">Infinite Streams</a> <li><a href="chap07.html">Higher-Order Functions</a> <li><a href="chap08.html">Parsing</a> <li><a href="chap09.html">Declarative Programming</a> </ol> <p>HOP is full of useful and interesting examples that Perl programmers will want to see. <a href="chap01.html">Chapter 1</a> is about writing recursive functions to walk the file system and to process recursively-structured data such as HTML documents. <a href="chap02.html">Chapter 2</a> discusses dispatch tables in the context of reading configuration files. <a href="chap04.html">Chapter 4</a> concludes with an improved replacement for the <tt>WWW::SimpleRobot</tt> module, with better functionality in <i>one-third</i> the code, and also contains a basic database query system that searches an HTTP log file backwards, delivering the most recent records first. <a href="chap06.html">Chapter 6</a> discusses the frequently asked question of how to generate a list of all the strings that will match a given regex and provides an astonishingly simple solution. Later on, in <a href="chap08.html">Chapter 8</a>, I show how to build a parser for regexes and attach it to the string generator. <a href="chap09.html">Chapter 9</a> presents <i>linogram</i>, a complete constraint-based diagram-drawing system. </p> <hr> <h2><a name="free">HOP is available for free</a></h2> <p><a href="book/">It is here</a>. You can download the publisher's final PDF proofs for the second printing, or you can download my uncorrected manuscript copy in MOD format.</p> <p>Sorry for the delay. I hoped to have it posted by the end of 2005. Oops.</p> <p>You can also use this <a href="grep.cgi">full-text search</a> to search the book.</p> <h2><a name="title">About the Title</a></h2> <p>For a long time, the provisional title of the book was "Perl Advanced Techniques Handbook". We never planned to use this title; we always planned to change it once we found something better. </p> <hr> <!-- <p><cite>HOP</cite> has 563 pages of main text, plus 16 pages of front matter (introduction and table of contents, mostly) and 18 pages of back matter (indexes mostly) for a total of 597 pages.</p> --> <a name="mlist"><h2>Mailing Lists</h2></a> <p><b><a href="mailto:mjd-book-subscribe@plover.com">mjd-book</a>:</b> I used to send occasional reports of my progress on HOP this mailing list. I have not sent any announcements since HOP was published, but it is possible that I will send some in the future, for example if a translation of the book is published. This list is <i>extremely</i> low volume. It is quite possible that it will never see another message. <a href="announce/">Complete archive of past messages</a>.</p> <p><b><a href="mailto:mjd-books-subscribe@plover.com">mjd-books</a>:</b> I am working on at least one more book. For announcements about this upcoming book and any other books I might write in the future, subscribe to this list. Subscribers get to see draft chapters, course materials related to my books, and other goodies. This list is low-volume. <a href="http://perl.plover.com/flagbook/announce/">Complete archive of past messages</a>.</p> <p><b><a href="mailto:hop-discuss-subscribe@plover.com">hop-discuss</a>:</b> A public discussion list for HOP-related matters. <a href="http://plover.com:8080/~alias/list.cgi/2/">Complete archive of past messages</a></p> <p><b><a href="mailto:hop-discuss-digest-subscribe@plover.com">hop-discuss-digest</a>:</b> Digest version of <tt>hop-discuss</tt>. Instead of one message at a time, this list receives the messages in batches, every few days.</p> <p><b><a name="mlist-v"><a href="mailto:hop-volunteer@plover.com">hop-volunteer</a></a>:</b> <img src="../pics/new.gif" border=0 ALT="**NEW**"> Would you like to help me put <cite>HOP</cite> online? Please read the <a href="http://plover.com:8080/~alias/list.cgi/3/">archived messages</a> and subscribe here.</p> <a name="subscribe"> <p>To subscribe to any of these lists, send mail to: <ul> <li><a href="mailto:mjd-book-subscribe@plover.com">mjd-book-subscribe@plover.com</a> <li><a href="mailto:mjd-books-subscribe@plover.com">mjd-books-subscribe@plover.com</a> <li><a href="mailto:hop-discuss-subscribe@plover.com">hop-discuss-subscribe@plover.com</a> <li><a href="mailto:hop-discuss-digest-subscribe@plover.com">hop-discuss-digest-subscribe@plover.com</a> <li><a href="mailto:hop-volunteer-subscribe@plover.com">hop-volunteer-subscribe@plover.com</a> </ul> or submit this form:</p> <form method=POST action='/subscribe.cgi'> Subscribe to:<br> <input type=checkbox name="list" value="mjd-book"> mjd-book <input type=checkbox name="list" value="mjd-books"> mjd-books <input type=checkbox name="list" value="hop-discuss"> hop-discuss <input type=checkbox name="list" value="hop-discuss-digest"> hop-discuss-digest <input type=checkbox name="list" value="hop-volunteer"> hop-volunteer<br> Your email address: <input type="text" name="/addr" value=""> <input type=hidden name='/return-name' value="Higher-Order Perl"> <input type=hidden name='/return' value="/#mlist"> <input type=submit value="Subscribe"> </form> <p align=center><font size="+2"><a href="ORDER.html">Order <cite>Higher-Order Perl</cite> online from Powell's Books</a></font></p> <hr> <img src="../pics/small-sigils.gif" align=right> <p>Return to: <a href="http://blog.plover.com/">Mark Dominus blog</a> | <a href="http://www.plover.com/~mjd/whatsnew.html">What's new page</a> | <a href="http://perl.plover.com/">Perl Paraphernalia</a> </p> <p><a href="mailto:mjd-perl-hop+@plover.com">mjd-perl-hop+@plover.com</a></p> </body> </html>