CINXE.COM

<html> <head><script type="text/javascript" src="/_static/js/bundle-playback.js?v=HxkREWBo" charset="utf-8"></script> <script type="text/javascript" src="/_static/js/wombat.js?v=txqj7nKC" charset="utf-8"></script> <script>window.RufflePlayer=window.RufflePlayer||{};window.RufflePlayer.config={"autoplay":"on","unmuteOverlay":"hidden"};</script> <script type="text/javascript" src="/_static/js/ruffle/ruffle.js"></script> <script type="text/javascript"> __wm.init("https://web.archive.org/web"); __wm.wombat("http://www.insightful.com:80/Hesterberg/bootstrap/","20060110182635","https://web.archive.org/","web","/_static/", "1136917595"); </script> <link rel="stylesheet" type="text/css" href="/_static/css/banner-styles.css?v=S1zqJCYt" /> <link rel="stylesheet" type="text/css" href="/_static/css/iconochive.css?v=3PDvdIFv" />   <title>Bootstrap and Resampling Software, and articles by Tim Hesterberg </title> <meta content="bootstrap, bootstrap tilting, Insightful, consulting, software, statistics, statistical, S-PLUS, s-plus, splus, S+Resample, teaching statistics, resampling, resample, resampling software, introductory statistics" name="KEYWORDS"> </head> <h1>Bootstrap and Permutation Tests</h1> <h2>Contents of this page:</h2> <li><a href="#Reference.software">Bootstrap/Resampling Software</a> <li><a href="#Reference.shortCourse">Resampling Short Course</a> <li><a href="#Reference.introStat">Bootstrap for Introductory Statistics</a> <li><a href="#Reference.articles">Articles and Technical Reports</a> <h2> <a name="Reference.software">Bootstrap/Resampling Software</a> </h2> The next generation of S-PLUS software for bootstrap and other resampling procedures is available for download from <a href="https://web.archive.org/web/20060110182635/http://www.insightful.com/downloads/libraries">http://www.insightful.com/downloads/libraries</a>. Please email <a href="https://web.archive.org/web/20060110182635/mailto:bootstrap-beta@insightful.com">bootstrap-beta@insightful.com</a> to let us know that you have downloaded this; we will contact you when there are new versions. <p> The code falls into five general categories: <li> Menu-driven-interface (Windows only). Easy access to common tasks. <li> General-purpose S-PLUS software, primarily upgrades of existing S-PLUS functions in order to support capabilities of bootstrap software. For example, many functions now have an optional "weights" argument. <li> New versions of `bootstrap' and related code, with added functionality or bug fixes. <li> New resampling functions, for permutation tests, cross validation, bootstrapping prediction errors, parametric and smoothed bootstrapping. <li> New functions for bootstrap tilting inferences and diagnostics. <p> A feature of this code is bootstrap tilting, which generally provides very accurate (second-order accurate) confidence intervals, with better finite-sample performance than other bootstrap methods, and require only 1/17 or 1/37 as many bootstrap replications as other bootstrap methods (for 95% confidence intervals), e.g. 60 replications instead of 1000 or 2000. <p> For more details on what is included see <a href="description.txt">description.txt</a>, <a href="ReleaseNotes.txt">ReleaseNotes.txt</a>, and <a href="ChangeLog.txt">ChangeLog.txt</a>, <p> This project is supported by the National Institutes of Health under SBIR Phase-II grant 2R44CA67734--02 "Statistical Software for Resampling Methods", and by the National Science Foundation, under SBIR Phase-II grant DMI-9861360, "Bootstrap Tilting Inference and Large Data Sets". <p> Contributors to this software include Andrea Borning, Steve Ellis, Chris Fraley, Tim Hesterberg, Shan Jin, Charles Roosen, James Schimert, and Robert Thurman. We also appreciate advice received from David S. Moore, Brad Efron, Art Owen, Luigi Salmaso, and Rob Tibshirani. <h2> <a name="Reference.shortCourse"> Short Course: Bootstrap Methods and Permutation Tests</a></h2> This is an introduction to the bootstrap, permutation tests, and other resampling methods. For a course description and details see <a href="course.asp">course.asp</a>. I've given this in various formats, ranging from a two-day hands-on course to half-day lecture-only, public or private, in Boston, San Francisco, Chicago, Toronto, London, Basingstoke, Zurich, and Basel. <h2> <a name="Reference.introStat"> Bootstrap for Introductory Statistics</a></h2> The bootstrap and permutation tests offer ways to help students better understand concepts such as sampling distributions, standard errors, confidence intervals, and P-values. <i>Bootstrap Methods and Permutation Tests</i> (BMPT) by Hesterberg, Moore, Monaghan, Clipson, and Epstein was written as an introduction to these methods, with a focus on the pedagogical value. There are two versions of BMPT, written as supplemental chapters for two different books, but either can be used independently by people who want an introduction to bootstrap methods and permutation tests. <p> The first version (<b>"BMPT/PBS"</b>) is a supplemental chapter for <a href="https://web.archive.org/web/20060110182635/http://www.whfreeman.com/pbs"> The Practice of Business Statistics: Using Data for Decisions </a> by Moore, McCabe, Duckworth and Sclove. This is available from W. H. Freeman, ISBN 0-7167-5726-5 for about $7, or is available at <a href="https://web.archive.org/web/20060110182635/http://bcs.whfreeman.com/pbs/cat_160/PBS18.pdf">http://bcs.whfreeman.com/pbs/cat_160/PBS18.pdf</a>. <p> The second version (<b>"BMPT/IPS"</b>) is a supplemental chapter for <a href="https://web.archive.org/web/20060110182635/http://www.whfreeman.com/ips5e"> Introduction to the Practice of Statistics, 5th Edition</a> by Moore and McCabe. This is available at <a href="https://web.archive.org/web/20060110182635/http://bcs.whfreeman.com/ips5e/content/cat_080/pdf/moore14.pdf">http://bcs.whfreeman.com/ips5e/content/cat_080/pdf/moore14.pdf</a>. See also <a href="https://web.archive.org/web/20060110182635/http://www.whfreeman.com/ipsresample">http://www.whfreeman.com/ipsresample</a>. <h4>S-PLUS data libraries and supplements for PBS and IPS</h4> There are S-PLUS libraries to accompany both versions, containing datasets, example scripts, and documentation. <br> For BMPT/PBS download <a href="PBSdata.zip">PBSdata.zip</a>. <br>For BMPT/IPS download <a href="IPSdata.zip">IPSdata.zip</a>. <br> Unzip either library, then follow instructions in INSTALL.txt. <p> For a general introduction to S-PLUS, see the <a href="https://web.archive.org/web/20060110182635/http://bcs.whfreeman.com/ips5e/content/cat_120/IPS_Splus.pdf"> S-PLUS Guide for Moore and McCabe's Introduction to the Practice of Statistics, Fifth Edition<a>. <h4><a name="SPLUS">Free Student version of S-PLUS</a></h4> You need S-PLUS to use either the IPSdata or PBSdata library. Students may get a free student version of S-PLUS from <a href="https://web.archive.org/web/20060110182635/http://elms03.e-academy.com/splus"> http://elms03.e-academy.com/splus</a>. The e-academy procedures can be difficult to follow, you may find these instructions helpful: <a href="e-academy-instructions.txt">e-academy-instructions.txt</a>. <p> Teachers may get a free evaluation copy from <a href="https://web.archive.org/web/20060110182635/http://www.insightful.com/contactus/request_cd.asp"> http://www.insightful.com/contactus/request_cd.asp</a>. <h2> <a name="Reference.articles"> Articles and Technical Reports related to this software:</a></h2> <p> <li> Tim Hesterberg, David S. Moore, Shaun Monaghan, Ashley Clipson, and Rachel Epstein (2005), <a href="https://web.archive.org/web/20060110182635/http://bcs.whfreeman.com/ips5e/content/cat_080/pdf/moore14.pdf"> Bootstrap Methods and Permutation Tests, 2nd edition</a>, W. H. Freeman, N.Y. <li> Gregory Snow, Laura Chihara, and Tim Hesterberg (2005), <a href="https://web.archive.org/web/20060110182635/http://bcs.whfreeman.com/ips5e/content/cat_120/IPS_Splus.pdf"> S-PLUS Guide for Moore and McCabe's Introduction to the Practice of Statistics, Fifth Edition<a> W. H. Freeman, N.Y. <li> Hesterberg, Tim (2005), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/Paris05-ResampleClinical.pdf"> Resampling for Planning Clinical Trials-Using S+Resample</a>, poster for "Statistical Methods in Biopharmacy" conference, Paris. <li> Hesterberg, Tim C. (2004), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/JSM04-bootknife.pdf"> Unbiasing the Bootstrap-Bootknife Sampling vs. Smoothing</a>, Proceedings of the Section on Statistics and the Environment, American Statistical Association, 2924-2930. <li> Tim Hesterberg, Shaun Monaghan, David S. Moore, Ashley Clipson, and Rachel Epstein (2003), <a href="https://web.archive.org/web/20060110182635/http://bcs.whfreeman.com/pbs/cat_160/PBS18.pdf"> Bootstrap Methods and Permutation Tests</a>, W. H. Freeman, N.Y. <li> Hesterberg, Tim C. (2002), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/Telecom02-permutation.pdf"> "Performance Evaluation using Fast Permutation Tests" </a> Proceedings of the Tenth International Conference on Telecommunication Systems, 465-474. <li> Hesterberg, Tim C. (2001), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/JSM01-diagnostics.pdf"> "Bootstrap Tilting Diagnostics"</a>, Proceedings of the Statistical Computing Section (CD-ROM), American Statistical Association. <li> Hesterberg, Tim C. (1999), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/Interface99-tiltingCI.pdf"> "Bootstrap Tilting Confidence Intervals and Hypothesis Tests"</a>, Computing Science and Statistics, 31, 389--393, Interface Foundation of North America, Fairfax Station, VA. <li> Hesterberg, Tim C. (1999), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/tech84-tiltingCI.pdf"> "Bootstrap Tilting Confidence Intervals"</a>, Technical Report No. 84, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109. <li> Ellis, Stephen J. and Tim C. Hesterberg (1999), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/tech85-weightedFunc.pdf"> "Computation of Weighted Functional Statistics Using Software That Does Not Support Weights"</a>, Technical Report No. 85, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109. <li> Hesterberg, Tim C. and Stephen J. Ellis (1999), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/tech86-linear.pdf"> "Linear Approximations for Functional Statistics in Large-Sample Applications"</a>, Technical Report No. 86, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109. <li> Hesterberg, Tim C. (1999), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/tech87-bootknife.pdf"> "Smoothed bootstrap and jackboot sampling"</a>, Technical Report No. 87, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109. <li> Hesterberg, Tim C. (1998), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/JSM98-boot-teaching.pdf"> "Simulation and Bootstrapping for Teaching Statistics"</a>, Proceedings of the Statistical Education Section, American Statistical Association, 44--52. <li> Hesterberg, Tim C. (1998), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/NSFTiltingProposal.pdf"> "Bootstrap Tilting Inference and Large Data Sets"</a>, Proposal to NSF SBIR Program. <li> Hesterberg, Tim C. (1997), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/Interface97-fast-boot.pdf"> "Fast Bootstrapping by Combining Importance Sampling and Concomitants"</a>, Computing Science and Statistics, 29(2), 72-78. Interface Foundation of North America, Fairfax Station, VA. Eds E. J. Wegman and S. Azen. <li> Hesterberg, T. C. (1997), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/JSM97-bootstrap-EL.pdf"> "The bootstrap and empirical likelihood"</a>, Proceedings of the Section on Statistical Computing, American Statistical Association, 34-36. <li> Hesterberg, Tim C. (1997), <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/tech66-matchBlock.pdf"> "Matched-Block Bootstrap for Long Memory Processes"</a>, Technical Report No. 66, Research Department, MathSoft, Inc. 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109. <p> For other articles (including references to published articles related to this software) see <a href="/web/20060110182635/http://www.insightful.com/Hesterberg/articles/default.asp">www.insightful.com/Hesterberg/articles</a> <hr><br> Tim Hesterberg <a href="https://web.archive.org/web/20060110182635/mailto:timh@insightful.com">timh@insightful.com</a>, 206-802-2319, <a href="..">www.insightful.com/Hesterberg</a> <br> Updated Oct 21, 2005. </html>