CINXE.COM

<html> <head> <title>Erowid Experience Vaults : Report Reviewing: An Overview of the Review Process</title> <meta name="description" content="Experience Report Reviewing: A brief overview of the experience report review process."> <meta name="keywords" content="psychoactive drugs entheogens psychedelics hallucinogens phenethylamines experience report review critique writing documenting"> <link href="/includes/general_default.css" rel="stylesheet" type="text/css"> <link href="/includes/article_default.css" rel="stylesheet" type="text/css"> <link href="/includes/article_general_lb.css" rel="stylesheet" type="text/css"> <link href="/includes/border-left.css" rel="stylesheet" type="text/css"> <style type="text/css">  </style> </head> <body>  <script language="Javascript" src="/includes/headers/menu_data.js" type="text/javascript"></script> <script language="Javascript" src="/includes/headers/mmenu.js" type="text/javascript"></script> <table border="0" cellpadding="0" cellspacing="0" bgcolor="#000000"><tr> <td><a href="/index.shtml"><img src="/includes/headers/images/h_sun1.gif" width="122" height="25" alt="" border="0"></a></td> <td background="/includes/headers/images/bar_squiggle.gif" width="100%" colspan="4"> </td> </tr><tr> <td><a href="/index.shtml"><img src="/includes/headers/images/h_sun2.gif" width="122" height="70" alt="Erowid" border="0"></a></td> <td><a href="/index.shtml"><img src="/includes/headers/images/h_vaults.gif" width="120" height="70" alt="" border="0"></a></td> <td width="100%"> <table border="0" cellpadding="0" cellspacing="0" width="100" bgcolor="#000000" height="70"><tr> <td> </td> </tr></table> </td><td> <table border="0" cellpadding="0" cellspacing="0" width="316" height="70" background="/includes/headers/images/center_spirit_1-1.jpg"><tr> <td valign="top" align="right"> </td> </tr></table> </td><td> <table border="0" cellpadding="0" cellspacing="0" width="130" height="70" background="/includes/headers/images/center_spirit_1-2.jpg"> <tr height="30"><td align="left" valign="bottom">  <form name="SearchForm" method="get" action="/search.php">  <input type="text" size="13" name="q" value="" id="header-search"> <input type="image" src="/includes/headers/images/go2.gif" border="0" alt="Search"> </td></form></tr> <tr height="4"><td></td></tr> <tr height="16"> <td align="left"><a href="/general/about/" onMouseOver="popup('about')" onMouseOut="popdn()"><img src="/includes/headers/images/h_about.gif" width="89" height="13" alt="" border="0"></a></td> </tr><tr height="16"> <td align="left"><a href="/donations/" onMouseOver="popup('donations')" onMouseOut="popdn()"><img src="/includes/headers/images/h_join.gif" width="89" height="13" alt="" border="0"></a></td> </tr><tr height="4"><td></td> </tr></table> </td></tr><tr><td colspan="6"> <table border="0" cellpadding="0" cellspacing="0" width="100%"><tr> <td width="101"><img src="/includes/headers/images/bar_solid_left.gif" width="101" height="22" alt=""></td> <td align="left" width="114"><a href="/psychoactives/" onMouseOver="popup('drugs')" onMouseOut="popdn()"> <img src="/includes/headers/images/menu_1.gif" width="114" height="22" border="0" alt="Plants - Drugs"></a></td> <td align="left" width="103"><a href="/spirit/" onMouseOver="popup('mind')" onMouseOut="popdn()"> <img src="/includes/headers/images/menu_2.gif" width="103" height="22" border="0" alt="Mind - Spirit"></a></td> <td align="left" width="117"><a href="/freedom/" onMouseOver="popup('freedom')" onMouseOut="popdn()"> <img src="/includes/headers/images/menu_3.gif" width="117" height="22" border="0" alt="Freedom - Law"></a></td> <td align="left" width="110"><a href="/culture/" onMouseOver="popup('arts')" onMouseOut="popdn()"> <img src="/includes/headers/images/menu_4.gif" width="104" height="22" border="0" alt="Arts - Culture"></a></td> <td align="left" width="77"><a href="/library/" onMouseOver="popup('library')" onMouseOut="popdn()"> <img src="/includes/headers/images/menu_5.gif" width="77" height="22" border="0" alt="Library"></a></td> <td width="100%" background="/includes/headers/images/bar_solid_right.gif"> </td> </tr></table> </td></tr></table> <div id="main-frame"> <div id="content-outer-frame">  <div id="content-messpath-frame"> <div id="path"> <div class="path-chunk"><a href="/experiences/">EXPERIENCES</a></div></div> <div id="message">   <style> #message { background: #eebb00; border: 1px solid #000000; padding-top: 6px; padding-bottom: 5px; border-radius: 13px; } #message-inner-lg { font: 700 13pt georgia,times,serif; color: #330000; } #message-inner-md { font: 700 10pt arial,sans-serif; color: #000000; } </style> <a href="/cgi-bin/r.php?message_id=2&url=/donations/membership.php&Q&src=ms2&E&"><div id="message-inner-md">Is it important to you that the world have accurate information about drugs?</div> <div id="message-inner-lg"><u>Please donate</u> to support Erowid Center's vision!</div></a> <script src="/includes/javascript/erowid_message_lib.js" type="text/javascript"></script> <script type="text/javascript"> var mTimeExpired = 1732553965000; var mDate = new Date(); if (mDate.getDate() < mTimeExpired) { SendQuietRequest("https://www.erowid.org/cgi-bin/messages/message_view_record.php?message_id=2&page_url=/experiences/exp_info2.shtml"); } </script>   </div> </div>   <div id="content-body-frame">  <div class="title-section"> <div class="ts-title-pre">A brief overview of the</div> <div class="ts-title">Experience Report Reviewing Process</div> <div class="ts-title-sub"></div> <div class="ts-author">by the Erowid Reviewing Crew</div> <div class="ts-ver-date">v 1.1, Jul 2005</div> <div class="ts-space"></div> <div class="ts-title-space"></div> <div class="ts-citation">Citation:   Erowid. "A brief overview of the Erowid Experience Report Reviewing Process 2005" <u>Erowid.org</u>. Feb 2005. erowid.org/experiences/exp_info2.shtml </div> <div class="content-section"> Currently, an experience report's time course from submission to final status takes an average of six months. Some reports are approved or discarded within a week, while there are still reports from 2002 (as of Jul 2005) that have not yet been read once. The new "triaging" system has more than doubled the number of people reading and rating unpublished reports, and continue to add new triagers.<br/><br/> The process for publishing experience reports on Erowid has changed several times since Erowid began publishing experiences in 1995. Anyone interested in this process should read "<a href="exp_info1.shtml">Report Reviewing: The Good, the Bad, and the Ugly</a>", from 2002.<br/><br/> <div class="h8">History</div> The first 'process' was simply that we would read people's descriptions on email lists and in other forums or hear someone describing an experience and ask them to write it up and send it to us. We would edit these reports and add them to the vaults for the topic substance as a static HTML page. We focused our time on reports that captured particular ideas or feelings in articulate ways. We also tended to choose reports that included clear dosage and timing information.<br/><br/> In 1999, we decided to change the direction and create the Erowid Experience Vaults. A web form shuttled submissions into a list of unreviewed reports that could be viewed by one of the handful of Erowid crew who had access. Any crew member could edit, delete, or publish a report, although Fire and Earth retained control of the substance lists. The stated goal was that each report received at least two readers before it was discarded or considered "final", but over a third of the published reports never saw a second reviewer. The reasons for this are many and varied, but the primary ones are that a substantial portion of reports submitted are tiresome, poorly written, and a bit depressing. Reading, categorizing, and editing reports is harder work than it first seems like it would be. Most volunteer reviewers simply quit because it was such slow going: the process was more like hunting for needles in a haystack than picking fruit off a full tree. The other major reason is that the web-based system was overly cumbersome and it took a lot of time to train new reviewers.<br/><br/> Changes were made over time, but the original system stayed mostly unchanged over the next 5 years.<br/><br/> In 2004, we launched a beta of an updated system designed to increase throughput of incoming reports and make it easier to train people as volunteer reviewers. The new part of the system is called Triaging ("assigning of priority order on the basis of where resources can be best used or are most needed") and involves a much simpler interface for grading reports as they come in. The goal for this design is to have 3 triagers read each report before a reviewer gets the report to check. With this model, better reports naturally get reviewed faster. As triagers who become comfortable with the system and the editorial process show themselves to be willing to put time in, they can become reviewers.<br/><br/> In January 2005, the second stage of the beta triage system was launched along with a major update to the existing review process. A third release is planned before it is no longer a beta and we have enough people to bootstrap into opening the process further. We believe that some time during 2006, we may be able to keep up with the incoming reports.<br/><br/> <div class="h8">The Process</div> The Erowid Experience Review Process (v3) is as follows: <ul> <li>Submission: Report is submitted by visitor.</li> <li>Submitted List: Each submitted report appears in the Submitted Reports List, viewable by any Erowid Review Crew and any experienced Triager.</li> <li>Triage: Report is read by at least 2 Triagers, who give the report a Rating (grade) from A+ to F. Ratings are based on a variety of factors, including novelty, clarity of writing, interest, and 'data value'. Reports believed to be falsified are given F's.</li> <li>Triaged List: After a report is triaged by 2 Triagers, it enters the Triaged Reports List where it is sorted by average Triage rating.</li> <li>Trashed: After being Triaged, some reports are Trashed by a Reviewer. The percentage of these varies by substance, from 1 in 10 being Trashed to 5 in 10 being Trashed. See <a href="exp_info1.shtml">"Report Reviewing: The Good, the Bad, and the Ugly</a> for more information about this.</li> <li>Edited & Approved: After being Triaged, some reports are Edited and Approved by a Reviewer. Erowid has an editorial policy to remove a number of things from reports before they are published live to the public. These include such things as full names, addresses, email addresses, names of vendors, overly promotional language, certain uses of the second person ("you will feel..."). Reports that are approved go into the New Reports list immediately.</li> </ul> As of Jul 2005, reports of varying quality are still published live to the site. Visitors can expect a gradual increase in quality reports as the triage system evolves. [Erowid Note: This is the original text of the 2005 article, reports have been continuously submitted and reviewed through Oct 2016 and probably continue today. ] </div>  </div></div> <div id="footer-frame"> <div class="footer-backto"> [ <a href="/experiences/">back to experiences</a> ]</div> <table border="0" cellpadding="0" cellspacing="0" width="100%" id="footer-bar"><tr> <td width="10" background="/includes/headers/images/bar_solid_footer.gif"> </td> <td width="140" background="/includes/headers/images/bar_solid_footer.gif" valign="top"> <table border="0" cellpadding="2" cellspacing="0" width="140"><tr> <td valign="top" id="footer-menu"><script language="JavaScript">PlaceMenu("footer")</script></td> </tr></table> </td> <td align="center"> <table border="0" cellpadding="2" cellspacing="0" width="100%" background="/includes/headers/images/bar_solid_footer.gif" height="22"><tr> <td align="center"><div class="footer1">Created by Erowid - Feb 15, 2005</div></td> </tr></table> </td> <td width="10" background="/includes/headers/images/bar_solid_footer.gif"> </td> <td width="140" background="/includes/headers/images/bar_solid_footer.gif" align="right"><div class="footer2">Modified - Oct 25, 2016</div></td> </tr></table> <div class="footer-copyright">Content & design © 1995-2023 Erowid.org. Permission required before publicly reproducing Erowid info.</div> </div> </div>  </body> </html>