Code Generation | Papers With Code

<!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> <script> const GTAG_ENABLED = true ; const GTAG_TRACKING_ID = "UA-121182717-1"; const SENTRY_DSN_FRONTEND = "".trim(); const GLOBAL_CSRF_TOKEN = 'APEgsZlD47daCFfZJ4XtPwqCVrKgNhB76dVYju6ALVxIO6n2LUZV8aFQFMcz4Wq6'; const MEDIA_URL = "https://production-media.paperswithcode.com/"; const ASSETS_URL = "https://production-assets.paperswithcode.com"; run_after_frontend_loaded = window.run_after_frontend_loaded || []; </script> <link rel="preconnect" href="https://production-assets.paperswithcode.com"><link rel="dns-prefetch" href="https://production-assets.paperswithcode.com"><link rel="preload" as="font" type="font/woff2" href="https://production-assets.paperswithcode.com/perf/fonts/65e877e527022735c1a1.woff2" crossorigin><link rel="preload" as="font" type="font/woff2" href="https://production-assets.paperswithcode.com/perf/fonts/917632e36982ca7933c8.woff2" crossorigin><link rel="preload" as="font" type="font/woff2" href="https://production-assets.paperswithcode.com/perf/fonts/f1405bd8a987c2ea8a67.woff2" crossorigin><script>(()=>{if(GTAG_ENABLED){const t=document.createElement("script");function n(){window.dataLayer.push(arguments)}t.src=`https://www.googletagmanager.com/gtag/js?id=${GTAG_TRACKING_ID}`,document.head.appendChild(t),window.dataLayer=window.dataLayer||[],window.gtag=n,n("js",new Date),n("config",GTAG_TRACKING_ID),window.captureOutboundLink=function(t){n("event","click",{event_category:"outbound",event_label:t})}}else window.captureOutboundLink=function(n){document.location=n}})();</script><link rel="preload" as="script" href="https://production-assets.paperswithcode.com/perf/766.4af6b88b.js"><link rel="preload" as="script" href="https://production-assets.paperswithcode.com/perf/2.6da00df7.js"><link rel="preload" as="script" href="https://production-assets.paperswithcode.com/perf/351.a22a9607.js"><link rel="preload" as="script" href="https://production-assets.paperswithcode.com/perf/101.5f271f23.js"><link rel="preload" as="style" href="https://production-assets.paperswithcode.com/perf/918.c41196c3.css"><link rel="preload" as="style" href="https://production-assets.paperswithcode.com/perf/view_task.8e3945a3.css"><link rel="preload" as="script" href="https://production-assets.paperswithcode.com/perf/view_task.e61ab167.js"><link rel="stylesheet" href="https://production-assets.paperswithcode.com/perf/918.c41196c3.css"><link rel="stylesheet" href="https://production-assets.paperswithcode.com/perf/view_task.8e3945a3.css">  <title>Code Generation | Papers With Code</title> <meta name="description" content="**Code Generation** is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity. <span class="description-source">Source: [Deep Learning for Source Code Modeling and Generation ](https://arxiv.org/abs/2002.05442)</span> Image source: [Measuring Coding Challenge Competence With APPS](https://paperswithcode.com/paper/measuring-coding-challenge-competence-with)" />  <meta property="og:title" content="Papers with Code - Code Generation"> <meta property="og:description" content="**Code Generation** is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity. <span class="description-source">Source: [Deep Learning for Source Code Modeling and Generation ](https://arxiv.org/abs/2002.05442)</span> Image source: [Measuring Coding Challenge Competence With APPS](https://paperswithcode.com/paper/measuring-coding-challenge-competence-with)"> <meta property="og:image" content="https://production-media.paperswithcode.com/tasks/code_generation_0P9CcWa.png"> <meta property="og:url" content="https://paperswithcode.com/task/code-generation">  <meta name="twitter:card" content="summary_large_image"> <meta name="twitter:site" content="@paperswithcode"> <meta name="twitter:title" content="Papers with Code - Code Generation"> <meta name="twitter:description" content="**Code Generation** is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity. <span class="description-source">Source: [Deep Learning for Source Code Modeling and Generation ](https://arxiv.org/abs/2002.05442)</span> Image source: [Measuring Coding Challenge Competence With APPS](https://paperswithcode.com/paper/measuring-coding-challenge-competence-with)"> <meta name="twitter:creator" content="@paperswithcode"> <meta name="twitter:url" content="https://paperswithcode.com/task/code-generation"> <meta name="twitter:domain" content="paperswithcode.com">  <script type="application/ld+json">{ "@context": "http://schema.org", "@graph": { "@type": "CreativeWork", "@id": "code-generation", "name": "Code Generation", "description": "**Code Generation** is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.\r\n\r\n\r\n\u003Cspan class=\"description-source\"\u003ESource: [Deep Learning for Source Code Modeling and Generation ](https://arxiv.org/abs/2002.05442)\u003C/span\u003E\r\n\r\nImage source: [Measuring Coding Challenge Competence With APPS](https://paperswithcode.com/paper/measuring-coding-challenge-competence-with)", "url": "https://paperswithcode.com/task/code-generation", "image": "https://production-media.paperswithcode.com/tasks/code_generation_0P9CcWa.png", "subjectOf": [ { "@type": "CreativeWork", "@id": "computer-code", "name": "Computer Code", "description": "Browse 66 tasks \u2022 437 datasets \u2022 237 ", "image": "https://paperswithcode.com/static/sota.jpeg", "headline": "Browse state-of-the-art in ML leaderboards \u2022 15887 papers with code." } ], "headline": "Code Generation" } }</script> <meta name="theme-color" content="#fff"/> <link rel="manifest" href="https://production-assets.paperswithcode.com/static/manifest.web.json"> </head> <body> <nav class="navbar navbar-expand-lg navbar-light header"> <a class="navbar-brand" href="/"> <span class=" icon-wrapper" data-name="pwc"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path d="M88 128h48v256H88zm144 0h48v256h-48zm-72 16h48v224h-48zm144 0h48v224h-48zm72-16h48v256h-48z"/><path d="M104 104V56H16v400h88v-48H64V104zm304-48v48h40v304h-40v48h88V56z"/></svg></span> </a> <div class="navbar-mobile-twitter d-lg-none"> <a rel="noreferrer" href="https://twitter.com/paperswithcode"> <span class=" icon-wrapper icon-fa icon-fa-brands" data-name="twitter"><svg viewBox="0 0 512.001 515.25" xmlns="http://www.w3.org/2000/svg"><path d="M459.37 152.016c.326 4.548.326 9.097.326 13.645 0 138.72-105.583 298.558-298.559 298.558C101.685 464.22 46.457 447 0 417.114c8.447.973 16.568 1.298 25.34 1.298 49.054 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.113-72.772 6.499.975 12.996 1.624 19.819 1.624 9.42 0 18.843-1.3 27.613-3.573-48.08-9.747-84.142-51.98-84.142-102.984v-1.3c13.968 7.798 30.213 12.67 47.43 13.32-28.263-18.843-46.78-51.006-46.78-87.391 0-19.492 5.196-37.36 14.294-52.954 51.654 63.674 129.3 105.258 216.364 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.827 46.782-104.934 104.934-104.934 30.214 0 57.502 12.67 76.671 33.136 23.715-4.548 46.455-13.319 66.599-25.34-7.798 24.367-24.366 44.834-46.132 57.828 21.117-2.274 41.584-8.122 60.426-16.244-14.292 20.791-32.161 39.309-52.628 54.253z"/></svg></span> </a> </div> <button class="navbar-toggler" type="button" data-toggle="collapse" data-bs-toggle="collapse" data-target="#top-menu" data-bs-target="#top-menu" aria-controls="top-menu" aria-expanded="false" aria-label="Toggle navigation" > <span class="navbar-toggler-icon"></span> </button> <div class="collapse navbar-collapse" id="top-menu"> <ul class="navbar-nav mr-auto navbar-nav__left light-header"> <li class="nav-item header-search"> <form action="/search" method="get" id="id_global_search_form" autocomplete="off"> <input type="text" name="q_meta" style="display:none" id="q_meta" /> <input type="hidden" name="q_type" id="q_type" /> <input id="id_global_search_input" autocomplete="off" value="" name='q' class="global-search" type="search" placeholder='Search'/> <button type="submit" class="icon"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="search"><svg viewBox="0 0 512.025 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M508.5 482.6c4.7 4.7 4.7 12.3 0 17l-9.9 9.9c-4.7 4.7-12.3 4.7-17 0l-129-129c-2.2-2.3-3.5-5.3-3.5-8.5v-10.2C312 396 262.5 417 208 417 93.1 417 0 323.9 0 209S93.1 1 208 1s208 93.1 208 208c0 54.5-21 104-55.3 141.1H371c3.2 0 6.2 1.2 8.5 3.5zM208 385c97.3 0 176-78.7 176-176S305.3 33 208 33 32 111.7 32 209s78.7 176 176 176z"/></svg></span></button> </form> </li> <li class="nav-item"> <a class="nav-link" href="/sota"> Browse State-of-the-Art </a> </li> <li class="nav-item"> <a class="nav-link" href="/datasets"> Datasets </a> </li> <li class="nav-item"> <a class="nav-link" href="/methods">Methods</a> </li> <li class="nav-item dropdown"> <a class="nav-link dropdown-toggle" role="button" id="navbarDropdownRepro" data-toggle="dropdown" data-bs-toggle="dropdown" aria-haspopup="true" aria-expanded="false" > More </a> <div class="dropdown-menu" aria-labelledby="navbarDropdownRepro"> <a class="dropdown-item" href="/newsletter">Newsletter</a> <a class="dropdown-item" href="/rc2022">RC2022</a> <div class="dropdown-divider"></div> <a class="dropdown-item" href="/about">About</a> <a class="dropdown-item" href="/trends">Trends</a> <a class="dropdown-item" href="https://portal.paperswithcode.com/"> Portals </a> <a class="dropdown-item" href="/libraries"> Libraries </a> </div> </li> </ul> <ul class="navbar-nav ml-auto navbar-nav__right navbar-subscribe justify-content-center align-items-center"> <li class="nav-item"> <a class="nav-link" rel="noreferrer" href="https://twitter.com/paperswithcode"> <span class="nav-link-social-icon icon-wrapper icon-fa icon-fa-brands" data-name="twitter"><svg viewBox="0 0 512.001 515.25" xmlns="http://www.w3.org/2000/svg"><path d="M459.37 152.016c.326 4.548.326 9.097.326 13.645 0 138.72-105.583 298.558-298.559 298.558C101.685 464.22 46.457 447 0 417.114c8.447.973 16.568 1.298 25.34 1.298 49.054 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.113-72.772 6.499.975 12.996 1.624 19.819 1.624 9.42 0 18.843-1.3 27.613-3.573-48.08-9.747-84.142-51.98-84.142-102.984v-1.3c13.968 7.798 30.213 12.67 47.43 13.32-28.263-18.843-46.78-51.006-46.78-87.391 0-19.492 5.196-37.36 14.294-52.954 51.654 63.674 129.3 105.258 216.364 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.827 46.782-104.934 104.934-104.934 30.214 0 57.502 12.67 76.671 33.136 23.715-4.548 46.455-13.319 66.599-25.34-7.798 24.367-24.366 44.834-46.132 57.828 21.117-2.274 41.584-8.122 60.426-16.244-14.292 20.791-32.161 39.309-52.628 54.253z"/></svg></span> </a> </li> <li class="nav-item"> <a id="signin-link" class="nav-link" href="/accounts/login?next=/task/code-generation">Sign In</a> </li> </ul> </div> </nav>  <div class="modal fade" id="emailModal" tabindex="-1" role="dialog" aria-labelledby="emailModalLabel" aria-hidden="true"> <div class="modal-dialog" role="document"> <div class="modal-content"> <div class="modal-header"> <h3 class="modal-title" id="emailModalLabel">Subscribe to the PwC Newsletter</h3> <button type="button" class="close" data-dismiss="modal" data-bs-dismiss="modal" aria-label="Close"> <span aria-hidden="true">×</span> </button> </div> <form action="" method="post"> <div class="modal-body"> <div class="modal-body-info-text"> Stay informed on the latest trending ML papers with code, research developments, libraries, methods, and datasets.<br/><br/> <a href="/newsletter">Read previous issues</a> </div> <input type="hidden" name="csrfmiddlewaretoken" value="APEgsZlD47daCFfZJ4XtPwqCVrKgNhB76dVYju6ALVxIO6n2LUZV8aFQFMcz4Wq6"> <input placeholder="Enter your email" type="email" class="form-control pwc-email" name="address" id="id_address" max_length="100" required> </div> <div class="modal-footer"> <button type="submit" class="btn btn-primary">Subscribe</button> </div> </form> </div> </div> </div>  <div class="modal fade" id="loginModal" tabindex="-1" role="dialog" aria-labelledby="loginModalLabel" aria-hidden="true"> <div class="modal-dialog" role="document"> <div class="modal-content"> <div class="modal-header"> <h5 class="modal-title" id="loginModalLabel">Join the community</h5> <button type="button" class="close btn-close" data-dismiss="modal" data-bs-dismiss="modal" aria-label="Close"> <span aria-hidden="true">×</span> </button> </div> <div class="login-modal-message"> You need to <a href="/accounts/login?next=/task/code-generation">log in</a> to edit.<br/> You can <a href="/accounts/register?next=/task/code-generation">create a new account</a> if you don't have one.<br/><br/> </div> </div> </div> </div>   <div class="modal fade" id="editTask" role="dialog" aria-labelledby="editTaskLabel" aria-hidden="true"> <div class="modal-dialog modal-lg" role="document"> <div class="modal-content"> <div class="modal-header"> <h5 class="modal-title" id="editTaskLabel">Edit task</h5> <button type="button" class="close btn-close" data-bs-dismiss="modal" aria-label="Close"> <span aria-hidden="true">×</span> </button> </div> <div class="modal-body"> <form action="" method="post" enctype="multipart/form-data"> <input type="hidden" name="csrfmiddlewaretoken" value="EyUOBBSDUTorlti5fcl4iBu7oJtjWa5i4wjKyru2rl8HSMRzAxlSeR6xVMPu6Jle"> <div id="div_id_task_name" class="form-group"> <label for="id_task_name" class="col-form-label requiredField"> Task name:<span class="asteriskField">*</span> </label> <div class=""> <input type="text" name="task_name" value="Code Generation" maxlength="200" class="textinput textInput form-control" required="" id="id_task_name" readonly > </div> </div> <div id="div_id_task_area" class="form-group"> <label for="id_task_area" class=" requiredField"> Top-level area:<span class="asteriskField">*</span> </label> <div class=""> <select name="task_area" class="select form-control" required id="id_task_area"> <option value="">---------</option> <option value="17">Adversarial</option> <option value="18">Audio</option> <option value="11" selected>Computer Code</option> <option value="3">Computer Vision</option> <option value="9">Graphs</option> <option value="15">Knowledge Base</option> <option value="7">Medical</option> <option value="6">Methodology</option> <option value="5">Miscellaneous</option> <option value="12">Music</option> <option value="4">Natural Language Processing</option> <option value="13">Playing Games</option> <option value="14">Reasoning</option> <option value="16">Robots</option> <option value="10">Speech</option> <option value="8">Time Series</option> </select> </div> </div> <div id="div_id_task_parent" class="form-group"> <label for="id_task_parent" class=""> Parent task (if any): </label> <div class=""> <select name="task_parent" class="modelselect2 form-control" id="id_task_parent" data-autocomplete-light-language="en" data-autocomplete-light-url="/tag-autocomplete/" data-autocomplete-light-function="select2"> <option value="" selected>---------</option> </select> </div> </div> <div id="div_id_description" class="form-group"> <label for="id_description" class=""> Description with markdown (optional): </label> <div class=""> <textarea name="description" cols="40" rows="3" class="textarea form-control" id="id_description"> **Code Generation** is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity. <span class="description-source">Source: [Deep Learning for Source Code Modeling and Generation ](https://arxiv.org/abs/2002.05442)</span> Image source: [Measuring Coding Challenge Competence With APPS](https://paperswithcode.com/paper/measuring-coding-challenge-competence-with)</textarea> </div> </div> <div id="div_id_image" class="form-group"> <label for="id_image" class=""> Image </label> <div class=""> Currently: <a href="https://production-media.paperswithcode.com/tasks/code_generation_0P9CcWa.png">tasks/code_generation_0P9CcWa.png</a> <input type="checkbox" name="image-clear" id="image-clear_id"> <label for="image-clear_id">Clear</label><br> Change: <input type="file" name="image" accept="image/*" class="clearablefileinput form-control-file" id="id_image"> </div> </div> <div class="modal-footer"> <button type="submit" class="btn btn-primary"> Submit </button> </div> </form> </div> </div> </div> </div>  <div class="modal fade" id="addRow" role="dialog" aria-labelledby="addRowLabel" aria-hidden="true"> <div class="modal-dialog" role="document"> <div class="modal-content"> <div class="modal-header"> <h5 class="modal-title" id="addRowLabel">Add a new evaluation result row</h5> <button type="button" class="close btn-close" data-bs-dismiss="modal" aria-label="Close"> <span aria-hidden="true">×</span> </button> </div> <form action="" method="post"> <div class="modal-body"> <input type="hidden" name="csrfmiddlewaretoken" value="EyUOBBSDUTorlti5fcl4iBu7oJtjWa5i4wjKyru2rl8HSMRzAxlSeR6xVMPu6Jle"> <input id="id_task" disabled="disabled" type="hidden" value="211"/> <div id="div_id_paper" class="form-group"> <label for="id_paper" class=" requiredField"> Paper title:<span class="asteriskField">*</span> </label> <div class=""> <select name="paper" class="modelselect2 form-control" required id="id_paper" data-autocomplete-light-language="en" data-autocomplete-light-url="/paper-autocomplete/" data-autocomplete-light-function="select2"> <option value="" selected>---------</option> </select> </div> </div> <div id="div_id_dataset" class="form-group"> <label for="id_dataset" class=" requiredField"> Dataset:<span class="asteriskField">*</span> </label> <div class=""> <select name="dataset" class="modelselect2 form-control" required id="id_dataset" data-autocomplete-light-language="en" data-autocomplete-light-url="/dataset-autocomplete/" data-autocomplete-light-function="select2"> <option value="" selected>---------</option> </select> </div> </div> <div id="div_id_model_name" class="form-group"> <label for="id_model_name" class=" requiredField"> Model name:<span class="asteriskField">*</span> </label> <div class=""> <input type="text" name="model_name" class="textinput textInput form-control" required id="id_model_name"> </div> </div> <div id="div_id_metric" class="form-group"> <label for="id_metric" class=" requiredField"> Metric name:<span class="asteriskField">*</span> </label> <div class=""> <select name="metric" class="modelselect2 form-control" required id="id_metric" data-autocomplete-light-language="en" data-autocomplete-light-url="/metric-autocomplete/" data-autocomplete-light-function="select2"> <option value="" selected>---------</option> </select> </div> </div> <div id="sota-metric-names"> </div> <div class="form-group"> <div id="div_id_metric_higher_is_better" class="form-check"> <input type="checkbox" name="metric_higher_is_better" class="checkboxinput form-check-input" id="id_metric_higher_is_better"> <label for="id_metric_higher_is_better" class="form-check-label"> Higher is better (for the metric) </label> </div> </div> <div id="div_id_metric_value" class="form-group"> <label for="id_metric_value" class=" requiredField"> Metric value:<span class="asteriskField">*</span> </label> <div class=""> <input type="text" name="metric_value" class="textinput textInput form-control" required id="id_metric_value"> </div> </div> <div id="sota-metric-values"> </div> <div class="form-group"> <div id="div_id_uses_additional_data" class="form-check"> <input type="checkbox" name="uses_additional_data" class="checkboxinput form-check-input" id="id_uses_additional_data"> <label for="id_uses_additional_data" class="form-check-label"> Uses extra training data </label> </div> </div> <div id="div_id_evaluated_on" class="form-group"> <label for="id_evaluated_on" class=""> Data evaluated on </label> <div class=""> <input type="text" name="evaluated_on" autocomplete="off" class="dateinput form-control" id="id_evaluated_on"> </div> </div> </div> <div class="modal-footer"> <button type="submit" class="btn btn-primary">Submit </button> </div> </form> </div> </div> </div> </div> <div class="container content content-buffer "> <main> <div class="row task-content" style="margin-top: 3rem;">  <div class="dataset-header"> <a href="/area/computer-code"> <span class="badge badge-primary"> <span class=" icon-wrapper icon-fa icon-fa-solid" data-name="images"><svg viewBox="0 0 576 514.999" xmlns="http://www.w3.org/2000/svg"><path d="M480 417.998v16c0 26.51-21.49 48-48 48H48c-26.51 0-48-21.49-48-48v-256c0-26.51 21.49-48 48-48h16v208c0 44.113 35.888 80 80 80h336zm96-80c0 26.51-21.49 48-48 48H144c-26.51 0-48-21.49-48-48v-256c0-26.51 21.49-48 48-48h384c26.51 0 48 21.49 48 48v256zm-320-208c0-26.51-21.49-48-48-48s-48 21.49-48 48 21.49 48 48 48 48-21.49 48-48zm-96 144v48h352v-112l-87.514-87.514c-4.687-4.687-12.285-4.687-16.971 0L272 257.999l-39.514-39.515c-4.688-4.686-12.285-4.686-16.972 0z"/></svg></span> <span>Computer Code</span> </span> </a> </div> <div class="artefact-header"> <div class="float-right task-edit"> <div class="dropdown edit-button"> <a data-bs-toggle="modal" data-bs-target="#loginModal"> <span class="badge badge-method-edit" style="padding-top:10px;"><span class=" icon-wrapper icon-fa icon-fa-solid" data-name="edit"><svg viewBox="0 0 576 514.999" xmlns="http://www.w3.org/2000/svg"><path d="M402.6 85.198l90.2 90.2c3.8 3.8 3.8 10 0 13.8l-218.399 218.4-92.8 10.3c-12.4 1.4-22.9-9.1-21.5-21.5l10.3-92.8 218.4-218.4c3.799-3.8 10-3.8 13.799 0zm162-22.9c15.2 15.2 15.2 39.9 0 55.2l-35.4 35.4c-3.8 3.8-10 3.8-13.8 0l-90.2-90.2c-3.8-3.8-3.8-10 0-13.8l35.4-35.4c15.3-15.2 40-15.2 55.2 0zM384 348.198c0-3.2 1.3-6.2 3.5-8.5l40-40c7.6-7.5 20.5-2.2 20.5 8.5v157.8c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48v-352c0-26.5 21.5-48 48-48h285.8c10.7 0 16.1 12.9 8.5 20.5l-40 40c-2.3 2.2-5.3 3.5-8.5 3.5H64v320h320v-101.8z"/></svg></span> Edit</span> </a> </div> </div> <h1 id="task-home">Code Generation</h1> <div class="artefact-information"> <p> 516 papers with code • 22 benchmarks • 53 datasets </p> </div> </div> <div class="col-lg-9">  <div class="description"> <div class="description-content"> <p><strong>Code Generation</strong> is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.</p> <p><span class="description-source">Source: <a href="https://arxiv.org/abs/2002.05442">Deep Learning for Source Code Modeling and Generation </a></span></p> <p>Image source: <a href="https://paperswithcode.com/paper/measuring-coding-challenge-competence-with">Measuring Coding Challenge Competence With APPS</a></p> </div> </div>  <div class="image-container task-image-mobile"> <a href="https://production-media.paperswithcode.com/thumbnails/task/658fcf6a-52ca-4865-b111-28bf61c346dc.jpg" data-lightbox="imageresource"> <img id="imageresource" width=100% src="https://production-media.paperswithcode.com/thumbnails/task/658fcf6a-52ca-4865-b111-28bf61c346dc.jpg"> </a> </div>  <div class="task-benchmarks"> <div id="benchmarks" class="collapsed"> <div class="title"> <h2 id="benchmarks">Benchmarks <div class="float-right"> <div class="dropdown edit-button task-add-a-result"> <a data-bs-toggle="modal" data-bs-target="#loginModal"> <span class="badge badge-primary" style="font-size:12px;"> Add a Result</span> </a> </div> </div> </h2> These leaderboards are used to track progress in Code Generation <hr> </div> <div class="sota-table-preview table-responsive"> <table id="benchmarksTable" class="table-striped table-responsive"> <thead> <tr> <th>Trend</th> <th style="padding-left:12px;">Dataset</th> <th style="min-width:200px">Best Model</th>  <th class="text-center">Paper</th> <th class="text-center">Code</th> <th class="text-center">Compare</th> </tr> </thead> <tbody> <tr onclick="window.location='/sota/code-generation-on-mbpp';"> <td> <a href="/sota/code-generation-on-mbpp"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-mbpp-small_555d4043.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-mbpp"> MBPP </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-mbpp"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> o1-mini + MapCoder (Hamming.ai) </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/mapcoder-multi-agent-code-generation-for"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/mapcoder-multi-agent-code-generation-for#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-mbpp" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-apps';"> <td> <a href="/sota/code-generation-on-apps"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-apps-small_e189f681.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-apps"> APPS </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-apps"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> code-davinci-002 175B (CodeT) </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/codet-code-generation-with-generated-tests"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/codet-code-generation-with-generated-tests#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-apps" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-conala';"> <td> <a href="/sota/code-generation-on-conala"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-conala-small_a443548f.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-conala"> CoNaLa </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-conala"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> PanGu-Coder-FT-I </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/fine-tuning-large-language-models-for"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-conala" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-django';"> <td> <a href="/sota/code-generation-on-django"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-django-small_cbc4f120.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-django"> Django </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-django"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> MarianCG </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/mariancg-a-code-generation-transformer-model"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/mariancg-a-code-generation-transformer-model#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-django" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-wikisql';"> <td> <a href="/sota/code-generation-on-wikisql"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-wikisql-small_e61da454.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-wikisql"> WikiSQL </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-wikisql"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> NL2SQL-RULE </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/content-enhanced-bert-based-text-to-sql"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/content-enhanced-bert-based-text-to-sql#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-wikisql" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-res-q';"> <td> <a href="/sota/code-generation-on-res-q"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-res-q-small_4ec89f20.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-res-q"> RES-Q </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-res-q"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> QurrentOS-coder + Claude 3.5 Sonnet </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/res-q-evaluating-code-editing-large-language"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/res-q-evaluating-code-editing-large-language#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-res-q" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-pecc';"> <td> <a href="/sota/code-generation-on-pecc"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-pecc-small_46cc92e8.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-pecc"> PECC </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-pecc"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> Claude 3 Haiku </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/pecc-problem-extraction-and-coding-challenges"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/pecc-problem-extraction-and-coding-challenges#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-pecc" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-webapp1k-react';"> <td> <a href="/sota/code-generation-on-webapp1k-react"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-webapp1k-react-small_ba80acfd.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-webapp1k-react"> WebApp1K-React </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-webapp1k-react"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> o1-preview </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/a-case-study-of-web-app-coding-with-openai"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/a-case-study-of-web-app-coding-with-openai#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-webapp1k-react" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-humaneval';"> <td> <a href="/sota/code-generation-on-humaneval"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-humaneval-small_d01cfecb.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-humaneval"> HumanEval </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-humaneval"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> LDB (GPT4o) </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/ldb-a-large-language-model-debugger-via"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/ldb-a-large-language-model-debugger-via#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-humaneval" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-conala-ext';"> <td> <a href="/sota/code-generation-on-conala-ext"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-conala-ext-small_e2d0fe69.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-conala-ext"> CoNaLa-Ext </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-conala-ext"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> BART Base </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/reading-stackoverflow-encourages-cheating"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/reading-stackoverflow-encourages-cheating#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-conala-ext" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-webapp1k-duo-react';"> <td> <a href="/sota/code-generation-on-webapp1k-duo-react"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-webapp1k-duo-react-small_0389808e.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-webapp1k-duo-react"> WebApp1k-Duo-React </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-webapp1k-duo-react"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> claude-3-5-sonnet </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/a-case-study-of-web-app-coding-with-openai"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/a-case-study-of-web-app-coding-with-openai#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-webapp1k-duo-react" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-turbulence';"> <td> <a href="/sota/code-generation-on-turbulence"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-turbulence-small_2f7a2254.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-turbulence"> Turbulence </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-turbulence"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> GPT-4 </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/turbulence-systematically-and-automatically"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/turbulence-systematically-and-automatically#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-turbulence" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-codecontests';"> <td> <a href="/sota/code-generation-on-codecontests"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-codecontests-small_2a2bea36.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-codecontests"> CodeContests </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-codecontests"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> MapCoder (GPT-4) </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/mapcoder-multi-agent-code-generation-for"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/mapcoder-multi-agent-code-generation-for#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-codecontests" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-shellcode-ia32';"> <td> <a href="/sota/code-generation-on-shellcode-ia32"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-shellcode-ia32-small_3e77d86a.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-shellcode-ia32"> Shellcode_IA32 </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-shellcode-ia32"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> CodeBERT </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/can-we-generate-shellcodes-via-natural"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/can-we-generate-shellcodes-via-natural#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-shellcode-ia32" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-taco-code';"> <td> <a href="/sota/code-generation-on-taco-code"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-taco-code-small_790baaaa.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-taco-code"> TACO-Code </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-taco-code"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> GPT-4 </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/taco-topics-in-algorithmic-code-generation"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/taco-topics-in-algorithmic-code-generation#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-taco-code" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-codexglue-codesearchnet';"> <td> <a href="/sota/code-generation-on-codexglue-codesearchnet"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-codexglue-codesearchnet-small_b361a07b.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-codexglue-codesearchnet"> CodeXGLUE - CodeSearchNet </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-codexglue-codesearchnet"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> Redcoder-ext </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/retrieval-augmented-code-generation-and"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/retrieval-augmented-code-generation-and#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-codexglue-codesearchnet" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-concode';"> <td> <a href="/sota/code-generation-on-concode"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-concode-small_c0eef710.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-concode"> CONCODE </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-concode"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> Redcoder-ext </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/retrieval-augmented-code-generation-and"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/retrieval-augmented-code-generation-and#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-concode" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-verified-smart-contract';"> <td> <a href="/sota/code-generation-on-verified-smart-contract"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-verified-smart-contract-small_b369890a.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-verified-smart-contract"> Verified Smart Contract Code Comments </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-verified-smart-contract"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> GPT-J 6B Smart Contract </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/efficient-avoidance-of-vulnerabilities-in"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-verified-smart-contract" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-bigcodebench-complete';"> <td> <a href="/sota/code-generation-on-bigcodebench-complete"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-bigcodebench-complete-small_3ec2455a.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-bigcodebench-complete"> BigCodeBench-Complete </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-bigcodebench-complete"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> GPT-4o-2024-05-13 </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/bigcodebench-benchmarking-code-generation"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/bigcodebench-benchmarking-code-generation#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-bigcodebench-complete" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-bigcodebench-instruct';"> <td> <a href="/sota/code-generation-on-bigcodebench-instruct"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-bigcodebench-instruct-small_ecd71ba0.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-bigcodebench-instruct"> BigCodeBench-Instruct </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-bigcodebench-instruct"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> GPT-4o-2024-05-13 </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/bigcodebench-benchmarking-code-generation"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/bigcodebench-benchmarking-code-generation#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-bigcodebench-instruct" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-android-repos';"> <td> <a href="/sota/code-generation-on-android-repos"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-android-repos-small_6a37d224.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-android-repos"> Android Repos </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-android-repos"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> Entity Type Model </a> </div> </td>  <td> <div class="text-center paper"> <a href="/paper/building-language-models-for-text-with-named"> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> </a> </div> </td> <td> <div class="text-center github"><a href="/paper/building-language-models-for-text-with-named#code"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </a></div> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-android-repos" class="btn btn-primary">See all</a> </div> </td> </tr> <tr onclick="window.location='/sota/code-generation-on-dseval-leetcode';"> <td> <a href="/sota/code-generation-on-dseval-leetcode"> <img class="sota-thumb" src="https://production-media.paperswithcode.com/sota-thumbs/code-generation-on-dseval-leetcode-small_edf81f51.png"/> </a> </td> <td> <div class="dataset black-links"> <a href="/sota/code-generation-on-dseval-leetcode"> DSEval-LeetCode </a> </div> </td> <td> <div class="black-links"> <a href="/sota/code-generation-on-dseval-leetcode"> <i class="em em-trophy" style="height:1em;position:relative;top:-2px"></i> Jupyter-AI </a> </div> </td>  <td> </td> <td> </td> <td class="text-center"> <div class="sota-table-link"> <a href="/sota/code-generation-on-dseval-leetcode" class="btn btn-primary">See all</a> </div> </td> </tr> </tbody> </table> <div class="table-options expand"> <a href="javascript:void(0)" onclick="document.getElementById('benchmarks').classList.remove('collapsed')" >Show all 22 benchmarks</a> </div> <div class="table-options collapse"> <a href="javascript:void(0)" onclick="document.getElementById('benchmarks').classList.add('collapsed')" >Collapse benchmarks</a> </div> </div> </div> </div>  <div class="task-started"> <div class="title task-libraries"> <h2 id="task-libraries">Libraries <span class="lib-info" data-bs-toggle="popover" data-bs-placement="top" data-bs-trigger="hover" data-bs-title="Libraries" data-bs-content="These libraries are updated daily, based on the papers assigned to this task. If you think a Library is missing, make sure this library is added as code to the papers it implements, and that the papers have been assigned to this task." ><span class=" icon-wrapper icon-ion" data-name="information-circle-outline"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M248 64C146.39 64 64 146.39 64 248s82.39 184 184 184 184-82.39 184-184S349.61 64 248 64z" fill="none" stroke="#000" stroke-miterlimit="10" stroke-width="32"/><path fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round" stroke-width="32" d="M220 220h32v116"/><path fill="none" stroke="#000" stroke-linecap="round" stroke-miterlimit="10" stroke-width="32" d="M208 340h88"/><path d="M248 130a26 26 0 1 0 26 26 26 26 0 0 0-26-26z"/></svg></span></span> </h2> Use these libraries to find Code Generation models and implementations <hr> <div id="libraries-short-list"> <div class="row task-library"> <div class="col-12 col-md-6"> <a href="https://github.com/geekan/metagpt" onclick="captureOutboundLink('https://github.com/geekan/metagpt'); return true;"> <div class="library-logo"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </div> geekan/metagpt </a> </div> <div class="col-6 col-md-3"> <span class="task-library-pwc-count"> 2 papers </span> </div> <div class="col-6 col-md-3"> <div class="library-stars text-nowrap"> 45,467 <span class=" icon-wrapper icon-ion" data-name="star"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M394 480a16 16 0 0 1-9.39-3L256 383.76 127.39 477a16 16 0 0 1-24.55-18.08L153 310.35 23 221.2a16 16 0 0 1 9-29.2h160.38l48.4-148.95a16 16 0 0 1 30.44 0l48.4 149H480a16 16 0 0 1 9.05 29.2L359 310.35l50.13 148.53A16 16 0 0 1 394 480z"/></svg></span> </div> </div> </div> <div class="row task-library"> <div class="col-12 col-md-6"> <a href="https://github.com/eleutherai/gpt-neox" onclick="captureOutboundLink('https://github.com/eleutherai/gpt-neox'); return true;"> <div class="library-logo"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </div> eleutherai/gpt-neox </a> </div> <div class="col-6 col-md-3"> <span class="task-library-pwc-count"> 2 papers </span> </div> <div class="col-6 col-md-3"> <div class="library-stars text-nowrap"> 6,956 <span class=" icon-wrapper icon-ion" data-name="star"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M394 480a16 16 0 0 1-9.39-3L256 383.76 127.39 477a16 16 0 0 1-24.55-18.08L153 310.35 23 221.2a16 16 0 0 1 9-29.2h160.38l48.4-148.95a16 16 0 0 1 30.44 0l48.4 149H480a16 16 0 0 1 9.05 29.2L359 310.35l50.13 148.53A16 16 0 0 1 394 480z"/></svg></span> </div> </div> </div> <div class="row task-library"> <div class="col-12 col-md-6"> <a href="https://github.com/CGCL-codes/naturalcc" onclick="captureOutboundLink('https://github.com/CGCL-codes/naturalcc'); return true;"> <div class="library-logo"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </div> CGCL-codes/naturalcc </a> </div> <div class="col-6 col-md-3"> <span class="task-library-pwc-count"> 2 papers </span> </div> <div class="col-6 col-md-3"> <div class="library-stars text-nowrap"> 276 <span class=" icon-wrapper icon-ion" data-name="star"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M394 480a16 16 0 0 1-9.39-3L256 383.76 127.39 477a16 16 0 0 1-24.55-18.08L153 310.35 23 221.2a16 16 0 0 1 9-29.2h160.38l48.4-148.95a16 16 0 0 1 30.44 0l48.4 149H480a16 16 0 0 1 9.05 29.2L359 310.35l50.13 148.53A16 16 0 0 1 394 480z"/></svg></span> </div> </div> </div> </div> <div id="libraries-full-list" style="display:none"> <div class="row task-library"> <div class="col-12 col-md-6"> <a href="https://github.com/geekan/metagpt" onclick="captureOutboundLink('https://github.com/geekan/metagpt'); return true;"> <div class="library-logo"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </div> geekan/metagpt </a> </div> <div class="col-6 col-md-3"> <span class="task-library-pwc-count"> 2 papers </span> </div> <div class="col-6 col-md-3"> <div class="library-stars text-nowrap"> 45,467 <span class=" icon-wrapper icon-ion" data-name="star"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M394 480a16 16 0 0 1-9.39-3L256 383.76 127.39 477a16 16 0 0 1-24.55-18.08L153 310.35 23 221.2a16 16 0 0 1 9-29.2h160.38l48.4-148.95a16 16 0 0 1 30.44 0l48.4 149H480a16 16 0 0 1 9.05 29.2L359 310.35l50.13 148.53A16 16 0 0 1 394 480z"/></svg></span> </div> </div> </div> <div class="row task-library"> <div class="col-12 col-md-6"> <a href="https://github.com/eleutherai/gpt-neox" onclick="captureOutboundLink('https://github.com/eleutherai/gpt-neox'); return true;"> <div class="library-logo"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </div> eleutherai/gpt-neox </a> </div> <div class="col-6 col-md-3"> <span class="task-library-pwc-count"> 2 papers </span> </div> <div class="col-6 col-md-3"> <div class="library-stars text-nowrap"> 6,956 <span class=" icon-wrapper icon-ion" data-name="star"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M394 480a16 16 0 0 1-9.39-3L256 383.76 127.39 477a16 16 0 0 1-24.55-18.08L153 310.35 23 221.2a16 16 0 0 1 9-29.2h160.38l48.4-148.95a16 16 0 0 1 30.44 0l48.4 149H480a16 16 0 0 1 9.05 29.2L359 310.35l50.13 148.53A16 16 0 0 1 394 480z"/></svg></span> </div> </div> </div> <div class="row task-library"> <div class="col-12 col-md-6"> <a href="https://github.com/CGCL-codes/naturalcc" onclick="captureOutboundLink('https://github.com/CGCL-codes/naturalcc'); return true;"> <div class="library-logo"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> </div> CGCL-codes/naturalcc </a> </div> <div class="col-6 col-md-3"> <span class="task-library-pwc-count"> 2 papers </span> </div> <div class="col-6 col-md-3"> <div class="library-stars text-nowrap"> 276 <span class=" icon-wrapper icon-ion" data-name="star"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M394 480a16 16 0 0 1-9.39-3L256 383.76 127.39 477a16 16 0 0 1-24.55-18.08L153 310.35 23 221.2a16 16 0 0 1 9-29.2h160.38l48.4-148.95a16 16 0 0 1 30.44 0l48.4 149H480a16 16 0 0 1 9.05 29.2L359 310.35l50.13 148.53A16 16 0 0 1 394 480z"/></svg></span> </div> </div> </div> </div> </div> </div>  <div class="title"> <h2 id="datasets">Datasets</h2> <hr> <div class="task-datasets"> <div class="col-md-12"> <ul class="list-unstyled"> <li> <a href="/dataset/humaneval"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/thumbnails/dataset/d8d4b036-c83e-4f2b-8ebe-ae0d76c25b84.jpg"> HumanEval </span> </a> </li> <li> <a href="/dataset/mbpp"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> MBPP </span> </a> </li> <li> <a href="/dataset/wikisql"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/thumbnails/dataset/dataset-0000000026-9722e05f_G57hZi3.jpg"> WikiSQL </span> </a> </li> <li> <a href="/dataset/codexglue"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/thumbnails/dataset/dataset-0000008543-4cfcb93f.jpg"> CodeXGLUE </span> </a> </li> <li> <a href="/dataset/apps"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/thumbnails/dataset/dataset-0000007444-e42500ff.jpg"> APPS </span> </a> </li> <li> <a href="/dataset/conala"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> CoNaLa </span> </a> </li> <li> <a href="/dataset/codecontests"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> CodeContests </span> </a> </li> <li> <a href="/dataset/ds-1000"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/thumbnails/dataset/0ac731d4-6bae-4b6b-8f04-e87549d9c624.jpg"> DS-1000 </span> </a> </li> <li> <a href="/dataset/concode"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> CONCODE </span> </a> </li> <li> <a href="/dataset/classeval"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> ClassEval </span> </a> </li> <a href="/datasets?task=code-generation"> <button class="dropdown-toggle badge badge-edit w-100 collapsed" type="button" > See all 53 code generation datasets </button> </a> </ul> </div> </div> </div>  <div class="title"> <h2 id="subtasks">Subtasks</h2> <hr> <div class="task-subtasks"> <div class="col-md-12"> <ul class="list-unstyled"> <li> <a href="/task/code-translation"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> <span>Code Translation</span> </span> </a> </li> <li> <a href="/task/code-documentation-generation"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> <span>Code Documentation Generation</span> </span> </a> </li> <li> <a href="/task/class-level-code-generation"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> <span>Class-level Code Generation</span> </span> </a> </li> <li> <a href="/task/library-oriented-code-generation"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> <span>Library-Oriented Code Generation</span> </span> </a> </li> <div class="collapse" id="remaining-tasks"> <li> <a href="/task/library-oriented-code-generation"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> <span>Library-Oriented Code Generation</span></span> </a> </li> <li> <a href="/task/github-issue-resolution"> <span class="badge badge-primary"> <img src="https://production-media.paperswithcode.com/tasks/default.gif"> <span>GitHub issue resolution</span></span> </a> </li> </div> <button class="dropdown-toggle badge badge-edit w-100 collapsed" type="button" data-bs-target="#remaining-tasks" data-bs-toggle="collapse" aria-haspopup="true" aria-expanded="false" onclick="this.innerHTML=this.classList.contains('collapsed')? 'Show all 5 subtasks': 'Show less'" > Show all 5 subtasks </button> </ul> </div> </div> </div>  <div class="title paper-list" id="code"> <h2 id="papers-list" class="home-page-title">Most implemented papers</h2> <div class="paper-filter-btn"> <div class="btn-group" role="group"> <a data-title="Most implemented papers" data-call-url="/tasklist/code-generation/greatest" data-target="/task/code-generation" class="list-papers-button list-button-active" style="margin-right:0">Most implemented</a> <a data-title="Hot papers on social media" data-call-url="/tasklist/code-generation/social" data-target="/task/code-generation/social" class="list-papers-button list-button" style="margin-right:0">Social</a> <a data-title="Latest papers" data-call-url="/tasklist/code-generation/latest" data-target="/task/code-generation/latest" class="list-papers-button list-button" style="margin-right:0">Latest</a> <a data-title="Latest papers with no code" data-call-url="/tasklist/code-generation/codeless" data-target="/task/code-generation/codeless" class="list-papers-button list-button">No code</a> </div> </div> </div>  <input id="paper-list-search" type="search" class="form-control form-control-sm" placeholder="Search for a paper, author or keyword"> <div class="loading-tab" style="display: none"> <div class="loader-ellips"> <span class="loader-ellips__dot"></span> <span class="loader-ellips__dot"></span> <span class="loader-ellips__dot"></span> <span class="loader-ellips__dot"></span> </div> </div> <div id="task-papers-list"> <div class="infinite-container text-center"> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/llama-open-and-efficient-foundation-language-1"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2302.13971.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/llama-open-and-efficient-foundation-language-1">LLaMA: Open and Efficient Foundation Language Models</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/facebookresearch/llama" onclick="captureOutboundLink('https://github.com/facebookresearch/llama'); return true;" style="font-size:13px"> facebookresearch/llama </a> </span> • <span class="item-framework-link"> <img class="" src="https://production-assets.paperswithcode.com/perf/images/frameworks/pytorch-2fbf2cb9.png" /> </span> • <span class="item-conference-link"> <a href="/conference/arxiv-2023-2"> arXiv 2023 </a> </span> </p> <p class="item-strip-abstract">We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 52</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/llama-open-and-efficient-foundation-language-1" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/llama-open-and-efficient-foundation-language-1#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/joint-face-detection-and-alignment-using"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/1604.02878.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/joint-face-detection-and-alignment-using">Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/serengil/deepface" onclick="captureOutboundLink('https://github.com/serengil/deepface'); return true;" style="font-size:13px"> serengil/deepface </a> </span> • <span class="item-framework-link"> <img class="" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAbYAAABiBAMAAADO/+vOAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAMFBMVEVHcExBUGZCUGZKUmNCUGZCUGZCUGb/igD/jwD/hwD/iwBCUGb/kAD/hAD/mwD/eABjkbvFAAAAC3RSTlMAxZ4pVHvkPbaF3mTG43EAAAjMSURBVHja7Zy/b9tGFMdPImlD8CLXyaRFdRIvWhQrCVJosRMnixblh7NwcWNXixYDXRwvCZChQNGm3rq06RhkSGLJNhf9A2khc81iQWtaCJqDAILK+/2LEk+2AsQEr4BDU7w7fu579967d1YBSEpSkpKUpCSFlh/BdlzRbn4AP+/FEy3T+QB++SeWys2+gWwn74uxRDsJ2Nrtv2OHZv3a6RxD3U7iB/dnp0N0a7dfxQtttwML1A3C/RQvtEEwJZFuEK73LD5oO92BpFu7Fxs3lxkOoG4drpv/X0zcXOZdp9uFsh1z3fyPsYCbfTccIjRBt16v/zEOaL8Nu12sG19vvu/3/z3/bH8Nh4PBQLGTgW5+/1UM0LpDIpusW79/zt3c7jAoAdqA2cljplu/f67d3A5EGxDdTlTd+v1z7OYywyGBU/wbLJBNdnOLeslP/6WWlS4ASM/Pz184FdqAognrrY3IgiK6Oaull/3ps1WULgBIBT+PJmzlKZYN+gDZTgZTskfono1nO/ja2QIfIPk3n+sWxfa169btDvGcZHbS96kxiYNuEE6xk34sdBsMBgMpnvTbvX5MdOsOFDvZY1NSct9ruNwNuvLI9cqXYTta4+WMun1+ItvJ9ndhdpIW2FXjyzndijYdzqTbXqYj20mwE2InzyXbc5CR4xI/CDTjwfYKpsvleDL48GUc2F7D6Ksj7QN8aBVfGLLd+XZ+8Tb9pVQqIpN6f3FdNDPXlN+DJ+6sPywqDV9fvFKMZrPX19eKZmyfUGQp7996aEP+1oTNLiBDfZH8Cn8rguU6vHeJcSxJz8CyCm94l8lvaYRzNfhZjWKz7qGauG27Vqs9xvdnarUtfHU3uFlFbJ+2AdVNsJM4lDZgs3LEC20JbA65V1We4WP/SK6F2JyWAZtVIDUXqMf12ExuiMMbsH3Gcb5mJxlcBNtd5mLxbgdirFCWJn7GZc8skFpXlTuQ7dAlrYxl422hgasjDkqEH6mjq4CNbD4VO9kj83knis3h4UOTdXGP3VtRnvFwu7Z6B7I1WwZsaV6zQbtbYUQrVMsGZHtOd3KKnXxNUw4RbK4QG1VpZ7xsqM9saFHVgfzKEWxi81nSUp4NF7sKHn7KMlmqbiwN9HIsGxGgzt9SZjuiUah3ZT3HxJXiUiScKZsj1oRtzdHxctjIYbsEnvAduJLnalO9ZvfGsd2ALf5QBNfr9BZeahdLJWzOSF/eCrUCkGQGTanbpVts+E3ZkODN26VVOuNT9NkZNropNj1E3eT85Og0EGcr0Ks0XV0FZv0qBAX2usmeqZJJiuRCJuWQsXlX1qATHMOGhrBIB3UDywUbAGVmh+eYXRuhW7APGH3awdgsOu6YKUv+xfbBIbgV5gxy+H2UWlTblleNiksc2R4f4baabFE3aPUVTTc5P9n/WIxiS3PTVyYvVOAvRmRymZl28V7W4U4pRfgR22bo/u1AYJvhRuwG3tlBJYW1UARifyPsZBvCbUexzXG/kyaTI8dnBLnkfRH+GcETEwnZlB7PJihqkxo54ta4qc6pwW5Gz08asFV4UsEmk6PA2cg0dXmAAiSJAZc5LUZxo9kKbDLjpZdlM9DmbkGLq/X11h9tTVKi58xyBRoym8t73xSru8IbVrDcaR7EjGOrC8Pk4sVbxreCBrwCGmiLWpcxdtJEt5zQF1lEOtucFDYrtcisNmOzxJkrTPAs+ueogqAcLUGl+TeYL4nUDfb1oIQLWds5jQ3b92+46aoLb5jCUGZsdovFjHhUiOL7iPRgjjWVBRF20o/UTct7heoGcNTC6VqCISMLTWNrzNOSldk8xS/YGD/oayONPp1Rl7dmJ32/F62bxlYMZbtBP77EbSNlI/5AYwv1b47K1mSmIxi/vI3aLasuQMuXoLxyFJttxmbV6ecXGBsQ2FqTsDFrSqsgk2+haY4Uqwj8IHT/htKT26dg09ebGOFuAmXVnJ3NhSwOaiUH+ysILYXaSf+PKeoGcChNw9sp6UbnJJqDKXTtwno51QVodtJ64Ruut3mhjGID9hJhOwxh807NRuOw6hxquQx/6nl8NS4Bsy98MzYlaR7OBsA1TOeF2EljNslO0sgNVs1XUA2onq3ucMLyk7NvzfybwpYbwUYnZlX3bw1TNivEvyHgjQLqCerqaC4gLD+ZiWarq1ul0brR5blhFpeE73FC4hIEfFBHLcLLVX24w/KTN43iyX1jNhpam8ST4WxqPLlPBrhJZnlwuSRIG24n22NTtozN1XPZGttMrVbb4PUO5Hcno2PI5gqLKUdHqCBnvfR0fnh+MoqtzPemANwKX29z/E3TPLxt6vs3A7Yyj/Ftllmr8LRuRcnwhsclZmzplrjJyYfqlpL3r2SzRgxlWth3G7CllB07W7JiQkg7yg05xzFgs1piprIRyubIeQeywyI7OlfIlxiw2XKmpckpefpOcwGn1E3Iaq0KuSDZljASm5rVAsnqgWUxz2XAhmY8ynMtc4GclpR21Sx3qJ00YMMh/uP1O0stPV/i8pxC63LgvXPUhKOp4z1U8pMmbGWSn7xV5xAWz86DVojHPaWdVHY5+VDdUmLueUxe2YRNyis3hL2ux41nqzgVOynvkZvh/o1vcdgyE2sdggnYpJT8puBHhBMjD0xnveE1JB5Sjdubhp3jtIoTsYnnOEXZ/9M52wRTsZPSSdrWyHiyoJzRibgLYCI2QfK8mBPc5w8ejtHteBLdyOEuPPEYHXPJZ53S2dYWmJCNtXVB+jTLt0z74CzrDf6pJj+7vp4jhhCV+4uLi8RQrQaP5YUz6u8fCCf5+PyR1nLEFmG9Ba1DymI9Es67ceUgqiNdWsFlHoy3k/5Ef+5wbS3y7wastbUSiPw7BbNiT1hT3XeDGJWJ7OS5Y5PsZC/GusVqTqKvZPL1Fq8vZaKv0lLdXoN4ldk3TLf4fQk6g7/ffdx+vw3iB4d1i+f/dGAHsrX3QCzLbsD2DMS0/B78l5SkJCUpSUlKUpKSlGmW/wErOKTxUOS81gAAAABJRU5ErkJggg==" /> </span> • <span class="author-name-text item-date-pub">11 Apr 2016</span> </p> <p class="item-strip-abstract">Face detection and alignment in unconstrained environment are challenging due to various poses, illuminations and occlusions.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 43</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/joint-face-detection-and-alignment-using" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/joint-face-detection-and-alignment-using#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/llama-2-open-foundation-and-fine-tuned-chat"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2307.09288.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/llama-2-open-foundation-and-fine-tuned-chat">Llama 2: Open Foundation and Fine-Tuned Chat Models</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/facebookresearch/llama" onclick="captureOutboundLink('https://github.com/facebookresearch/llama'); return true;" style="font-size:13px"> facebookresearch/llama </a> </span> • <span class="item-framework-link"> <img class="" src="https://production-assets.paperswithcode.com/perf/images/frameworks/pytorch-2fbf2cb9.png" /> </span> • <span class="author-name-text item-date-pub">18 Jul 2023</span> </p> <p class="item-strip-abstract">In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 19</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/llama-2-open-foundation-and-fine-tuned-chat" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/llama-2-open-foundation-and-fine-tuned-chat#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/evaluating-large-language-models-trained-on"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2107.03374.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/evaluating-large-language-models-trained-on">Evaluating Large Language Models Trained on Code</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/openai/human-eval" onclick="captureOutboundLink('https://github.com/openai/human-eval'); return true;" style="font-size:13px"> openai/human-eval </a> </span> • <span class="author-name-text item-date-pub">7 Jul 2021</span> </p> <p class="item-strip-abstract">We introduce Codex, a GPT language model fine-tuned on publicly available code from GitHub, and study its Python code-writing capabilities.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 13</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/evaluating-large-language-models-trained-on" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/evaluating-large-language-models-trained-on#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/pix2code-generating-code-from-a-graphical"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/1705.07962.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/pix2code-generating-code-from-a-graphical">pix2code: Generating Code from a Graphical User Interface Screenshot</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/tonybeltramelli/pix2code" onclick="captureOutboundLink('https://github.com/tonybeltramelli/pix2code'); return true;" style="font-size:13px"> tonybeltramelli/pix2code </a> </span> • <span class="author-name-text item-date-pub">22 May 2017</span> </p> <p class="item-strip-abstract">Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 11</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/pix2code-generating-code-from-a-graphical" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/pix2code-generating-code-from-a-graphical#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/gpt-4-technical-report-1"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2303.08774.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/gpt-4-technical-report-1">GPT-4 Technical Report</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/openai/evals" onclick="captureOutboundLink('https://github.com/openai/evals'); return true;" style="font-size:13px"> openai/evals </a> </span> • <span class="item-conference-link"> <a href="/conference/preprint-2023-3"> Preprint 2023 </a> </span> </p> <p class="item-strip-abstract">We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 11</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/gpt-4-technical-report-1" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/gpt-4-technical-report-1#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/a-conversational-paradigm-for-program"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2203.13474.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/a-conversational-paradigm-for-program">CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/salesforce/CodeGen" onclick="captureOutboundLink('https://github.com/salesforce/CodeGen'); return true;" style="font-size:13px"> salesforce/CodeGen </a> </span> • <span class="item-framework-link"> <img class="" src="https://production-assets.paperswithcode.com/perf/images/frameworks/jax-6ee30fa5.png" /> </span> • <span class="author-name-text item-date-pub">25 Mar 2022</span> </p> <p class="item-strip-abstract">To democratize this, we train and release a family of large language models up to 16. 1B parameters, called CODEGEN, on natural language and programming language data, and open source the training library JAXFORMER.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 8</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/a-conversational-paradigm-for-program" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/a-conversational-paradigm-for-program#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/structvae-tree-structured-latent-variable"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/1806.07832.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/structvae-tree-structured-latent-variable">StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/pcyin/tranX" onclick="captureOutboundLink('https://github.com/pcyin/tranX'); return true;" style="font-size:13px"> pcyin/tranX </a> </span> • <span class="item-framework-link"> <img class="" src="https://production-assets.paperswithcode.com/perf/images/frameworks/pytorch-2fbf2cb9.png" /> </span> • <span class="item-conference-link"> <a href="/conference/acl-2018-7"> ACL 2018 </a> </span> </p> <p class="item-strip-abstract">Semantic parsing is the task of transducing natural language (NL) utterances into formal meaning representations (MRs), commonly represented as tree structures.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 7</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/structvae-tree-structured-latent-variable" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/structvae-tree-structured-latent-variable#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/palm-scaling-language-modeling-with-pathways-1"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2204.02311.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/palm-scaling-language-modeling-with-pathways-1">PaLM: Scaling Language Modeling with Pathways</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/lucidrains/CoCa-pytorch" onclick="captureOutboundLink('https://github.com/lucidrains/CoCa-pytorch'); return true;" style="font-size:13px"> lucidrains/CoCa-pytorch </a> </span> • <span class="item-framework-link"> <img class="" src="https://production-assets.paperswithcode.com/perf/images/frameworks/pytorch-2fbf2cb9.png" /> </span> • <span class="item-conference-link"> <a href="/conference/google-research-2022-4"> Google Research 2022 </a> </span> </p> <p class="item-strip-abstract">To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 7</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/palm-scaling-language-modeling-with-pathways-1" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/palm-scaling-language-modeling-with-pathways-1#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> <div class="paper-card infinite-item">  <div class="container-fluid"> <div class="row"> <div class="col-lg-3"> <a href="/paper/santacoder-don-t-reach-for-the-stars"> <div class="item-image" style="background-image: url('https://production-media.paperswithcode.com/thumbnails/paper/2301.03988.jpg');"> </div> </a> </div> <div class="col-lg-9"> <div class="row"> <div class="col-lg-9 item-content"> <h1><a href="/paper/santacoder-don-t-reach-for-the-stars">SantaCoder: don't reach for the stars!</a></h1> <p class="author-section" style="padding-top:2px"> <span class="item-github-link"> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> <a href="https://github.com/bigcode-project/starcoder" onclick="captureOutboundLink('https://github.com/bigcode-project/starcoder'); return true;" style="font-size:13px"> bigcode-project/starcoder </a> </span> • <span class="item-framework-link"> <img class="" src="https://production-assets.paperswithcode.com/perf/images/frameworks/pytorch-2fbf2cb9.png" /> </span> • <span class="author-name-text item-date-pub">9 Jan 2023</span> </p> <p class="item-strip-abstract">The BigCode project is an open-scientific collaboration working on the responsible development of large language models for code.</p> </div> <div class="col-lg-3 item-interact text-center"> <div class="entity-stars"> <span class="badge badge-secondary"><span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> 7</span> </div> <div class="entity" style="margin-bottom: 20px;"> <a href="/paper/santacoder-don-t-reach-for-the-stars" class="badge badge-light "> <span class=" icon-wrapper icon-ion" data-name="document"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M428 224H288a48 48 0 0 1-48-48V36a4 4 0 0 0-4-4h-92a64 64 0 0 0-64 64v320a64 64 0 0 0 64 64h224a64 64 0 0 0 64-64V228a4 4 0 0 0-4-4z"/><path d="M419.22 188.59L275.41 44.78a2 2 0 0 0-3.41 1.41V176a16 16 0 0 0 16 16h129.81a2 2 0 0 0 1.41-3.41z"/></svg></span> Paper </a> <br/> <a href="/paper/santacoder-don-t-reach-for-the-stars#code" class="badge badge-dark "> <span class=" icon-wrapper icon-ion" data-name="logo-github"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M256 32C132.3 32 32 134.9 32 261.7c0 101.5 64.2 187.5 153.2 217.9a17.56 17.56 0 0 0 3.8.4c8.3 0 11.5-6.1 11.5-11.4 0-5.5-.2-19.9-.3-39.1a102.4 102.4 0 0 1-22.6 2.7c-43.1 0-52.9-33.5-52.9-33.5-10.2-26.5-24.9-33.6-24.9-33.6-19.5-13.7-.1-14.1 1.4-14.1h.1c22.5 2 34.3 23.8 34.3 23.8 11.2 19.6 26.2 25.1 39.6 25.1a63 63 0 0 0 25.6-6c2-14.8 7.8-24.9 14.2-30.7-49.7-5.8-102-25.5-102-113.5 0-25.1 8.7-45.6 23-61.6-2.3-5.8-10-29.2 2.2-60.8a18.64 18.64 0 0 1 5-.5c8.1 0 26.4 3.1 56.6 24.1a208.21 208.21 0 0 1 112.2 0c30.2-21 48.5-24.1 56.6-24.1a18.64 18.64 0 0 1 5 .5c12.2 31.6 4.5 55 2.2 60.8 14.3 16.1 23 36.6 23 61.6 0 88.2-52.4 107.6-102.3 113.3 8 7.1 15.2 21.1 15.2 42.5 0 30.7-.3 55.5-.3 63 0 5.4 3.1 11.5 11.4 11.5a19.35 19.35 0 0 0 4-.4C415.9 449.2 480 363.1 480 261.7 480 134.9 379.7 32 256 32z"/></svg></span> Code </a> <br/> </div> </div> </div> </div> </div> </div> </div> </div> <div class="loading" style="display: none"> <div class="loader-ellips infinite-scroll-request"> <span class="loader-ellips__dot"></span> <span class="loader-ellips__dot"></span> <span class="loader-ellips__dot"></span> <span class="loader-ellips__dot"></span> </div> </div> <a class="infinite-more-link" href="?page=2&q="></a> </div> <div class="loading-trigger"></div> </div> <div class="col-lg-3 wide-sidebar task-infobox" id="task-sidebar"> <div class="image-container"> <a href="https://production-media.paperswithcode.com/thumbnails/task/658fcf6a-52ca-4865-b111-28bf61c346dc.jpg" data-lightbox="imageresource"> <img id="imageresource" width=100% src="https://production-media.paperswithcode.com/thumbnails/task/658fcf6a-52ca-4865-b111-28bf61c346dc.jpg"> </a> </div> <div class="task-toc"> <h4>Content</h4> <hr> <nav> <a class="toc-link" href="#task-home"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="book"><svg viewBox="0 0 448 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M356 161H188c-6.6 0-12-5.4-12-12v-8c0-6.6 5.4-12 12-12h168c6.6 0 12 5.4 12 12v8c0 6.6-5.4 12-12 12zm12 52c0 6.6-5.4 12-12 12H188c-6.6 0-12-5.4-12-12v-8c0-6.6 5.4-12 12-12h168c6.6 0 12 5.4 12 12v8zm64.7 268h3.3c6.6 0 12 5.4 12 12v8c0 6.6-5.4 12-12 12H80c-44.2 0-80-35.8-80-80V81C0 36.8 35.8 1 80 1h344c13.3 0 24 10.7 24 24v368c0 10-6.2 18.6-14.9 22.2-3.6 16.1-4.4 45.6-.4 65.8zM128 385h288V33H128v352zm-96 16c13.4-10 30-16 48-16h16V33H80c-26.5 0-48 21.5-48 48v320zm372.3 80c-3.1-20.4-2.9-45.2 0-64H80c-64 0-64 64 0 64h324.3z"/></svg></span> Introduction</a> <a class="toc-link" href="#benchmarks"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="chart-line"><svg viewBox="0 0 512 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M504 417c4.42 0 8 3.58 8 8v16c0 4.42-3.58 8-8 8H16c-8.84 0-16-7.16-16-16V73c0-4.42 3.58-8 8-8h16c4.42 0 8 3.58 8 8v344h472zM98.34 264.03l84.12-83.32c6.25-6.2 16.34-6.18 22.57.05l84.63 84.63 82.22-82.22-44.71-44.71C311.87 123.16 322.7 97 344.34 97h119.47c8.94 0 16.19 7.25 16.19 16.19v119.47c0 14.64-11.98 24.34-24.46 24.34-5.97 0-12.05-2.21-17-7.16L394.5 205.8l-93.53 93.53c-6.25 6.25-16.38 6.25-22.63 0l-84.69-84.69-72.69 72.01c-3.12 3.12-8.19 3.12-11.31 0l-11.31-11.31c-3.12-3.12-3.12-8.19 0-11.31zM362.96 129L448 214.04V129h-85.04z"/></svg></span> Benchmarks</a> <a class="toc-link" href="#datasets"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="database"><svg viewBox="0 0 448 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M224 33C118 33 32 61.75 32 97v32c0 35.25 86 64 192 64s192-28.75 192-64V97c0-35.25-86-64-192-64zm192 149.5c-41.251 29-116.75 42.5-192 42.5S73.25 211.5 32 182.5V225c0 35.25 86 64 192 64s192-28.75 192-64v-42.5zm0 96c-41.251 29-116.75 42.5-192 42.5S73.25 307.5 32 278.5V321c0 35.25 86 64 192 64s192-28.75 192-64v-42.5zm0 96c-41.251 29-116.75 42.5-192 42.5S73.25 403.5 32 374.5V417c0 35.25 86 64 192 64s192-28.75 192-64v-42.5zM224 1c77.904 0 224 18.662 224 96v320c0 77.2-145.858 96-224 96-77.904 0-224-18.662-224-96V97C0 19.8 145.858 1 224 1z"/></svg></span> Datasets</a> <a class="toc-link" href="#subtasks"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="sitemap"><svg viewBox="0 0 640 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M608 353c17.67 0 32 14.33 32 32v96c0 17.67-14.33 32-32 32h-96c-17.67 0-32-14.33-32-32v-96c0-17.67 14.33-32 32-32h32v-96H336v96h32c17.67 0 32 14.33 32 32v96c0 17.67-14.33 32-32 32h-96c-17.67 0-32-14.33-32-32v-96c0-17.67 14.33-32 32-32h32v-96H96v96h32c17.67 0 32 14.33 32 32v96c0 17.67-14.33 32-32 32H32c-17.67 0-32-14.33-32-32v-96c0-17.67 14.33-32 32-32h32v-97.59C64 238.64 77.62 225 94.41 225H304v-64h-48c-17.67 0-32-14.33-32-32V33c0-17.67 14.33-32 32-32h128c17.67 0 32 14.33 32 32v96c0 17.67-14.33 32-32 32h-48v64h209.59c16.79 0 30.41 13.64 30.41 30.41V353h32zm-480 32H32v96h96v-96zm240 0h-96v96h96v-96zM256 129h128V33H256v96zm352 352v-96h-96v96h96z"/></svg></span> Subtasks</a> <a class="toc-link" href="#task-libraries"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="file-code"><svg viewBox="0 0 384 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M369.941 98.941c7.76 7.76 14.059 22.966 14.059 33.94V465c0 26.51-21.49 48-48 48H48c-26.51 0-48-21.49-48-48V49C0 22.49 21.49 1 48 1h204.118c10.975 0 26.18 6.3 33.94 14.059zm-22.627 22.628l-83.883-83.884c-1.728-1.73-5.057-3.608-7.431-4.194V129h95.509c-.586-2.374-2.465-5.703-4.195-7.431zM336 481c8.837 0 16-7.163 16-16V161H248c-13.254 0-24-10.745-24-24V33H48c-8.836 0-16 7.163-16 16v416c0 8.837 7.164 16 16 16h288zm-161.47-67.404l-25.93-7.527c-2.03-.59-3.677-2.784-3.677-4.898 0-.4.09-1.037.202-1.422l58.027-199.869c.59-2.03 2.784-3.678 4.898-3.678.4 0 1.038.09 1.422.202l25.928 7.527c2.03.59 3.677 2.784 3.677 4.898 0 .4-.09 1.037-.202 1.422l-58.026 199.87c-.59 2.03-2.784 3.677-4.898 3.677-.4 0-1.037-.09-1.422-.202zm-48.447-47.674c-.834.89-2.5 1.612-3.72 1.612-1.115 0-2.677-.618-3.49-1.38L57.611 308.72c-.89-.834-1.613-2.501-1.613-3.721 0-1.219.723-2.886 1.613-3.72l61.262-57.434c.813-.761 2.375-1.38 3.489-1.38 1.22 0 2.886.723 3.72 1.613l18.493 19.724c.761.812 1.38 2.375 1.38 3.488 0 1.273-.776 2.988-1.732 3.83L105.725 305l38.5 33.88c.954.842 1.73 2.557 1.73 3.83 0 1.113-.619 2.676-1.38 3.488zm139.043.232c-.812.762-2.375 1.38-3.488 1.38-1.22 0-2.887-.722-3.72-1.612l-18.493-19.724c-.762-.812-1.38-2.375-1.38-3.488 0-1.273.776-2.988 1.732-3.83L278.276 305l-38.5-33.88c-.954-.842-1.73-2.557-1.73-3.83 0-1.113.619-2.676 1.38-3.488l18.491-19.724c.834-.89 2.501-1.612 3.721-1.612 1.113 0 2.676.618 3.488 1.379l61.262 57.434c.89.834 1.612 2.501 1.612 3.72 0 1.22-.722 2.887-1.612 3.72z"/></svg></span> Libraries</a> <a class="toc-link" href="#papers-list"><span class=" icon-wrapper icon-fa icon-fa-light" data-name="file"><svg viewBox="0 0 384 520.146" xmlns="http://www.w3.org/2000/svg"><path d="M369.9 98.9c9 9 14.1 21.3 14.1 34V465c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V49C0 22.5 21.5 1 48 .9h204.1C264.8.9 277 6 286 15zm-22.6 22.7l-83.899-83.9c-2.1-2.1-4.6-3.5-7.4-4.2V129h95.5c-.7-2.8-2.1-5.3-4.2-7.4zM336 481c8.8 0 16-7.2 16-16V161H248c-13.3 0-24-10.7-24-24V33H48c-8.8 0-16 7.2-16 16v416c0 8.8 7.2 16 16 16h288z"/></svg></span> Papers</a> <a data-call-url="/tasklist/code-generation/greatest" data-target="/task/code-generation" class="toc-papers-button" style="padding-left: 16px;" href="#papers-list">- Most implemented</a> <a data-call-url="/tasklist/code-generation/social" data-target="/task/code-generation/social" class="toc-papers-button" style="padding-left: 16px;" href="#papers-list">- Social</a> <a data-call-url="/tasklist/code-generation/latest" data-target="/task/code-generation/latest" class="toc-papers-button" style="padding-left: 16px;" href="#papers-list">- Latest</a> <a data-call-url="/tasklist/code-generation/codeless" data-target="/task/code-generation/codeless" class="toc-papers-button" style="padding-left: 16px;" href="#papers-list">- No code</a> </nav> </div> </div> </main> </div> <div class="footer"> <div class="footer-contact"> <span class="footer-contact-item">Contact us on:</span> <a class="footer-contact-item" href="mailto:hello@paperswithcode.com"> <span class=" icon-wrapper icon-ion" data-name="mail"><svg xmlns="http://www.w3.org/2000/svg" width="512" height="512" viewBox="0 0 512 512"><path d="M424 80H88a56.06 56.06 0 0 0-56 56v240a56.06 56.06 0 0 0 56 56h336a56.06 56.06 0 0 0 56-56V136a56.06 56.06 0 0 0-56-56zm-14.18 92.63l-144 112a16 16 0 0 1-19.64 0l-144-112a16 16 0 1 1 19.64-25.26L256 251.73l134.18-104.36a16 16 0 0 1 19.64 25.26z"/></svg></span> hello@paperswithcode.com </a>. <span class="footer-contact-item"> Papers With Code is a free resource with all data licensed under <a rel="noreferrer" href="https://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA</a>. </span> </div> <div class="footer-links"> <a href="/site/terms">Terms</a> <a href="/site/data-policy">Data policy</a> <a href="/site/cookies-policy">Cookies policy</a> <a href="/about#team" class="fair-logo"> from <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAANAAAAAgCAMAAABU6AZfAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAJcEhZcwAAFiUAABYlAUlSJPAAAABFUExURUdwTBwqMhwqMxsqMhkqMxsqMhwqMgCA+hwrMxJIgBsrMxsqMgJ28AF58wF38BsqMwB58hsqMwF17wF07hwrMwRm4QJz7Wj6SIIAAAAUdFJOUwDP87wcPIT+4A1tVti1Ta0smZVzG3JP8wAABR9JREFUWMO1memWpCoMgF0QxX1//0e9kCAkAadq5tzKjzndQmM+szNFEWQ9puu6xn02BXm4j23bTsdapKJAMguFgRVT/Ejyx4uH5hgvL1PUfm69jEd6bN05GTJvXF5X/hfRcPyWe2kTLDFdRA4ENVMbZZJGMt3ppEttNMDC2X/Qa7MK1OrveZoKz2/445I+U4znuvaExxKZLFCqtym/A6rzn+OjbHj8ubwDmfESslvtgWea13WeckQPUKJTf/4USHkDnVXzCrT74DnmeX+8rjgcxA4QBmPpyAKdOm+5XwFpgHH/bG9AMzLMqM9DxxCQaM0qLr7U4xE/AgIDVRBHlcoDeYd7lFee6GZOBvaaskD8S6nut0Dg0ItZEt+IQAfjseIzRDvS/WCxWQJ17phqEGqepQBS/VaXZa0H/4XUYMVt6nr309DEjYvduPT2gWELQTr0iQbC1+SADOg/kjVvspGqX6zSRAgEKbqOf6zgd82AVB+8s0YNm5NL6Y8MGzttwKt0krP9+9A/+hzQTALoUX5MnxW7iCIEUmD7IVZb8G0G1HRE9UqbWKkEUFPSR0MWqH5eB65XmgzQdN3WGjxReROxPD2LROeBIEiD7UGLraBAjMcS9W9AquTPckBgoMqEWG1SIGN57otn5KO9Y30N4rq6MQFC5TX1cEWBfJLY+mbQ5ZMUm8UK7F1A9GNc90T3enkpCZhCdUzfdQq0Wp774gnZao55YU3SgkmAVBez1eDfR4BABd/XqY36ichyaLUnyJZ8jatimUBjqQTouK2M3OGs4miiiduN5bkHCL15C9Zw7heBRMHYSMRxIGyYFsPqpwTqactT8w0P0OSA9iRY9jQvrDyIAhCoAjrrR90I1PNCpcivHEh+cATUmS5xoCaNB3ggMzqgRO/RYPIb1WviDkB4sv22kB8ghQcgUIFWzyUmaQ6kpf5DCoTFh5fwQQCt493e9ypD5Xjq7S5cMQeEubpBf2oKCoSMohPzduBAi2yimhRIc3NvrOd+gCxPexvhcGPM3SRoJpbmIhAGSudTNgNCR+qIRL05UCebsxTIiAYOX6sEkONphRkw9A9ZjADIZIDg857we5MBSiQHVMlWJgXyeTBIyVpGD4RttHC4yVtENHn7K5ASdeM3QGX2sKcKBCBmITYmrGii9TOQT7JYwxOgrhbyby4XJrvs54kuR8vlCg4XEgEOEs8Q8R5DYZboCwEESpTmi/Hhc1Lo8zxPlghZjpbLqWVGUGxSes1y4W2lkkC+Wf0C6GPaxtZo0VQW4nOhsJLqAg01HXqgGN0+083MegKoYLdisbDqzHVG1iZJYe0EUDoB+dj149gDRCCgt2lZ1zA5nhvCyEwvrc/b3N/HiZlMgINmZaR/aX3MJluf7Kepo8+F5tRfUh1wR0odzg8Srnm9w7L5SyB/p6H9Ptt0Vj310ngAlDHbnLo3mGc00sJiQ+4KEM+I8xC7fWv5VGcz3Y0C2ZCa70sgf0tXbnbY1jXpln3W6jYXDG4jNthdrfVWn8n4gAVAZe+0GgaEaeGFx4XRQyTM9yWQnNuIAy5/HPAWPuDJ8Yc66sYvSeY/8dhlYqH0kuQzkFQ03nnHCyI/gtc0GfM7BVPmL5J0yHPkXm6d3u6v/TLw3GL5ayDr6WW47awHYmS1VC+XJOVQcCCZBPk13SCvgmcb8uI/UqjqdvlOlk3j5OU20C0putdO1ZWNo0a8oumXslx0vMYaNrfPURt2hnp5G2rhtsEP5j/3Wqt0fQd1YgAAAABJRU5ErkJggg=="> </a> </div> </div> <script> // MathJax window.MathJax = { tex: { inlineMath: [ ["$", "$"], ["\$", "\$"], ], }, }; const mathjaxScript = document.createElement("script"); mathjaxScript.src = "https://production-assets.paperswithcode.com/static/js/mathjax/tex-chtml.js"; document.head.appendChild(mathjaxScript); </script> <script src="https://production-assets.paperswithcode.com/perf/766.4af6b88b.js" defer></script><script src="https://production-assets.paperswithcode.com/perf/2.6da00df7.js" defer></script><script src="https://production-assets.paperswithcode.com/perf/351.a22a9607.js" defer></script><script src="https://production-assets.paperswithcode.com/perf/101.5f271f23.js" defer></script><script src="https://production-assets.paperswithcode.com/perf/view_task.e61ab167.js" defer></script> </body> </html>

CINXE.COM

Code Generation | Papers With Code