CINXE.COM

<html> <head> <title>Yuandong Tian's webpage</title> <link rel="stylesheet" type="text/css" href="homepage.css"> <script src="homepage.js"></script> <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script> </head> <body>  <font face="helvetica, ariel, 'sans serif'"> <table border="0" cellpadding="15" cellspacing="0" width="1400"> <tr> <td width = "65%"> <table cellspacing="15"> <tr><td> <table cellspacing="15"> <tr> <td>  <img width=200 src="imgs/bio.png" border="0"> </td> <td> <font face="helvetica, ariel, 'sans serif'" size="6"> <b>Yuandong Tian </b> <img width=60 src="imgs/name.gif"> <a href="http://www.statcounter.com/" target="_blank"> <span style='font-size:20pt;font-family:"Verdana","sans-serif";color:windowtext; mso-no-proof:yes;text-decoration:none;text-underline:none'><img border=0 width=80 height=20 id="_x0000_i1030" src="http://c26.statcounter.com/counter.php?sc_project=2610795&java=0&security=94325305&invisible=0" alt="web metrics"></span></a> <br><br> </font> <font face="helvetica, ariel, 'sans serif'" size="4"> Research Scientist Director <br> <a href="https://research.facebook.com/ai"> Meta AI (FAIR) </a><br> Email: yuandong [at] meta [dot] com<br> </font> </td> </tr> </table> </td></tr> <tr><td> <h2>Brief Bio</h2> <p class="bio"> Yuandong Tian is a Research Scientist Director in Meta AI Research (FAIR), leading the group of reasoning, planning and decision-making with Large Language Models (LLMs). He is the project lead for <button onclick="openPaper('OpenGo')">OpenGo</button> project that beats professional players with a single GPU during inference, serves as the main mentor of <button onclick="openPaper('StreamingLLM')">StreamingLLM</button> and <button onclick="openPaper('GaLore')">GaLore</button> that improve the training and inference of LLM, and is the first-author recipient of 2021 ICML Outstanding Paper Honorable Mentions <button onclick="openPaper('DirectPred')">DirectPred</button> and 2013 ICCV Marr Prize Honorable Mentions <button onclick="openPaper('HierarchicalDataDrivenDescent')">HierarchicalDataDrivenDescent</button>, and also received the 2022 CGO Distinguished Paper Award <button onclick="openPaper('CompilerGym')">CompilerGym</button>. Prior to that, he worked in Google Self-driving Car team in 2013-2014 and received a Ph.D in <a href='http://www.ri.cmu.edu'>Robotics Institute</a>, Carnegie Mellon University in 2013. He has been appointed as area chairs for NeurIPS, ICML, AAAI, CVPR and AIStats. <br><br> <a href='https://scholar.google.com/citations?user=0mgEF28AAAAJ&hl=en'>Google Scholar</a> and <a href='resume.pdf'>CV</a>.<br><br> <b>Research Directions:</b> Decision-Making, Optimization, Representation Learning, LLMs<br> <br> <a href="https://twitter.com/tydsh?ref_src=twsrc%5Etfw" class="twitter-follow-button" data-show-count="true">@tydsh</a><script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> </p> </td></tr> </table> </td> <td> <h2>News</h2> <section> <div class="news" style="padding-left: 30px;"> <b>[Sep. 26, 2024]</b> 2 papers (<button onclick="openPaper('attn-transfer')">attn-transfer</button> <button onclick="openPaper('reverse-curse')">reverse-curse</button>) are accepted in NeurIPS2024!<br><br> <b>[Sep. 20, 2024]</b> 1 papers (<button onclick="openPaper('PerSE')">PerSE</button>) is accepted in EMNLP2024!<br><br> <b>[Jul. 27, 2024]</b> Keynote in <a href='https://sites.google.com/view/tf2m'>Theoretical Foundation for Foundation Models</a> (T2FM) workshop in ICML 2024 (<button onclick="openPaper('joma')">joma</button>, <button onclick="openPaper('GaLore')">GaLore</button>, <button onclick="openPaper('scan-snap')">scan-snap</button>). <a href='talks/icml24_tf2m_workshop2.pdf'>Slides</a>.<br><br> <b>[Jul. 09, 2024]</b> 2 papers (<button onclick="openPaper('TriForce')">TriForce</button> <button onclick="openPaper('SearchFormer')">SearchFormer</button>) are accepted in COLM2024!<br><br> <b>[Jul. 08, 2024]</b> Guest lecture in UCLA summer school (remote).<br><br> <b>[May. 23, 2024]</b> Invited talk in Cybersecurity CRC and CSIRO's Data61 Australia seminar series.<br><br> <b>[May. 11, 2024]</b> Invited <a href='https://iclr.cc/virtual/2024/workshop/20589'>talk</a> in 2nd Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) in ICLR 2024 (<button onclick="openPaper('scan-snap')">scan-snap</button>,<button onclick="openPaper('joma')">joma</button>,<button onclick="openPaper('GaLore')">GaLore</button>,<button onclick="openPaper('MobileLLM')">MobileLLM</button>). <a href='talks/iclr24_fomo_workshop.pdf'>Slides</a>.<br><br> <b>[May. 11, 2024]</b> Invited talk in Generative Models for Decision Making <a href='https://sites.google.com/view/genai4dm-iclr2024'>Gen4DM</a> in ICLR 2024 (<button onclick="openPaper('AdvPrompter')">AdvPrompter</button>,<button onclick="openPaper('SearchFormer')">SearchFormer</button>,<button onclick="openPaper('SurCo')">SurCo</button>,<button onclick="openPaper('LANCER')">LANCER</button>,<button onclick="openPaper('GenCo')">GenCo</button>). <a href='talks/talk_generative4dm.pdf'>Slides</a>.<br><br> <b>[May. 01, 2024]</b> 6 papers (<button onclick="openPaper('GaLore')">GaLore</button> <button onclick="openPaper('LoCoCo')">LoCoCo</button> <button onclick="openPaper('MobileLLM')">MobileLLM</button> <button onclick="openPaper('TravelPlanner')">TravelPlanner</button> <button onclick="openPaper('ConPas')">ConPas</button> <button onclick="openPaper('GenCo')">GenCo</button>) are accepted in ICML2024!<br><br> <b>[Apr. 18, 2024]</b> Guest lecture in Caltech <a href='https://gkioxari.github.io/teaching/cs148_sp2023/'>CS 148</a> (<button onclick="openPaper('DirectCLR')">DirectCLR</button>,<button onclick="openPaper('alpha-CL')">alpha-CL</button>,<button onclick="openPaper('nonlinear-CL')">nonlinear-CL</button>,<button onclick="openPaper('DirectPred')">DirectPred</button>,<button onclick="openPaper('scan-snap')">scan-snap</button>,<button onclick="openPaper('joma')">joma</button>). <a href='talks/talk_caltech_cs148.pdf'>Slides</a>.<br><br> <b>[Apr. 18, 2024]</b> Guest lecture in Caltech <a href='https://sites.google.com/view/cs-159-2024'>CS 159</a> (<button onclick="openPaper('SearchFormer')">SearchFormer</button>,<button onclick="openPaper('SurCo')">SurCo</button>,<button onclick="openPaper('LANCER')">LANCER</button>,<button onclick="openPaper('GenCo')">GenCo</button>). <a href='https://www.youtube.com/watch?v=uxHkNeCs2PI'>Video</a> <a href='https://drive.google.com/file/d/1_f0PawkoYN_XwlfhqrQ3i0g_9D1TQxHf/view'>Slides</a>.<br><br> <b>[Apr. 11, 2024]</b> <a href='https://twitter.com/stanfordnlp/status/1777781623712604624'>Seminar talk</a> in LLMs understanding and applications in Stanford NLP seminar (<button onclick="openPaper('joma')">joma</button>, <button onclick="openPaper('scan-snap')">scan-snap</button>, <button onclick="openPaper('StreamingLLM')">StreamingLLM</button>, <button onclick="openPaper('GaLore')">GaLore</button>, <button onclick="openPaper('MobileLLM')">MobileLLM</button>). Slides <a href='talks/stanford_nlp_2024.pdf'>here</a>.<br><br> <b>[Feb. 06, 2024]</b> Invited lecture on understanding LLMs and its applications in UC Berkeley <a href='https://rdi.berkeley.edu/understanding_llms/s24'>CS 194</a> (<button onclick="openPaper('joma')">joma</button>, <button onclick="openPaper('scan-snap')">scan-snap</button>, <button onclick="openPaper('DejaVu')">DejaVu</button>,<button onclick="openPaper('H2O')">H2O</button>,<button onclick="openPaper('pos-interp')">pos-interp</button>,<button onclick="openPaper('StreamingLLM')">StreamingLLM</button>). Slides <a href='https://www.dropbox.com/scl/fi/p9uhabbe0k5v7l1l1mmdq/berkeley_transformer.pdf?rlkey=fjmivphhwsitcf3rrndk41ap1&dl=0'>here</a>.<br><br> <b>[Jan. 16, 2024]</b> 4 papers (<button onclick="openPaper('StreamingLLM')">StreamingLLM</button> <button onclick="openPaper('RLCD')">RLCD</button> <button onclick="openPaper('H-GAP')">H-GAP</button> <button onclick="openPaper('joma')">joma</button>) are accepted in ICLR2024!<br><br> <b>[Dec. 06, 2023]</b> Invited <a href='https://www.youtube.com/watch?v=eXPhvQgAT_I'>talk</a> about Efficient LLM inference in long context (<button onclick="openPaper('DejaVu')">DejaVu</button>,<button onclick="openPaper('H2O')">H2O</button>,<button onclick="openPaper('pos-interp')">pos-interp</button>,<button onclick="openPaper('StreamingLLM')">StreamingLLM</button>). Slides <a href='https://www.dropbox.com/scl/fi/eev737hn0zy5o4r7e3of5/london_talk.pdf?rlkey=ff7gcfiezmaomyo4xyb3z1fbc&dl=0'>here</a>.<br><br> <b>[Oct. 06, 2023]</b> Invited talk in RIKEN AIP, Tokyo, Japan (<button onclick="openPaper('joma')">joma</button>, <button onclick="openPaper('scan-snap')">scan-snap</button>, <button onclick="openPaper('DejaVu')">DejaVu</button>,<button onclick="openPaper('H2O')">H2O</button>,<button onclick="openPaper('pos-interp')">pos-interp</button>,<button onclick="openPaper('StreamingLLM')">StreamingLLM</button>). Talk <a href='https://www.youtube.com/watch?v=u05Z74dF0Gg'>here</a>, Slides <a href='https://www.dropbox.com/scl/fi/cvorzhrhxkjz7sidxu9b7/riken_aip_talk.pdf?rlkey=blr7ubklmuoy1ns337b8e9pzc&dl=0'>here</a>.<br><br> <b>[Sep. 26, 2023]</b> Invited talk in HKU Institution of Data Science (HKU IDS) (<button onclick="openPaper('joma')">joma</button>, <button onclick="openPaper('scan-snap')">scan-snap</button>, <button onclick="openPaper('DejaVu')">DejaVu</button>,<button onclick="openPaper('H2O')">H2O</button>,<button onclick="openPaper('pos-interp')">pos-interp</button>). Talk <a href='https://datascience.hku.hk/2023/09/ids-seminar-demystifying-attention-mechanism-in-transformer-and-its-application-to-better-inference-of-large-language-models-llms/'>here</a>, Slides <a href='https://www.dropbox.com/scl/fi/96huccf2vmxll95l49ju3/hku_talk.pdf?rlkey=bca53o7h5ipsnpx34zyrocyyw&dl=0'>here</a>.<br><br> <b>[Sep. 21, 2023]</b> 3 papers (<button onclick="openPaper('H2O')">H2O</button> <button onclick="openPaper('scan-snap')">scan-snap</button> <button onclick="openPaper('LANCER')">LANCER</button>) are accepted in NeurIPS2023!<br><br> <b>[May. 02, 2023]</b> 1 papers (<button onclick="openPaper('DOC')">DOC</button>) is accepted in ACL2023!<br><br> <b>[Apr. 24, 2023]</b> 4 papers (<button onclick="openPaper('DejaVu')">DejaVu</button> <button onclick="openPaper('CL-LNS')">CL-LNS</button> <button onclick="openPaper('SurCo')">SurCo</button> <button onclick="openPaper('CompilerOpt')">CompilerOpt</button>) are accepted in ICML2023!<br><br> <b>[Mar. 02, 2023]</b> Invited talk in Microsoft on Long-form Story Generation (<button onclick="openPaper('Re3')">Re3</button>, <button onclick="openPaper('DOC')">DOC</button>). Slides <a href='https://www.dropbox.com/s/ovgh92pv8m8597k/microsoft_storygen_Mar2.pdf?dl=0'>here</a>.<br><br> <b>[Feb. 28, 2023]</b> Invited talk in <a href='http://www.ipam.ucla.edu/programs/workshops/artificial-intelligence-and-discrete-optimization/?tab=schedule'>IPAM</a> about AI-guided optimization (<button onclick="openPaper('SurCo')">SurCo</button>, <button onclick="openPaper('CZP')">CZP</button>). Slides <a href='https://www.dropbox.com/s/p5wh2p31p7gdhge/IPAM_Feb28_2.pdf?dl=0'>here</a>.<br><br> <b>[Feb. 17, 2023]</b> 1 papers (<button onclick="openPaper('NeuroShard')">NeuroShard</button>) is accepted in MLSys2023!<br><br> <b>[Jan. 25, 2023]</b> 1 papers (<button onclick="openPaper('LB-RELAX')">LB-RELAX</button>) is accepted in CPAIOR2023!<br><br> <b>[Jan. 20, 2023]</b> 3 papers (<button onclick="openPaper('TAP')">TAP</button> <button onclick="openPaper('nonlinear-CL')">nonlinear-CL</button> <button onclick="openPaper('MACTA')">MACTA</button>) are accepted in ICLR2023!<br><br> <b>[Oct. 17, 2022]</b> 1 papers (<button onclick="openPaper('AutoCAT')">AutoCAT</button>) is accepted in HPCA2023!<br><br> <b>[Oct. 06, 2022]</b> 1 papers (<button onclick="openPaper('Re3')">Re3</button>) is accepted in EMNLP2022!<br><br> <b>[Oct. 03, 2022]</b> Invited talk in MIT <a href='https://poggio-lab.mit.edu/'>Poggio's lab</a> about recent works on contrastive learning (<button onclick="openPaper('alpha-CL')">alpha-CL</button>, <button onclick="openPaper('nonlinear-CL')">nonlinear-CL</button>). Slides <a href='https://www.dropbox.com/s/a8uo5zk83kx9cyy/talk_poggio.pdf?dl=0'>here</a>.<br><br> <b>[Sep. 14, 2022]</b> 2 papers (<button onclick="openPaper('alpha-CL')">alpha-CL</button> <button onclick="openPaper('DreamShard')">DreamShard</button>) are accepted in NeurIPS2022!<br><br> <b>[Sep. 13, 2022]</b> Co-organize AAAI'23 workshop "Reinforcement Learning Ready for Production".<br><br> <b>[Aug. 21, 2022]</b> Keynote <a href='https://ieee-cog.org/2022/assets/video/Keynote%20III.mp4'>talk</a> in IEEE Conference on Games.<br><br> <b>[Aug. 04, 2022]</b> Invited talk in TTIC workshop on representation learning theory. <a href='https://twitter.com/tydsh/status/1554804042949279744'>Link</a><br><br> <b>[Jun. 08, 2022]</b> Invited talk in VALSE Webinar about SSL.<br><br> <b>[May. 18, 2022]</b> 1 papers (<button onclick="openPaper('AutoShard')">AutoShard</button>) is accepted in KDD2022!<br><br> <b>[May. 15, 2022]</b> 1 papers (<button onclick="openPaper('DenoisedMDP')">DenoisedMDP</button>) is accepted in ICML2022!<br><br> <b>[Apr. 28, 2022]</b> Guest lecture in Tianqi Chen's group in CMU.<br><br> <b>[Apr. 07, 2022]</b> Invited talk in UIUC about representation learning.<br><br> <b>[Mar. 05, 2022]</b> 1 papers (<button onclick="openPaper('Asym-Siam')">Asym-Siam</button>) is accepted in CVPR2022!<br><br> <b>[Jan. 24, 2022]</b> 3 papers (<button onclick="openPaper('DirectCLR')">DirectCLR</button> <button onclick="openPaper('NASViT')">NASViT</button> <button onclick="openPaper('LaMOO')">LaMOO</button>) are accepted in ICLR2022!<br><br> <b>[Nov. 29, 2021]</b> 1 papers (<button onclick="openPaper('LSTMvsTransformer')">LSTMvsTransformer</button>) is accepted in AAAI2022!<br><br> <b>[Nov. 05, 2021]</b> 1 papers (<button onclick="openPaper('CompilerGym')">CompilerGym</button>) is accepted in CGO2022!<br><br> <b>[Sep. 28, 2021]</b> 4 papers (<button onclick="openPaper('BeBold')">BeBold</button> <button onclick="openPaper('MADE')">MADE</button> <button onclick="openPaper('LaP3')">LaP3</button> <button onclick="openPaper('LaSynth')">LaSynth</button>) are accepted in NeurIPS2021!<br><br> <font color='red'><b>[Jul. 19, 2021]</b></font> Our paper <button onclick="openPaper('DirectPred')">DirectPred</button> got <a href='https://icml.cc/virtual/2021/awards_detail'>ICML Outstanding Paper Award Honorable Mention</a>!<br><br> <b>[Jun. 04, 2021]</b> Invited talk in University of Washington <a href='https://github.com/shlizee/NeuroAI'>NeuralAI Lab</a> about <button onclick="openPaper('DirectPred')">DirectPred</button>. Slides <a href='uw_seminar.pdf'>here</a>. Thanks Eli Shlizerman for inviting!<br><br> <b>[May. 08, 2021]</b> 3 papers (<button onclick="openPaper('DirectPred')">DirectPred</button> <button onclick="openPaper('FewShotNAS')">FewShotNAS</button> <button onclick="openPaper('Learn2Share')">Learn2Share</button>) are accepted in ICML2021!<br><br> <b>[Apr. 29, 2021]</b> 1 papers (<button onclick="openPaper('NeuroPlan')">NeuroPlan</button>) is accepted in SIGCOMM2021!<br><br> <b>[Apr. 21, 2021]</b> Invited talk in VALSE Webinar about understanding self-supervised learning. Slides <a href='valse_seminar_yuandong.pdf'>here</a><br><br> <b>[Apr. 21, 2021]</b> Invited Guest Lecture in UPenn (Thanks <a href='https://www.seas.upenn.edu/~janeli/'>Jing Li</a>) for the invitation. Slides <a href='ml_sys_jing_talk.pdf'>here</a>.<br><br> <b>[Apr. 12, 2021]</b> Invited <a href='https://www.youtube.com/watch?v=sH4a2a0ntUA'>Talk</a> in UCL DARK Lab. Slides <a href='ucl_dark_talk_2021.pdf'>here</a>.<br><br> <b>[Feb. 28, 2021]</b> 2 papers (<button onclick="openPaper('FBNetV3')">FBNetV3</button> <button onclick="openPaper('FPNAS')">FPNAS</button>) are accepted in CVPR2021!<br><br> <font color='red'><b>[Jan. 31, 2021]</b></font> In <a href='https://bbochallenge.com/leaderboard'>Black-box optimization challenge</a> of NeurIPS'20, two teams extended our <button onclick="openPaper('LaMCTS')">LaMCTS</button> and won 3rd and 8th place! See their reports (<a href='https://valohaichirpprod.blob.core.windows.net/papers/jetbrains.pdf'>JetBrains</a>, <a href='https://valohaichirpprod.blob.core.windows.net/papers/kaist_osi.pdf'>KAIST</a>).<br><br> <b>[Jan. 22, 2021]</b> 1 papers (<button onclick="openPaper('RobustTeacherStudent')">RobustTeacherStudent</button>) is accepted in AIStats2021!<br><br> <b>[Dec. 12, 2020]</b> Invited <a href='https://slideslive.com/38938390'>talk</a> in NeurIPS 2020 <a href='https://neurips.cc/virtual/2020/public/workshop_16128.html'>workshop</a> of Learning meets Combinatorial Algorithms.<br><br> <b>[Dec. 12, 2020]</b> Contributed <a href='https://slideslive.com/38941552'>talk</a> in NeurIPS 2020 <a href='https://neurips.cc/virtual/2020/public/workshop_16146.html'>workshop</a> of Self-supervised Learning, Theory and Practice.<br><br> <b>[Nov. 30, 2020]</b> Invited <a href='https://simons.berkeley.edu/talks/tbd-235'>talk</a> (<a href='simon_talk.pdf'>Slides</a>) at <a href='https://simons.berkeley.edu/workshops/rl-2020-3'>Workshop of Reinforcement Learning from Batch Data and Simulation</a> in Simons Institute of UC Berkeley.<br><br> <b>[Oct. 20, 2020]</b> Invited Guest <a href='uwm_talk.pdf'>lecture</a> in University of Wisconsin Madison (Class <a href='http://pages.cs.wisc.edu/~sharonli/courses/cs839_fall2020/schedule.html'>syllabus</a>).<br><br> <b>[Oct. 14, 2020]</b> Distinguished Guest <a href='tsinghua_lecture.pdf'>lecture</a> in IIIS, Tsinghua University.<br><br> <b>[Jun. 06, 2020]</b> Invited guest lecture in UCLA.<br><br> <b>[Nov. 07, 2019]</b> Invited <a href='./ias_talk_Nov_7_public.pdf'>talk</a> in IAS "Workshop on New Directions in Reinforcement Learning and Control" in Princeton University.<br><br> <b>[Nov. 06, 2019]</b> Invited <a href='./nec_talk_Nov_6.pdf'>talk</a> in NEC Laboratories Princeton.<br><br> <b>[Oct. 27, 2019]</b> Invited <a href='./sosp_ai_sys.pdf'>talk</a> in AI Sys Workshop in SOSP'19<br><br> <b>[Jun. 15, 2019]</b> Long oral <a href='./opengo_talk.pdf'>talk</a> about <button onclick="openPaper('OpenGo')">OpenGo</button> in ICML 2019.<br><br> <b>[Jan. 15, 2019]</b> Talks in <a href='./deeplearning_summit.pdf'>Deep Learning Summit</a>, AAAI 2019 Workshops (<a href='./reproducibility.pdf'>Reproducible AI</a> and <a href='./aaai19_workshop_game_environment.pdf'>Game and Environments in Artificial Intelligence</a>).<br><br> <b>[Jun. 01, 2018]</b> Multiple talks in Stanford, AI NextCon, etc. <a href='./presentation2018h1.pdf'>link</a><br><br> <b>[Dec. 20, 2017]</b> Keynote at Future Leaders of AI Retreat (<a href='https://ai-flair.org/program/'>FLAIR</a>), Shanghai. Slides <a href='./flair2017.pdf'>here</a>.<br><br> <b>[Dec. 06, 2017]</b> Oral talk about <button onclick="openPaper('ELF')">ELF</button> platform, NIPS 2017, Long Beach. Slides <a href='./nips17_oral_final.pdf'>link</a>.<br><br> <b>[Nov. 05, 2017]</b> DRL and Game Tutorial in AI Frontier, Santa Clara. Slides <a href='./elf-tutorial/tutorial.html'>link</a>.<br><br> <b>[Oct. 27, 2017]</b> DRL and Game Tutorial in Mountain View, ACMMM 2017. Slides <a href='ACMMM17_tutorial.pdf'>link</a>.<br><br> <b>[Aug. 10, 2017]</b> Presentation in Video Games and Machine Learning <a href='https://syhw.github.io/vgml_workshop_icml2017/index.html'>VGML</a> Workshop, ICML 2017. Slides <a href='icml17_workshop.pdf'>here</a>. The same talk is also presented in University of Sydney on Aug. 11, hosted by Dong Xu. <br><br> <b>[Jul. 15, 2017]</b> On topic "AI In Games: Achievements and Challenges", giving 5 talks in China (CASIA, Tsinghua, Shanghai Tech, Brain-AI workshop and CCF-GAIR 2017) located in Beijing, Shanghai and Shenzhen. Slides <a href='./Yuandong_Jun2017.pdf'>here</a>.<br><br> </div> </section> </td> </tr> </table>  <div id="pubblog" class="tab" width="1400px"> <button class="tablinks" id="allpub" onclick="openAll()">All Publications</button> <button class="tablinks" id="allmajorpub" onclick="openAllMajor()">All Major Publications</button> <button class="tablinks" id="allfirstauthorpub" onclick="openAllFirstAuthor()">All First-author Publications</button> <button class="tablinks" onclick="window.open('novel.html','_self')">Blog & Novels</button> </div> <div id="categories" class="tab" width="1400px"> <button class="tablinks" id="llm-efficiency" onclick="openCat('llm-efficiency')">LLM Efficiency</button> <button class="tablinks" id="llm-agent" onclick="openCat('llm-agent')">LLM Agent</button> <button class="tablinks" id="rl-search" onclick="openCat('rl-search')">Reinforcement Learning and Search</button> <button class="tablinks" id="rep" onclick="openCat('rep')">Understanding Neural Networks</button> <button class="tablinks" id="ai-opt" onclick="openCat('ai-opt')">ML-guided Optimization</button> <button class="tablinks" id="ml-sys" onclick="openCat('ml-sys')">ML+Sys</button> <button class="tablinks" id="other" onclick="openCat('other')">Other Research Topics</button> <button class="tablinks" id="phd-work" onclick="openCat('phd-work')">PhD work</button> </div> <div id="tags" class="tab_hash" width="1400px"> <button class="tablinks" id="rl" onclick="toggleTag('rl')">#rl</button> <button class="tablinks" id="optimization" onclick="toggleTag('optimization')">#optimization</button> <button class="tablinks" id="representation" onclick="toggleTag('representation')">#representation</button> <button class="tablinks" id="llm" onclick="toggleTag('llm')">#llm</button> <button class="tablinks" id="theory" onclick="toggleTag('theory')">#theory</button> <button class="tablinks" id="search" onclick="toggleTag('search')">#search</button> <button class="tablinks" id="transformer" onclick="toggleTag('transformer')">#transformer</button> <button class="tablinks" id="attention" onclick="toggleTag('attention')">#attention</button> <button class="tablinks" id="combinatorial" onclick="toggleTag('combinatorial')">#combinatorial</button> <button class="tablinks" id="ssl" onclick="toggleTag('ssl')">#ssl</button> <button class="tablinks" id="mcts" onclick="toggleTag('mcts')">#mcts</button> <button class="tablinks" id="nonlinear" onclick="toggleTag('nonlinear')">#nonlinear</button> <button class="tablinks" id="latent-action" onclick="toggleTag('latent-action')">#latent-action</button> <button class="tablinks" id="teacher-student" onclick="toggleTag('teacher-student')">#teacher-student</button> <button class="tablinks" id="game" onclick="toggleTag('game')">#game</button> </div> <div class="tabcontent"> <div class="researchtopic" style="padding-left: 30px;"> <table cellspacing="15" width="1400px"> <tr id='Dualformer' catid='rl-search' tags='rl;transformer;search;planning;llm' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2410.09918'><img width=401 src='images/dualformer.png' border='0'></a> </td> <td> <br> <b>Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces</b> [<a href='https://arxiv.org/abs/2410.09918'>link</a>] <br> Andy Su, Sainbayar Sukhbaatar, Michael Rabbat, Yuandong Tian, Qinqing Zheng<br> <br><font color="#00008B">arXiv 2024</font> <br> </td> </tr> <tr id='cogo' catid='rep' tags='theory;representation;feature-emergence;algebra;group' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2410.01779'><img width=401 src='images/cogo2.png' border='0'></a> </td> <td> <br> <b>Composing Global Optimizers to Reasoning Tasks via Algebraic Objects in Neural Nets</b> [<a href='https://arxiv.org/abs/2410.01779'>link</a>] <br> Yuandong Tian<br> <br><font color="#00008B">arXiv 2024</font> <br> </td> </tr> <tr id='attn-transfer' catid='rep' tags='attention;transformer;representation' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>Who Needs Features? On the Surprising Effectiveness of Attention Transfer for Vision Transformers</b> <br> Alexander Cong Li, Yuandong Tian, Beidi Chen, Deepak Pathak, Xinlei Chen<br> <br><font color="#00008B">NeurIPS 2024</font> <br> </td> </tr> <tr id='reverse-curse' catid='rep' tags='theory;attention;transformer;representation' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics</b> [<a href='https://arxiv.org/abs/2405.04669'>link</a>] <br> Hanlin Zhu, Baihe Huang, Shaolun Zhang, Michael Jordan, Jiantao Jiao, Yuandong Tian, Stuart Russell<br> <br><font color="#00008B">NeurIPS 2024</font> <br> </td> </tr> <tr id='PerSE' catid='llm-agent' tags='rl;llm;story-gen' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2310.03304'><img width=401 src='images/perse.jpg' border='0'></a> </td> <td> <br> <b>Learning Personalized Story Evaluation</b> [<a href='https://arxiv.org/abs/2310.03304'>link</a>] [<a href='https://github.com/dqwang122/perse'>code</a>] <br> Danqing Wang, Kevin Yang, Hanlin Zhu, Xiaomeng Yang, Andrew Cohen, Lei Li, Yuandong Tian<br> <br><font color="#00008B">EMNLP 2024</font> <br> </td> </tr> <tr id='TriForce' catid='llm-efficiency' tags='llm;system;speculative-decoding;long-context' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2404.11912'><img width=401 src='images/triforce.png' border='0'></a> </td> <td> <br> <b>TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding</b> [<a href='https://arxiv.org/abs/2404.11912'>link</a>] [<a href='https://github.com/Infini-AI-Lab/TriForce'>code</a>] <br> Hanshi Sun, Zhuoming Chen, Xinyu Yang, Yuandong Tian, Beidi Chen<br> <br><font color="#00008B">COLM 2024</font> <br> </td> </tr> <tr id='SearchFormer' catid='rl-search' tags='rl;transformer;search;planning;llm' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2402.14083'><img width=401 src='images/searchformer.png' border='0'></a> </td> <td> <br> <b>Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping</b> [<a href='https://arxiv.org/abs/2402.14083'>link</a>] <br> Lucas Lehnert, Sainbayar Sukhbaatar, Paul Mcvay, Michael Rabbat, Yuandong Tian<br> <br><font color="#00008B">COLM 2024</font> <br> </td> </tr> <tr id='GaLore' catid='llm-efficiency' tags='llm;low-rank;optimization;system;pre-training' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2403.03507'><img width=401 src='images/galore.png' border='0'></a> </td> <td> <br> <b>GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection</b> [<a href='https://arxiv.org/abs/2403.03507'>link</a>] [<a href='https://github.com/jiaweizzhao/galore'>code</a>] [<a href='https://huggingface.co/blog/galore'>Huggingface blogpost</a>] [<a href='https://x.com/tydsh/status/1766520473029091709'>3-party reproduction</a>] [<a href='https://x.com/tydsh/status/1770920626070667612'>used by startups</a>] <br> Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian<br> <br><font color="#00008B">ICML 2024</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr id='LoCoCo' catid='llm-efficiency' tags='llm;attention;long-context' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2406.05317'><img width=401 src='images/lococo.png' border='0'></a> </td> <td> <br> <b>LoCoCo: Dropping In Convolutions for Long Context Compression</b> [<a href='https://arxiv.org/abs/2406.05317'>link</a>] <br> Ruisi Cai, Yuandong Tian, Zhangyang Wang, Beidi Chen<br> <br><font color="#00008B">ICML 2024</font> <br> </td> </tr> <tr id='MobileLLM' catid='llm-efficiency' tags='llm;system;pre-training' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2402.14905'><img width=401 src='images/mobilellm.png' border='0'></a> </td> <td> <br> <b>MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases</b> [<a href='https://arxiv.org/abs/2402.14905'>link</a>] [<a href='https://github.com/facebookresearch/MobileLLM'>code</a>] [<a href='https://medium.com/@simeon.emanuilov/mobilellm-revolutionizing-on-device-language-models-with-sub-billion-parameters-441ac961f33d'>3rd party blogpost</a>] <br> Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, Vikas Chandra<br> <br><font color="#00008B">ICML 2024</font> <br> </td> </tr> <tr id='TravelPlanner' catid='llm-agent' tags='llm;agent;planning;optimization' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2402.01622'><img width=401 src='images/travelplanner.png' border='0'></a> </td> <td> <br> <b>TravelPlanner: A Benchmark for Real-World Planning with Language Agents</b> [<a href='https://arxiv.org/abs/2402.01622'>link</a>] [<a href='https://github.com/OSU-NLP-Group/TravelPlanner'>code</a>] <br> Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, Yu Su<br> <br><font color="#00008B">ICML 2024</font> <font color='red'><b><i>(Spotlight)</i></b></font><br> </td> </tr> <tr id='ConPas' catid='ai-opt' tags='optimization;combinatorial;ilp;contrastive' major=1 first_author=0> <td width='25%'> <img width=401 src='images/conpas.png' border='0'> </td> <td> <br> <b>Contrastive Predict-and-Search for Mixed Integer Linear Programs</b> <br> Taoan Huang, Aaron M Ferber, Arman Zharmagambetov, Yuandong Tian, Bistra Dilkina<br> <br><font color="#00008B">ICML 2024</font> <br> </td> </tr> <tr id='GenCo' catid='ai-opt' tags='optimization;combinatorial;nonlinear;surrogate;generation' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2310.02442'><img width=401 src='images/GenCo.png' border='0'></a> </td> <td> <br> <b>GenCO: Generating Diverse Solutions to Design Problems with Combinatorial Nature</b> [<a href='https://arxiv.org/abs/2310.02442'>link</a>] <br> Aaron Ferber, Arman Zharmagambetov, Taoan Huang, Bistra Dilkina, Yuandong Tian<br> <br><font color="#00008B">ICML 2024</font> <br> </td> </tr> <tr id='AdvPrompter' catid='ai-opt' tags='optimization;combinatorial;nonlinear;generation;llm;adversarial;robustness' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2404.16873'><img width=401 src='images/AdvPrompter.png' border='0'></a> </td> <td> <br> <b>AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs</b> [<a href='https://arxiv.org/abs/2404.16873'>link</a>] [<a href='https://github.com/facebookresearch/advprompter'>code</a>] [<a href='https://medium.com/@techsachin/advprompter-fast-adaptive-adversarial-prompting-for-llms-red-teaming-testing-8d9e6920912f'>Third-party blogpost</a>] [<a href='https://www.marktechpost.com/2024/05/01/fine-tuning-advprompter-a-novel-ai-method-to-generate-human-readable-adversarial-prompt/'>MarkTechPost</a>] <br> Anselm Paulus*, Arman Zharmagambetov*, Chuan Guo, Brandon Amos**, Yuandong Tian**<br> <font color='#7F7F7F'>(* = Equal 1st authors, ** = Equal advising)</font><br> <br><font color="#00008B">arXiv 2024</font> <br> </td> </tr> <tr id='StreamingLLM' catid='llm-efficiency' tags='attention;transformer;long-context;llm' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2309.17453'><img width=401 src='images/streamingllm.png' border='0'></a> </td> <td> <br> <b>Efficient Streaming Language Models with Attention Sinks</b> [<a href='https://arxiv.org/abs/2309.17453'>link</a>] [<a href='https://github.com/mit-han-lab/streaming-llm'>code</a>] [<a href='https://news.mit.edu/2024/new-way-let-ai-chatbots-converse-all-day-without-crashing-0213'>MIT News</a>] [<a href='https://www.youtube.com/watch?v=409tNlaByds'>Yannic Kilcher's video introduction</a>] [<a href='https://venturebeat.com/ai/streamingllm-shows-how-one-token-can-keep-ai-models-running-smoothly-indefinitely/'>VentureBeat</a>] [<a href='https://github.com/huggingface/transformers/pull/26681'>Huggingface library</a>] [<a href='https://twitter.com/HaihaoShen/status/1715335763032780853'>Intel extension of Transformers</a>] [<a href='https://twitter.com/davidpissarra/status/1735761373261427189'>MLC Chat</a>] [<a href='https://news.ycombinator.com/item?id=37740932'>Y-combinator</a>] <br> Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis<br> <br><font color="#00008B">ICLR 2024</font> <br> </td> </tr> <tr id='RLCD' catid='llm-efficiency' tags='rl;llm;rlhf' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2307.12950'><img width=401 src='images/rlcd.png' border='0'></a> </td> <td> <br> <b>RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment</b> [<a href='https://arxiv.org/abs/2307.12950'>link</a>] [<a href='https://github.com/facebookresearch/rlcd'>code</a>] <br> Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian<br> <br><font color="#00008B">ICLR 2024</font> <br> </td> </tr> <tr id='H-GAP' catid='rl-search' tags='rl;model-based;transformer;latent-action;representation;search' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2312.02682'><img width=401 src='images/h-gap.png' border='0'></a> </td> <td> <br> <b>H-GAP: Humanoid Control with a Generalist Planner</b> [<a href='https://arxiv.org/abs/2312.02682'>link</a>] [<a href='https://yingchenxu.com/hgap/'>website</a>] <br> Zhengyao Jiang*, Yingchen Xu*, Nolan Wagener, Yicheng Luo, Michael Janner, Edward Grefenstette, Tim Rocktaschel, Yuandong Tian<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">ICLR 2024</font> <font color='red'><b><i>(Spotlight)</i></b></font><br> </td> </tr> <tr id='joma' catid='rep' tags='theory;attention;transformer;representation;feature-emergence' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2310.00535'><img width=401 src='images/joma.png' border='0'></a> </td> <td> <br> <b>JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention</b> [<a href='https://arxiv.org/abs/2310.00535'>link</a>] [<a href='https://recorder-v3.slideslive.com/?share=90925&s=1c8d3c17-e60c-43fb-975e-6fd4ca545935'>5min ICLR talk</a>] [<a href='talks/joma_5min_iclr.pdf'>5min ICLR slides</a>] <br> Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Du<br> <br><font color="#00008B">ICLR 2024</font> <br> </td> </tr> <tr id='e2e-story-plot' catid='llm-agent' tags='rl;llm;story-gen' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2310.08796'><img width=401 src='images/e2e-story-plot.png' border='0'></a> </td> <td> <br> <b>End-to-end Story Plot Generator</b> [<a href='https://arxiv.org/abs/2310.08796'>link</a>] <br> Hanlin Zhu*, Andrew Cohen*, Danqing Wang, Kevin Yang, Xiaomeng Yang, Jiantao Jiao, Yuandong Tian<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">arXiv 2023</font> <br> </td> </tr> <tr id='H2O' catid='llm-efficiency' tags='llm;attention;sparsity;long-context' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2306.14048'><img width=401 src='images/h2o.png' border='0'></a> </td> <td> <br> <b>H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models</b> [<a href='https://arxiv.org/abs/2306.14048'>link</a>] [<a href='https://github.com/FMInference/H2O'>code</a>] <br> Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, Beidi Chen<br> <br><font color="#00008B">NeurIPS 2023</font> <br> </td> </tr> <tr id='scan-snap' catid='rep' tags='theory;attention;transformer;representation;feature-emergence' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2305.16380'><img width=401 src='images/scan-snap.png' border='0'></a> </td> <td> <br> <b>Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer</b> [<a href='https://arxiv.org/abs/2305.16380'>link</a>] [<a href='https://icml.cc/virtual/2023/21528'>talk</a>] [<a href='talks/scan_and_snap_workshop.pdf'>slides</a>] [<a href='https://recorder-v3.slideslive.com/?share=90155&s=ab5bb95d-e7f3-4724-8cb2-67b42f65be7e'>5min NeurIPS talk</a>] [<a href='talks/scan_and_snap_neurips_5min.pdf'>5min NeurIPS slides</a>] [<a href='posters/poster_scan_snap.pdf'>poster</a>] <br> Yuandong Tian, Yiping Wang, Beidi Chen, Simon Du<br> <br><font color="#00008B">NeurIPS 2023</font> <br> </td> </tr> <tr id='LANCER' catid='ai-opt' tags='optimization;combinatorial;nonlinear;surrogate' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2307.08964'><img width=401 src='images/lancer.png' border='0'></a> </td> <td> <br> <b>Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information</b> [<a href='https://arxiv.org/abs/2307.08964'>link</a>] [<a href='https://github.com/facebookresearch/LANCER'>code</a>] <br> Arman Zharmagambetov, Brandon Amos, Aaron Ferber, Taoan Huang, Bistra Dilkina, Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2023</font> <br> </td> </tr> <tr id='pos-interp' catid='llm-efficiency' tags='transformer;llm;long-context;attention' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2306.15595'><img width=401 src='images/pos-interp.png' border='0'></a> </td> <td> <br> <b>Extending Context Window of Large Language Models via Positional Interpolation</b> [<a href='https://arxiv.org/abs/2306.15595'>link</a>] <br> Shouyuan Chen, Sherman Wong, Liangjian Chen, Yuandong Tian<br> <br><font color="#00008B">arXiv 2023</font> <br> </td> </tr> <tr id='DOC' catid='llm-agent' tags='llm;story-gen' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2212.10077'><img width=401 src='images/doc-story-gen.png' border='0'></a> </td> <td> <br> <b>DOC: Improving Long Story Coherence With Detailed Outline Control</b> [<a href='https://arxiv.org/abs/2212.10077'>link</a>] [<a href='https://github.com/yangkevin2/doc-story-generation'>code</a>] <br> Kevin Yang, Dan Klein, Nanyun Peng, Yuandong Tian<br> <br><font color="#00008B">ACL 2023</font> <br> </td> </tr> <tr id='DejaVu' catid='llm-efficiency' tags='llm;attention;optimization;sparsity' major=1 first_author=0> <td width='25%'> <a href='https://openreview.net/forum?id=wIPIhHd00i'><img width=401 src='images/dejavu.png' border='0'></a> </td> <td> <br> <b>Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time</b> [<a href='https://openreview.net/forum?id=wIPIhHd00i'>link</a>] <br> Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen<br> <br><font color="#00008B">ICML 2023</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr id='cookbook-ssl' catid='rep' tags='contrastive-learning;representation;ssl' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2304.12210'><img width=401 src='images/cookbook.png' border='0'></a> </td> <td> <br> <b>A Cookbook of Self-Supervised Learning</b> [<a href='https://arxiv.org/abs/2304.12210'>link</a>] <br> Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, Micah Goldblum<br> <br><font color="#00008B">arXiv 2023</font> <br> </td> </tr> <tr id='CL-LNS' catid='ai-opt' tags='optimization;combinatorial;ilp;contrastive' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2302.01578'><img width=401 src='images/CL-LNS.png' border='0'></a> </td> <td> <br> <b>Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning</b> [<a href='https://arxiv.org/abs/2302.01578'>link</a>] <br> Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra Dilkina, Benoit Steiner<br> <br><font color="#00008B">ICML 2023</font> <br> </td> </tr> <tr id='SurCo' catid='ai-opt' tags='optimization;combinatorial;nonlinear' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2210.12547'><img width=401 src='images/SurCo.png' border='0'></a> </td> <td> <br> <b>SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems</b> [<a href='https://arxiv.org/abs/2210.12547'>link</a>] [<a href='https://github.com/facebookresearch/SurCo'>code</a>] [<a href='talks/SurCo_OR2022.pdf'>Slides</a>] <br> Aaron Ferber, Taoan Huang, Daochen Zha, Martin Schubert, Benoit Steiner, Bistra Dilkina, Yuandong Tian<br> <br><font color="#00008B">ICML 2023</font> <font color='red'><b><i>(Outstanding paper in Sampling and Optimization in Discrete Space (SODS) Workshop)</i></b></font><br> </td> </tr> <tr id='CompilerOpt' catid='ml-sys' tags='optimization;compilation;rl;program' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2301.05104'><img width=401 src='images/compiler_opt.png' border='0'></a> </td> <td> <br> <b>Learning Compiler Pass Orders using Coreset and Normalized Value Prediction</b> [<a href='https://arxiv.org/abs/2301.05104'>link</a>] <br> Youwei Liang*, Kevin Stone*, Ali Shameli, Chris Cummins, Mostafa Elhoushi, Jiadong Guo, Benoit Steiner, Xiaomeng Yang, Pengtao Xie, Hugh Leather, Yuandong Tian<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">ICML 2023</font> <br> </td> </tr> <tr id='NeuroShard' catid='ml-sys' tags='rl;ads;application;optimization;surrogate-model' major=0 first_author=0> <td width='25%'> </td> <td> <br> <b>Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models</b> [<a href='https://arxiv.org/abs/2305.01868'>link</a>] [<a href='https://github.com/daochenzha/neuroshard'>code</a>] <br> Daochen Zha, Louis Feng, Liang Luo, Bhargav Bhushanam, Zirui Liu, Yusuo Hu, Jade Nie, Yuzhen Huang, Yuandong Tian, Arun Kejariwal, Xia Hu<br> <br><font color="#00008B">MLSys 2023</font> <br> </td> </tr> <tr id='LB-RELAX' catid='ai-opt' tags='optimization;combinatorial;ilp' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2212.08183'><img width=401 src='images/lb-relax.png' border='0'></a> </td> <td> <br> <b>Local Branching Relaxation Heuristics for Integer Linear Programs</b> [<a href='https://arxiv.org/abs/2212.08183'>link</a>] <br> Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra Dilkina, Benoit Steiner<br> <br><font color="#00008B">CPAIOR 2023</font> <br> </td> </tr> <tr id='TAP' catid='rl-search' tags='rl;model-based;transformer;latent-action;representation;search' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2208.10291'><img width=401 src='images/TAP.png' border='0'></a> </td> <td> <br> <b>Efficient Planning in a Compact Latent Action Space</b> [<a href='https://arxiv.org/abs/2208.10291'>link</a>] [<a href='https://github.com/ZhengyaoJiang/latentplan'>code</a>] [<a href='https://sites.google.com/view/latentplan'>website</a>] <br> Zhengyao Jiang, Tianjun Zhang, Michael Janner, Yueying Li, Tim Rocktaschel, Edward Grefenstette, Yuandong Tian<br> <br><font color="#00008B">ICLR 2023</font> <br> </td> </tr> <tr id='nonlinear-CL' catid='rep' tags='theory;contrastive-learning;representation;nonlinearity;feature-emergence;ssl' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2206.01342'><img width=401 src='images/nonlinear_CL.png' border='0'></a> </td> <td> <br> <b>Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning</b> [<a href='https://arxiv.org/abs/2206.01342'>link</a>] [<a href='https://github.com/facebookresearch/luckmatters/tree/yuandong3/ssl/real-dataset'>code</a>] [<a href='https://openreview.net/pdf?id=vgewUBbTTw2'>workshop version</a>] [<a href='posters/nonlinear_cl_ssl_workshop.pdf'>workshop poster</a>] [<a href='https://recorder-v3.slideslive.com/#/share?share=79199&s=d0eee68e-d8e9-4889-b7c1-55bda5374f42'>5min talk</a>] <br> Yuandong Tian<br> <br><font color="#00008B">ICLR 2023</font> <br> </td> </tr> <tr id='MACTA' catid='ml-sys' tags='rl;multi-agent;transformer;application;cache-timing-attack' major=1 first_author=0> <td width='25%'> <a href='https://openreview.net/forum?id=CDlHZ78-Xzi'><img width=401 src='images/MACTA.png' border='0'></a> </td> <td> <br> <b>MACTA: A Multi-agent Reinforcement Learning Approach for Cache Timing Attacks and Detection</b> [<a href='https://openreview.net/forum?id=CDlHZ78-Xzi'>link</a>] <br> Jiaxun Cui, Xiaomeng Yang*, Geunbae Lee*, Mulong Luo*, Peter Stone, Hsien-Hsin S. Lee, Benjamin Lee, G. Edward Suh, Wenjie Xiong**, Yuandong Tian**<br> <font color='#7F7F7F'>(* = Equal 2nd authors, ** = Equal advising)</font><br> <br><font color="#00008B">ICLR 2023</font> <br> </td> </tr> <tr id='CZP' catid='ai-opt' tags='optimization;pde;antenna;em-wave' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2301.02747'><img width=401 src='images/czp.png' border='0'></a> </td> <td> <br> <b>Modeling Scattering Coefficients using Self-Attentive Complex Polynomials with Image-based Representation</b> [<a href='https://arxiv.org/abs/2301.02747'>link</a>] <br> Andrew Cohen*, Weiping Dou, Jiang Zhu, Slawomir Koziel, Peter Renner, Jan-Ove Mattsson, Xiaomeng Yang, Beidi Chen, Kevin Stone, Yuandong Tian*<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">arXiv 2023</font> <br> </td> </tr> <tr id='AutoCAT' catid='ml-sys' tags='rl;application;cache-timing-attack' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2208.08025'><img width=401 src='images/AutoCAT.png' border='0'></a> </td> <td> <br> <b>AutoCAT: Reinforcement Learning for Automated Exploration of Cache Timing-Channel Attacks</b> [<a href='https://arxiv.org/abs/2208.08025'>link</a>] [<a href='https://github.com/facebookresearch/AutoCAT'>code</a>] <br> Mulong Luo*, Wenjie Xiong*, Geunbae Lee, Yueying Li, Xiaomeng Yang, Amy Zhang, Yuandong Tian, Hsien Hsin S Lee, G Edward Suh<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">HPCA 2023</font> <br> </td> </tr> <tr id='Re3' catid='llm-agent' tags='llm;story-gen' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2210.06774'><img width=401 src='images/re3_story_generation.png' border='0'></a> </td> <td> <br> <b>Re3: Generating Longer Stories With Recursive Reprompting and Revision</b> [<a href='https://arxiv.org/abs/2210.06774'>link</a>] [<a href='https://github.com/yangkevin2/emnlp22-re3-story-generation'>code</a>] <br> Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein<br> <br><font color="#00008B">EMNLP 2022</font> <br> </td> </tr> <tr id='alpha-CL' catid='rep' tags='theory;contrastive-learning;representation;collapsing;ssl' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2201.12680'><img width=401 src='images/alpha_CL.png' border='0'></a> </td> <td> <br> <b>Understanding Deep Contrastive Learning via Coordinate-wise Optimization</b> [<a href='https://arxiv.org/abs/2201.12680'>link</a>] [<a href='https://github.com/facebookresearch/luckmatters/tree/main/ssl/real-dataset'>code</a>] [<a href='https://recorder-v3.slideslive.com/?share=73033&s=a326603a-ca96-41c3-a9b1-4ed90c647633'>video</a>] [<a href='talks/neurips2022_alphaCL_talk.pdf'>5min talk slides</a>] [<a href='posters/alpha_cl_neurips_oral.pdf'>poster</a>] <br> Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2022</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr id='DreamShard' catid='ml-sys' tags='rl;ads;application;optimization;surrogate-model' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2210.02023'><img width=401 src='images/DreamShard.png' border='0'></a> </td> <td> <br> <b>DreamShard: Generalizable Embedding Table Placement for Recommender Systems</b> [<a href='https://arxiv.org/abs/2210.02023'>link</a>] [<a href='https://github.com/daochenzha/dreamshard'>code</a>] <br> Daochen Zha, Louis Feng, Qiaoyu Tan, Zirui Liu, Kwei-Herng Lai, Bhargav Bhushanam, Yuandong Tian, Arun Kejariwal, Xia Hu<br> <br><font color="#00008B">NeurIPS 2022</font> <br> </td> </tr> <tr id='AutoShard' catid='ml-sys' tags='application;ads;rl' major=0 first_author=0> <td width='25%'> </td> <td> <br> <b>AutoShard: Automated Embedding Table Sharding for Recommender Systems</b> [<a href='https://dl.acm.org/doi/abs/10.1145/3534678.3539034'>link</a>] [<a href='https://github.com/daochenzha/autoshard'>code</a>] <br> Daochen Zha, Louis Feng, Bhargav Bhushanam, Dhruv Choudhary, Jade Nie, Yuandong Tian, Jay Chae, Yinbin Ma, Arun Kejariwal, Xia Hu<br> <br><font color="#00008B">KDD 2022</font> <br> </td> </tr> <tr id='DenoisedMDP' catid='rl-search' tags='rl;representation' major=1 first_author=0> <td width='25%'> <a href='https://proceedings.mlr.press/v162/wang22c/wang22c.pdf'><img width=401 src='images/DenoisedMDP.png' border='0'></a> </td> <td> <br> <b>Denoised MDPs: Learning World Models Better Than the World Itself</b> [<a href='https://proceedings.mlr.press/v162/wang22c/wang22c.pdf'>link</a>] [<a href='https://github.com/facebookresearch/denoised_mdp'>code</a>] [<a href='https://www.tongzhouwang.info/denoised_mdp/'>website</a>] <br> Tongzhou Wang, Simon S Du, Antonio Torralba, Phillip Isola, Amy Zhang, Yuandong Tian<br> <br><font color="#00008B">ICML 2022</font> <br> </td> </tr> <tr id='Asym-Siam' catid='rep' tags='representation;ssl' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>On the Importance of Asymmetry for Siamese Representation Learning</b> [<a href='https://arxiv.org/abs/2204.00613'>link</a>] [<a href='https://github.com/facebookresearch/asym-siam'>code</a>] <br> Xiao Wang*, Haoqi Fan*, Yuandong Tian, Daisuke Kihara, Xinlei Chen<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">CVPR 2022</font> <br> </td> </tr> <tr id='DirectCLR' catid='rep' tags='contrastive-learning;representation;collapsing;ssl' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2110.09348'><img width=401 src='images/directclr.png' border='0'></a> </td> <td> <br> <b>Understanding Dimensional Collapse in Contrastive Self-supervised Learning</b> [<a href='https://arxiv.org/abs/2110.09348'>link</a>] [<a href='https://github.com/facebookresearch/directclr'>code</a>] <br> Li Jing, Pascal Vincent, Yann LeCun, Yuandong Tian<br> <br><font color="#00008B">ICLR 2022</font> <br> </td> </tr> <tr id='NASViT' catid='ai-opt' tags='nas;search' major=1 first_author=0> <td width='25%'> <a href='https://openreview.net/forum?id=Qaw16njk6L'><img width=401 src='images/NASViT.png' border='0'></a> </td> <td> <br> <b>NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training</b> [<a href='https://openreview.net/forum?id=Qaw16njk6L'>link</a>] [<a href='https://github.com/facebookresearch/NASViT'>code</a>] <br> Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, Vikas Chandra<br> <br><font color="#00008B">ICLR 2022</font> <br> </td> </tr> <tr id='LaMOO' catid='ai-opt' tags='optimization;search;mcts;latent-action;blackboxopt' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2110.03173'><img width=401 src='images/lamoo.png' border='0'></a> </td> <td> <br> <b>Multi-objective Optimization by Learning Space Partitions</b> [<a href='https://arxiv.org/abs/2110.03173'>link</a>] [<a href='https://github.com/aoiang/LaMOO'>code</a>] <br> Yiyang Zhao, Linnan Wang, Kevin Yang, Tianjun Zhang, Tian Guo, Yuandong Tian<br> <br><font color="#00008B">ICLR 2022</font> <br> </td> </tr> <tr id='DirectSet' catid='rep' tags='non-contrastive-learning;representation;theory;ssl' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2110.04947'><img width=401 src='images/DirectSet.png' border='0'></a> </td> <td> <br> <b>Towards demystifying representation learning with non-contrastive self-supervision</b> [<a href='https://arxiv.org/abs/2110.04947'>link</a>] <br> Xiang Wang, Xinlei Chen, Simon S Du, Yuandong Tian<br> <br><font color="#00008B">arXiv 2021</font> <br> </td> </tr> <tr id='LaNAS' catid='ai-opt' tags='optimization;search;mcts;latent-action;nas;nonlinear' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1906.06832'><img width=401 src='images/nas.png' border='0'></a> </td> <td> <br> <b>Sample-Efficient Neural Architecture Search by Learning Action Space</b> [<a href='https://arxiv.org/abs/1906.06832'>link</a>] [<a href='https://github.com/facebookresearch/LaMCTS'>code</a>] <br> Linnan Wang, Saining Xie, Teng Li, Rodrigo Fonseca, Yuandong Tian<br> <br><font color="#00008B">T-PAMI 2021</font> <br> </td> </tr> <tr id='LSTMvsTransformer' catid='rep' tags='theory;representation;transformer' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2112.09174'><img width=401 src='images/LSTMvsTransformer.png' border='0'></a> </td> <td> <br> <b>Learning Bounded Context-Free-Grammar via LSTM and the Transformer: Difference and Explanations</b> [<a href='https://arxiv.org/abs/2112.09174'>link</a>] [<a href='https://github.com/shihui2010/learn_cfg_with_neural_network'>code</a>] <br> Hui Shi, Sicun Gao, Yuandong Tian, Xinyun Chen, Jishen Zhao<br> <br><font color="#00008B">AAAI 2022</font> <br> </td> </tr> <tr id='CompilerGym' catid='ml-sys' tags='program;compilation;rl' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2109.08267'><img width=401 src='images/CompilerGym.png' border='0'></a> </td> <td> <br> <b>CompilerGym: robust, performant compiler optimization environments for AI research</b> [<a href='https://arxiv.org/abs/2109.08267'>link</a>] [<a href='https://github.com/facebookresearch/CompilerGym'>code</a>] <br> Chris Cummins, Bram Wasti, Jiadong Guo, Brandon Cui, Jason Ansel, Sahir Gomez, Somya Jain, Jia Liu, Olivier Teytaud, Benoit Steiner, Yuandong Tian, Hugh Leather<br> <br><font color="#00008B">CGO 2022</font> <font color='red'><b><i>(Outstanding Paper)</i></b></font><br> </td> </tr> <tr id='BeBold' catid='rl-search' tags='exploration;rl' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2012.08621'><img width=401 src='images/BeBold2.png' border='0'></a> </td> <td> <br> <b>NovelD: A Simple yet Effective Exploration Criterion</b> [<a href='https://arxiv.org/abs/2012.08621'>link</a>] [<a href='https://github.com/tianjunz/NovelD'>code</a>] [<a href='http://yuandong-tian.com/BeBold.mp4'>video</a>] <br> Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2021</font> <br> </td> </tr> <tr id='MADE' catid='rl-search' tags='exploration;rl' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2106.10268'><img width=401 src='images/made.png' border='0'></a> </td> <td> <br> <b>MADE: Exploration via Maximizing Deviation from Explored Regions</b> [<a href='https://arxiv.org/abs/2106.10268'>link</a>] [<a href='https://github.com/tianjunz/MADE'>code</a>] <br> Tianjun Zhang*, Paria Rashidinejad*, Jiantao Jiao, Yuandong Tian, Joseph Gonzalez, Stuart Russell<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">NeurIPS 2021</font> <br> </td> </tr> <tr id='LaP3' catid='ai-opt' tags='optimization;search;mcts;latent-action' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2106.10544'><img width=401 src='images/path-planning.png' border='0'></a> </td> <td> <br> <b>Learning Space Partitions for Path Planning</b> [<a href='https://arxiv.org/abs/2106.10544'>link</a>] [<a href='https://github.com/yangkevin2/plalam'>code</a>] <br> Kevin Yang*, Tianjun Zhang*, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">NeurIPS 2021</font> <br> </td> </tr> <tr id='LaSynth' catid='ml-sys' tags='program;representation;symbolic' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2107.00101'><img width=401 src='images/la-synth.png' border='0'></a> </td> <td> <br> <b>Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages</b> [<a href='https://arxiv.org/abs/2107.00101'>link</a>] [<a href='https://github.com/Jungyhuk/latent-execution'>code</a>] <br> Xinyun Chen, Dawn Song, Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2021</font> <br> </td> </tr> <tr id='DirectPred' catid='rep' tags='non-contrastive-learning;representation;theory;ssl' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2102.06810'><img width=401 src='images/byol_eigenalign.png' border='0'></a> </td> <td> <br> <b>Understanding Self-supervised Learning Dynamics without Contrastive Pairs</b> [<a href='https://arxiv.org/abs/2102.06810'>link</a>] [<a href='https://github.com/facebookresearch/luckmatters/tree/master/ssl'>code</a>] [<a href='https://recorder-v3.slideslive.com/#/share?share=38773&s=9b5d53d7-c27b-4240-8dfc-14c66e297d57'>video</a>] [<a href='https://yuandong-tian.com/directpred_icml_talk.pdf'>Slides</a>] [<a href='https://ai.facebook.com/blog/demystifying-a-key-self-supervised-learning-technique-non-contrastive-learning/'>Blogpost</a>] [<a href='https://openreview.net/forum?id=r4xe3nMQ3AY'>Independent Reproduction</a>] <br> Yuandong Tian, Xinlei Chen, Surya Ganguli<br> <br><font color="#00008B">ICML 2021</font> <font color='red'><b><i>(Outstanding Paper Award Honorable Mention)</i></b></font><br> </td> </tr> <tr id='FewShotNAS' catid='ai-opt' tags='nas;search;optimization' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2006.06863'><img width=401 src='images/few-shot_NAS2.png' border='0'></a> </td> <td> <br> <b>Few-shot Neural Architecture Search</b> [<a href='https://arxiv.org/abs/2006.06863'>link</a>] [<a href='https://github.com/aoiang/few-shot-NAS'>code</a>] [<a href='https://ai.facebook.com/blog/introducing-few-shot-neural-architecture-search/'>Blogpost</a>] <br> Yiyang Zhao*, Linnan Wang*, Yuandong Tian, Rodrigo Fonseca, Tian Guo<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">ICML 2021</font> <font color='red'><b><i>(Long Oral)</i></b></font><br> </td> </tr> <tr id='Learn2Share' catid='ml-sys' tags='application;transformer' major=1 first_author=0> <td width='25%'> <a href='https://proceedings.mlr.press/v139/fu21a.html'><img width=401 src='images/Learn2Share.png' border='0'></a> </td> <td> <br> <b>Learn-to-Share: A Hardware-friendly Transfer Learning Framework Exploiting Computation and Parameter Sharing</b> [<a href='https://proceedings.mlr.press/v139/fu21a.html'>link</a>] <br> Cheng Fu, Hanxian Huang, Xinyun Chen, Yuandong Tian, Jishen Zhao (UCSD)<br> <br><font color="#00008B">ICML 2021</font> <font color='red'><b><i>(Long Oral)</i></b></font><br> </td> </tr> <tr id='NeuroPlan' catid='ai-opt' tags='optimization;rl;ilp' major=0 first_author=0> <td width='25%'> <a href='https://dl.acm.org/doi/10.1145/3452296.3472902'><img width=401 src='images/NeuroPlan.png' border='0'></a> </td> <td> <br> <b>Network Planning with Deep Reinforcement Learning</b> [<a href='https://dl.acm.org/doi/10.1145/3452296.3472902'>link</a>] [<a href='https://github.com/netx-repo/neuroplan'>code</a>] <br> Hang Zhu (JHU), Varun Gupta, Satyajeet Singh Ahuja, Yuandong Tian, Ying Zhang, Xin Jin<br> <br><font color="#00008B">SIGCOMM 2021</font> <br> </td> </tr> <tr id='FBNetV3' catid='ai-opt' tags='nas;search' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2006.02049'><img width=401 src='images/FBNetV3.png' border='0'></a> </td> <td> <br> <b>FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining</b> [<a href='https://arxiv.org/abs/2006.02049'>link</a>] <br> Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, Joseph Gonzalez<br> <br><font color="#00008B">CVPR 2021</font> <br> </td> </tr> <tr id='FPNAS' catid='ai-opt' tags='nas;search' major=0 first_author=0> <td width='25%'> </td> <td> <br> <b>FPNAS: Fast Probabilistic Neural Architecture Search</b> [<a href='https://arxiv.org/abs/2011.10949'>link</a>] <br> Zhicheng Yan, Xiaoliang Dai, Peizhao Zhang, Yuandong Tian, Bichen Wu, Matt Feiszli<br> <br><font color="#00008B">CVPR 2021</font> <br> </td> </tr> <tr id='RobustTeacherStudent' catid='rep' tags='teacher-student;adversarial;theory' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2102.13170'><img width=401 src='images/low_rank_proj.png' border='0'></a> </td> <td> <br> <b>Understanding Robustness in Teacher-Student Setting: A New Perspective</b> [<a href='https://arxiv.org/abs/2102.13170'>link</a>] [<a href='https://yuandong-tian.com/icml2020_talk.pdf'>Slides</a>] <br> Zhuolin Yang*, Zhaoxi Chen, Tiffany Cai, Xinyun Chen, Bo Li, Yuandong Tian*<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">AIStats 2021</font> <br> </td> </tr> <tr id='CollaQ' catid='rl-search' tags='rl;multi-agent' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2010.08531'><img width=401 src='images/collaQ.png' border='0'></a> </td> <td> <br> <b>Multi-Agent Collaboration via Reward Attribution Decomposition</b> [<a href='https://arxiv.org/abs/2010.08531'>link</a>] [<a href='https://github.com/facebookresearch/CollaQ'>code</a>] [<a href='http://yuandong-tian.com/collaQ.mp4'>video</a>] [<a href='https://sites.google.com/view/collaq-starcraft'>website</a>] <br> Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E. Gonzalez, Yuandong Tian<br> <br><font color="#00008B">arxiv 2020</font> <br> </td> </tr> <tr id='JPS' catid='rl-search' tags='rl;multi-agent;imperfect-information;game' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2008.06495'><img width=401 src='images/jps_bridge.png' border='0'></a> </td> <td> <br> <b>Joint Policy Search for Multi-agent Collaboration with Imperfect Information</b> [<a href='https://arxiv.org/abs/2008.06495'>link</a>] [<a href='https://github.com/facebookresearch/jps'>code</a>] [<a href='http://yuandong-tian.com/jps_short_video.mp4'>video</a>] <br> Yuandong Tian, Qucheng Gong, Tina Jiang<br> <br><font color="#00008B">NeurIPS 2020</font> <br> </td> </tr> <tr id='DDCL' catid='rep' tags='contrastive-learning;representation;theory;ssl' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/2010.00578'><img width=401 src='images/ssl.png' border='0'></a> </td> <td> <br> <b>Understanding Self-supervised Learning with Dual Deep Networks</b> [<a href='https://arxiv.org/abs/2010.00578'>link</a>] [<a href='https://github.com/facebookresearch/luckmatters/tree/master/ssl'>code</a>] [<a href='images/understand_simclr2.mp4'>video</a>] <br> Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli<br> <br><font color="#00008B">arXiv 2020</font> <br> </td> </tr> <tr id='StudentSpecialization' catid='rep' tags='teacher-student;theory' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/1909.13458'><img width=401 src='images/catalyst.png' border='0'></a> </td> <td> <br> <b>Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension</b> [<a href='https://arxiv.org/abs/1909.13458'>link</a>] [<a href='https://github.com/facebookresearch/luckmatters'>code</a>] <br> Yuandong Tian<br> <br><font color="#00008B">ICML 2020</font> <br> </td> </tr> <tr id='LTH-RL-NLP' catid='rep' tags='lth' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1906.02768'><img width=401 src='images/rl_nlp.png' border='0'></a> </td> <td> <br> <b>Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP</b> [<a href='https://arxiv.org/abs/1906.02768'>link</a>] <br> Haonan Yu, Sergey Edunov, Yuandong Tian, Ari S. Morcos<br> <br><font color="#00008B">ICLR 2020</font> <br> </td> </tr> <tr id='LaMCTS' catid='ai-opt' tags='optimization;search;mcts;latent-action;blackboxopt;nonlinear' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/2007.00708'><img width=401 src='images/mujoco_experiments.png' border='0'></a> </td> <td> <br> <b>Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search</b> [<a href='https://arxiv.org/abs/2007.00708'>link</a>] [<a href='https://github.com/facebookresearch/LaMCTS'>code</a>] <br> Linnan Wang, Rodrigo Fonseca, Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2020</font> <br> </td> </tr> <tr catid='ai-opt' tags='ads;nas' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction</b> [<a href='https://arxiv.org/abs/2007.06434'>link</a>] <br> Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, Xia Hu<br> <br><font color="#00008B">KDD 2020</font> <br> </td> </tr> <tr id='FBNetV2' catid='ai-opt' tags='nas;search' major=0 first_author=0> <td width='25%'> </td> <td> <br> <b>FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions</b> [<a href='https://arxiv.org/abs/2004.05565'>link</a>] [<a href='https://github.com/facebookresearch/mobile-vision'>code</a>] <br> Alvin Wan, Xiaoliang Dai, Peizhao Zhang, Zijian He, Yuandong Tian, Saining Xie, Bichen Wu, Matthew Yu, Tao Xu, Kan Chen, Peter Vajda, Joseph Gonzalez<br> <br><font color="#00008B">CVPR 2020</font> <br> </td> </tr> <tr id='N-Bref' catid='ml-sys' tags='program;representation;decompilation;symbolic' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>N-Bref : A High-fidelity Decompiler Exploiting Programming Structures</b> [<a href='https://openreview.net/forum?id=6GkL6qM3LV'>link</a>] [<a href='https://github.com/facebookresearch/nbref'>code</a>] [<a href='https://ai.facebook.com/blog/introducing-n-bref-a-neural-based-decompiler-framework/'>Blogpost</a>] <br> Cheng Fu, Kunlin Yang, Xinyun Chen, Yuandong Tian, Jishen Zhao<br> <br><font color="#00008B">arxiv 2020</font> <br> </td> </tr> <tr id='AlphaX' catid='ml-sys' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>AlphaX: Exploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search</b> [<a href='https://arxiv.org/abs/1903.11059'>link</a>] <br> Linnan Wang, Yiyang Zhao, Yuu Jinnai, Yuandong Tian, Rodrigo Fonseca<br> <br><font color="#00008B">AAAI 2020</font> <br> </td> </tr> <tr id='HISS' catid='ml-sys' tags='search;mcts;symbolic;combinatorial' major=1 first_author=0> <td width='25%'> <a href='https://openreview.net/forum?id=r1egIyBFPS'><img width=401 src='images/HISS.png' border='0'></a> </td> <td> <br> <b>Deep Symbolic Superoptimization Without Human Knowledge</b> [<a href='https://openreview.net/forum?id=r1egIyBFPS'>link</a>] [<a href='https://github.com/shihui2010/symbolic_simplifier'>code</a>] <br> Hui Shi, Yang Zhang, Xinyun Chen, Yuandong Tian, Jishen Zhao<br> <br><font color="#00008B">ICLR 2020</font> <br> </td> </tr> <tr id='MiniRTSv2' catid='rl-search' tags='rl;multi-agent;game;nlp' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1906.00744'><img width=401 src='images/MiniRTSv2.png' border='0'></a> </td> <td> <br> <b>Hierarchical Decision Making by Generating and Following Natural Language Instructions</b> [<a href='https://arxiv.org/abs/1906.00744'>link</a>] [<a href='https://github.com/facebookresearch/minirts'>code</a>] <br> Hengyuan Hu*, Denis Yarats*, Qucheng Gong, Yuandong Tian, Mike Lewis<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">NeurIPS 2019</font> <br> </td> </tr> <tr id='OpenGo' catid='rl-search' tags='go;alphazero;mcts;search;elf;game;rl;model-based' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/1902.04522'><img width=401 src='images/elfgo.gif' border='0'></a> </td> <td> <br> <b>ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero</b> [<a href='https://arxiv.org/abs/1902.04522'>link</a>] [<a href='https://github.com/pytorch/ELF'>code</a>] [<a href='https://ai.facebook.com/tools/elf-opengo/'>website</a>] [<a href='https://github.com/pytorch/ELF/releases'>pretrained model and game records</a>] [<a href='https://ai.facebook.com/blog/open-sourcing-new-elf-opengo-bot-and-go-research/'>Blogpost</a>] [<a href='https://yuandong-tian.com/opengo_talk.pdf'>Talk</a>] [<a href='https://www.forbes.com/sites/samshead/2018/05/03/facebook-reveals-elf-bot-code-while-deepminds-alphago-ai-remains-secret/?sh=571cb8c757cf'>Forbes</a>] [<a href='https://techcrunch.com/2018/05/02/facebooks-open-source-go-bot-can-now-beat-professional-players/'>TechCrunch</a>] [<a href='https://www.reddit.com/r/MachineLearning/comments/8gll9t/n_facebook_open_sources_elf_opengo_alphazero/'>reddit</a>] <br> <br>An open source reimplementation of DeepMind's zero-knowledge training and its application to the game of Go. Trained on 2000 GPUs for 9 days. With a single GPU and 50 seconds per move, the model won 20-0 versus 4 top 30 professional players, given human unlimited thinking time. It also won 980-18 versus LeelaZero (version Apr. 25).<br><br> Yuandong Tian, Jerry Ma*, Qucheng Gong*, Shubho Sengupta*, Zhuoyuan Chen, James Pinkerton, Larry Zitnick<br> <font color='#7F7F7F'>(* = Equal 2nd authors)</font><br> <br><font color="#00008B">ICML 2019</font> <font color='red'><b><i>(Long Oral)</i></b></font><br> </td> </tr> <tr catid='rl-search' tags='rl;model-based;theory' major=0 first_author=0> <td width='25%'> <a href='https://openreview.net/forum?id=BJe1E2R5KX'><img width=401 src='images/model-based.png' border='0'></a> </td> <td> <br> <b>Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees</b> [<a href='https://openreview.net/forum?id=BJe1E2R5KX'>link</a>] [<a href='https://github.com/facebookresearch/slbo'>code</a>] <br> Yuping Luo*, Huazhe Xu*, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">ICLR 2019</font> <br> </td> </tr> <tr id='M3RL' catid='rl-search' tags='rl;multi-agent' major=1 first_author=0> <td width='25%'> <a href='https://openreview.net/forum?id=BkzeUiRcY7'><img width=401 src='images/m3rl.png' border='0'></a> </td> <td> <br> <b>M^3RL: Mind-aware Multi-agent Management Reinforcement Learning</b> [<a href='https://openreview.net/forum?id=BkzeUiRcY7'>link</a>] [<a href='https://github.com/facebookresearch/M3RL'>code</a>] <br> Tianmin Shu, Yuandong Tian<br> <br><font color="#00008B">ICLR 2019</font> <br> </td> </tr> <tr id='LuckMatter' catid='rep' tags='theory;teacher-student;lth' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/1905.13405'><img width=401 src='images/luckmatters2.png' border='0'></a> </td> <td> <br> <b>Luck Matters: Understanding Training Dynamics of Deep ReLU Networks</b> [<a href='https://arxiv.org/abs/1905.13405'>link</a>] [<a href='https://github.com/facebookresearch/luckmatters'>code</a>] [<a href='https://yuandong-tian.com/luckmatters_workshop_poster.pdf'>Poster</a>] <br> Yuandong Tian, Tina Jiang, Qucheng Gong, Ari Morcos<br> <br><font color="#00008B">ICML-workshop 2019</font> <br> </td> </tr> <tr id='LTH-Transfer' catid='rep' tags='lth' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1906.02773'><img width=401 src='images/lt_transfer.png' border='0'></a> </td> <td> <br> <b>One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers</b> [<a href='https://arxiv.org/abs/1906.02773'>link</a>] <br> Ari S. Morcos, Haonan Yu, Michela Paganini, Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2019</font> <br> </td> </tr> <tr catid='ai-opt' tags='rl;optimization' major=0 first_author=0> <td width='25%'> </td> <td> <br> <b>Real-world video adaptation with reinforcement learning</b> [<a href='https://arxiv.org/abs/2008.12858'>link</a>] <br> Hongzi Mao, Shannon Chen, Drew Dimmery, Shaun Singh, Drew Blaisdell, Yuandong Tian, Mohammad Alizadeh, Eytan Bakshy<br> <br><font color="#00008B">ICML-Workshop 2019</font> <br> </td> </tr> <tr id='FBNet' catid='ai-opt' tags='nas;search' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1812.03443'><img width=401 src='images/FBNet.png' border='0'></a> </td> <td> <br> <b>FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search</b> [<a href='https://arxiv.org/abs/1812.03443'>link</a>] [<a href='https://github.com/facebookresearch/mobile-vision'>code</a>] <br> Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, Kurt Keutzer<br> <br><font color="#00008B">CVPR 2019</font> <br> </td> </tr> <tr id='NeuRewriter' catid='ai-opt' tags='optimization;rl;combinatorial' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1810.00337'><img width=401 src='images/NeuRewriter.png' border='0'></a> </td> <td> <br> <b>Learning to Perform Local Rewriting for Combinatorial Optimization</b> [<a href='https://arxiv.org/abs/1810.00337'>link</a>] [<a href='https://github.com/facebookresearch/neural-rewriter'>code</a>] <br> Xinyun Chen, Yuandong Tian<br> <br><font color="#00008B">NeurIPS 2019</font> <br> </td> </tr> <tr id='Coda' catid='ml-sys' tags='program;decompilation;symbolic' major=0 first_author=0> <td width='25%'> </td> <td> <br> <b>Coda: An End-to-End Neural Program Decompiler</b> [<a href='https://papers.nips.cc/paper/2019/hash/093b60fd0557804c8ba0cbf1453da22f-Abstract.html'>link</a>] <br> Cheng Fu, Huili Chen, Haolan Liu, Xinyun Chen, Yuandong Tian, Farinaz Koushanfar, Jishen Zhao<br> <br><font color="#00008B">NeurIPS 2019</font> <br> </td> </tr> <tr id='House3D' catid='rl-search' tags='platform;rl' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1801.02209'><img width=401 src='images/house3d.gif' border='0'></a> </td> <td> <br> <b>Building Generalizable Agents with a Realistic and Rich 3D Environment</b> [<a href='https://arxiv.org/abs/1801.02209'>link</a>] [<a href='https://github.com/facebookresearch/House3D'>code</a>] <br> Yi Wu, Yuxin Wu, Georgia Gkioxari, Yuandong Tian<br> <br><font color="#00008B">ICLR-Workshop 2018</font> <br> </td> </tr> <tr catid='rep' tags='theory;teacher-student' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/1809.10829'><img width=401 src='images/theoretical_framework.png' border='0'></a> </td> <td> <br> <b>A Theoretical Framework for Deep Locally Connected ReLU Network</b> [<a href='https://arxiv.org/abs/1809.10829'>link</a>] [<a href='https://yuandong-tian.com/Yuandong_TheoreticalFramework_48x36.pdf'>Poster</a>] <br> Yuandong Tian<br> <br><font color="#00008B">arxiv 2018</font> <br> </td> </tr> <tr id='LocalMinNoFear' catid='rep' tags='theory;teacher-student;gaussian-input' major=0 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1712.00779'><img width=401 src='images/local_minima_failure.png' border='0'></a> </td> <td> <br> <b>Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima</b> [<a href='https://arxiv.org/abs/1712.00779'>link</a>] <br> Simon S. Du, Jason D. Lee, Yuandong Tian, Barnabas Poczos, Aarti Singh<br> <br><font color="#00008B">ICML 2018</font> <font color='red'><b><i>(Long Oral)</i></b></font><br> </td> </tr> <tr id='ConvEasy2Learn' catid='rep' tags='theory;teacher-student;parameterized-input' major=1 first_author=0> <td width='25%'> <a href='https://arxiv.org/abs/1709.06129'><img width=401 src='images/convolutional_filter_easy_to_learn.png' border='0'></a> </td> <td> <br> <b>When is a Convolutional Filter Easy To Learn?</b> [<a href='https://arxiv.org/abs/1709.06129'>link</a>] <br> Simon S. Du, Jason D. Lee, Yuandong Tian<br> <br><font color="#00008B">ICLR 2018</font> <br> </td> </tr> <tr id='ELF' catid='rl-search' tags='platform;rl;game' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/1707.01067'><img width=401 src='images/elf.png' border='0'></a> </td> <td> <br> <b>ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games</b> [<a href='https://arxiv.org/abs/1707.01067'>link</a>] [<a href='https://github.com/facebookresearch/ELF'>code</a>] [<a href='https://www.youtube.com/watch?v=YgZyWobkqfw'>video</a>] <br> Yuandong Tian, Qucheng Gong, Wenling Shang, Yuxin Wu, Larry Zitnick<br> <br><font color="#00008B">NeurIPS 2017</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr id='F1' catid='rl-search' tags='rl;game' major=1 first_author=0> <td width='25%'> <a href='http://openreview.net/pdf?id=Hk3mPK5gg'><img width=401 src='images/doom_screen_shot.jpg' border='0'></a> </td> <td> <br> <b>Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning</b> [<a href='http://openreview.net/pdf?id=Hk3mPK5gg'>link</a>] <br> Yuxin Wu, Yuandong Tian<br> <br><font color="#00008B">ICLR 2017</font> <br> </td> </tr> <tr id='ReLUAnalytic' catid='rep' tags='theory;teacher-student;gaussian-input' major=1 first_author=1> <td width='25%'> <a href='https://arxiv.org/abs/1703.00560'><img width=401 src='images/symmetry-breaking.jpg' border='0'></a> </td> <td> <br> <b>An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis</b> [<a href='https://arxiv.org/abs/1703.00560'>link</a>] [<a href='https://github.com/yuandong-tian/ICML17_ReLU'>code</a>] <br> Yuandong Tian<br> <br><font color="#00008B">ICML 2017</font> <br> </td> </tr> <tr catid='other' tags='cv;segmentation;dataset' major=1 first_author=0> <td width='25%'> <a href='http://arxiv.org/abs/1509.01329'><img width=401 src='images/amodal.png' border='0'></a> </td> <td> <br> <b>Semantic Amodal Segmentation</b> [<a href='http://arxiv.org/abs/1509.01329'>link</a>] <br> Yan Zhu, Yuandong Tian, Dimitris Mexatas, Piotr Doll谩r<br> <br><font color="#00008B">CVPR 2017</font> <br> </td> </tr> <tr id='DarkForest' catid='rl-search' tags='rl;game;go;mcts;search' major=1 first_author=1> <td width='25%'> <a href='http://arxiv.org/abs/1511.06410'><img width=401 src='images/go.png' border='0'></a> </td> <td> <br> <b>Better Computer Go Player with Neural Network and Long-term Prediction</b> [<a href='http://arxiv.org/abs/1511.06410'>link</a>] [<a href='https://github.com/facebookresearch/darkforestGo'>code</a>] [<a href='https://www.dropbox.com/sh/6nm8g8z163omb9f/AABQxJyV7EIdbHKd9rnPQGnha?dl=0'>pretrained model</a>] [<a href='https://www.technologyreview.com/2015/12/04/164717/how-facebooks-ai-researchers-built-a-game-changing-go-engine/'>mit tech review</a>] [<a href='https://www.wired.com/2015/11/facebook-is-aiming-its-ai-at-go-the-game-no-computer-can-crack/'>wired</a>] <br> Yuandong Tian, Yan Zhu<br> <br><font color="#00008B">ICLR 2016</font> <br> </td> </tr> <tr catid='other' tags='cv;3d-reconstruction' major=0 first_author=0> <td width='25%'> <a href='http://arxiv.org/abs/1604.08685'><img width=401 src='images/single3d.jpg' border='0'></a> </td> <td> <br> <b>Single Image 3D Interpreter Network</b> [<a href='http://arxiv.org/abs/1604.08685'>link</a>] <br> Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman<br> <font color='#7F7F7F'>(* = Equal 1st authors)</font><br> <br><font color="#00008B">ECCV 2016</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr catid='other' tags='vqa;baselines' major=1 first_author=0> <td width='25%'> <a href='http://arxiv.org/abs/1512.02167'><img width=401 src='images/simple-baseline.jpg' border='0'></a> </td> <td> <br> <b>Simple Baseline for Visual Question Answering</b> [<a href='http://arxiv.org/abs/1512.02167'>link</a>] [<a href='https://github.com/zhoubolei/VQAbaseline'>code</a>] <br> Bolei Zhou, Yuandong Tian, Sainbayar Sukhbaatar, Arthur Szlam, Rob Fergus<br> <br><font color="#00008B">arxiv 2016</font> <br> </td> </tr> <tr catid='phd-work' tags='optimization;cv' major=1 first_author=1> <td width='25%'> </td> <td> <br> <b>Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation</b> [<a href='http://link.springer.com/article/10.1007/s11263-015-0838-5?wt_mc=email.event.1.SEM.ArticleAuthorOnlineFirst'>link</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan<br> <br><font color="#00008B">IJCV 2015</font> <br> </td> </tr> <tr catid='phd-work' tags='thesis' major=1 first_author=1> <td width='25%'> </td> <td> <br> <b>Theory and Practice of Globally Optimal Deformation Estimation</b> [<a href='https://yuandong-tian.com/mythesis8.pdf'>link</a>] <br> Yuandong Tian<br> <br><font color="#00008B">PhD thesis 2013</font> <br> </td> </tr> <tr id='HierarchicalDataDrivenDescent' catid='phd-work' tags='optimization;cv' major=1 first_author=1> <td width='25%'> <a href='https://www.cs.cmu.edu/~ILIM/publications/PDFs/TN-ICCV13.pdf'><img width=401 src='images/hddd.png' border='0'></a> </td> <td> <br> <b>Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation</b> [<a href='https://www.cs.cmu.edu/~ILIM/publications/PDFs/TN-ICCV13.pdf'>link</a>] [<a href='https://yuandong-tian.com/iccv13_tech_report.pdf'>Proofs</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan<br> <br><font color="#00008B">ICCV 2013</font> <font color='red'><b><i>(Marr Prize Honorable Mentions)</i></b></font><br> </td> </tr> <tr catid='phd-work' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>Integrating Perceptual Learning with External World Knowledge in a Simulated Student</b> [<a href='https://www.cs.cmu.edu/~wcohen/postscript/aied-2013.pdf'>link</a>] <br> Nan Li, Yuandong Tian, William W. Cohen, Ken Koedinger<br> <br><font color="#00008B">AIED 2013</font> <br> </td> </tr> <tr catid='phd-work' tags='pose-estimation;cv' major=1 first_author=1> <td width='25%'> <a href='https://yuandong-tian.com/eccv_pose_est_camera_ready.pdf'><img width=401 src='images/humanpose-est.jpg' border='0'></a> </td> <td> <br> <b>Exploring the Spatial Hierarchy of Mixture Models for Human Pose Estimation</b> [<a href='https://yuandong-tian.com/eccv_pose_est_camera_ready.pdf'>link</a>] [<a href='https://yuandong-tian.com/pose_est_hier.tar'>code</a>] [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/humanpose/humanpose.html'>website</a>] <br> Yuandong Tian, Larry Zitnick, Srinivasa G. Narasimhan<br> <br><font color="#00008B">ECCV 2012</font> <br> </td> </tr> <tr catid='phd-work' tags='optimization' major=1 first_author=1> <td width='25%'> </td> <td> <br> <b>Learning from Crowds in the Presence of Schools of Thought</b> [<a href='https://yuandong-tian.com/kdd2012-tian.pdf'>link</a>] [<a href='https://yuandong-tian.com/kdd2012-code.zip'>code</a>] [<a href='https://yuandong-tian.com/kdd2012-presentation.pdf'>Slides</a>] [<a href='https://yuandong-tian.com/kdd2012-dataset.zip'>Dataset</a>] <br> Yuandong Tian, Jun Zhu<br> <br><font color="#00008B">KDD 2012</font> <br> </td> </tr> <tr catid='phd-work' tags='physics;optimization;cv' major=1 first_author=1> <td width='25%'> <a href='http://www.cs.cmu.edu/~ILIM/projects/IM/turbulence/cvpr2012_YuandongTian.pdf'><img width=401 src='images/principle_turbulence.png' border='0'></a> </td> <td> <br> <b>Depth from Optical Turbulence</b> [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/turbulence/cvpr2012_YuandongTian.pdf'>link</a>] [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/turbulence/turbulence.html'>website</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan, Alan J. Vannevel<br> <br><font color="#00008B">CVPR 2012</font> <br> </td> </tr> <tr catid='phd-work' tags='physics;cv' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>A Combined Theory of Defocused Illumination and Global Light Transport</b> [<a href='http://www.cs.cmu.edu/~ILIM/publications/PDFs/GTNZ-IJCV11.pdf'>link</a>] <br> Mohit Gupta, Yuandong Tian, Srinivasa G. Narasimhan, Li Zhang<br> <br><font color="#00008B">IJCV 2011</font> <br> </td> </tr> <tr catid='phd-work' tags='optimization;cv' major=1 first_author=1> <td width='25%'> </td> <td> <br> <b>Globally Optimal Estimation of Nonrigid Image Distortion</b> [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/globalopt/paper-ijcv-yuandong.pdf'>link</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan<br> <br><font color="#00008B">IJCV 2011</font> <br> </td> </tr> <tr catid='phd-work' tags='optimization;ocr' major=1 first_author=1> <td width='25%'> <a href='http://www.cs.cmu.edu/~ILIM/projects/IM/document_rectification/cvpr2011_YuandongTian.pdf'><img width=401 src='images/document_thumb.png' border='0'></a> </td> <td> <br> <b>Rectification and 3D reconstruction of Curved Document Images</b> [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/document_rectification/cvpr2011_YuandongTian.pdf'>link</a>] [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/document_rectification/document_rectification.html'>website</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan<br> <br><font color="#00008B">CVPR 2011</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr catid='phd-work' tags='kernel' major=1 first_author=0> <td width='25%'> </td> <td> <br> <b>Local Isomorphism to Solve the Pre-image Problem in Kernel Methods</b> [<a href='https://ieeexplore.ieee.org/document/5995685'>link</a>] <br> Dong Huang, Yuandong Tian, Fernando De la Torre<br> <br><font color="#00008B">CVPR 2011</font> <br> </td> </tr> <tr catid='phd-work' tags='optimization;cv' major=1 first_author=1> <td width='25%'> <a href='http://www.cs.cmu.edu/~ILIM/projects/IM/globalopt/cvpr10_globalopt.pdf'><img width=401 src='images/globalopt.png' border='0'></a> </td> <td> <br> <b>A Globally Optimal Data-Driven Approach for Image Distortion Estimation</b> [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/globalopt/cvpr10_globalopt.pdf'>link</a>] [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/globalopt/research_globalopt.html'>website</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan<br> <br><font color="#00008B">CVPR 2010</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> <tr catid='phd-work' tags='optimization;cv;physics' major=1 first_author=1> <td width='25%'> <a href='http://www.cs.cmu.edu/~ILIM/projects/IM/water/water.pdf'><img width=401 src='images/waterproj.png' border='0'></a> </td> <td> <br> <b>Seeing through water: Image restoration using model-based tracking</b> [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/water/water.pdf'>link</a>] [<a href='http://www.cs.cmu.edu/~ILIM/projects/IM/water/research_water.html'>website</a>] <br> Yuandong Tian, Srinivasa G. Narasimhan<br> <br><font color="#00008B">ICCV 2009</font> <br> </td> </tr> <tr catid='phd-work' tags='physics' major=1 first_author=0> <td width='25%'> <a href='https://yuandong-tian.com/defocus.pdf'><img width=401 src='images/defocus.png' border='0'></a> </td> <td> <br> <b>(De) Focusing on Global Light Transport for Active Scene Recovery</b> [<a href='https://yuandong-tian.com/defocus.pdf'>link</a>] <br> Mohit Gupta, Yuandong Tian, Srinivasa G. Narasimhan, Li Zhang<br> <br><font color="#00008B">CVPR 2009</font> <font color='red'><b><i>(Oral)</i></b></font><br> </td> </tr> </table> </div> </div> <script> openAll() </script> </body> </html>