Wei-Chiu Ma, Cornell

<!doctype html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Wei-Chiu Ma, Cornell</title> <style> h1 { padding : 0; margin : 0; } body { padding : 20px 0; font-family : Arial; font-size : 16px; background-image : url("img/bg.png"); } #container { width : 900px; margin : 0 auto; background-color : #fff; padding : 50px; text-align: left; box-shadow: 0px 0px 10px #999;; } #me { margin-left : 25px; border : 0 solid black; float : right; margin-bottom : 0; margin-right:25px;} #content { display : block; margin-right : 275px;} a { text-decoration : none; } a:hover { text-decoration : underline; } a:link,a:visited {color: #1367a7;} a.invisible { color : inherit; text-decoration : inherit; } .publogo { margin-top : 20px; margin-right : 10px; float : left; border : 0; width: 200px; vertical-align: middle;} .publication { clear : left; padding-bottom : 10px; line-height:22px;} .codelogo { margin-right : 10px; float : left; border : 0;} .code { clear : left; padding-bottom : 10px; vertical-align :middle;} .code .download a { display : block; margin : 0 15px; float : left;} #simpsons { margin : 5px auto; text-align : center; color : #B7B7B7; } span.highlight, span.highlight a:link, span.highlight a:visited{color:#BB2222} span.collaborator, span.collaborator a:link, span.collaborator a:visited{color:#666666} </style> <script type="text/javascript"> function showPubs(id) { if (id == 0) { document.getElementById('pubs').innerHTML = document.getElementById('pubs_selected').innerHTML; document.getElementById('select0').style = 'text-decoration:underline;color:#000000'; document.getElementById('select1').style = ''; // document.getElementById('select2').style = ''; } else if (id == 1) { document.getElementById('pubs').innerHTML = document.getElementById('pubs_by_date').innerHTML; document.getElementById('select1').style = 'text-decoration:underline;color:#000000'; document.getElementById('select0').style = ''; // document.getElementById('select2').style = ''; } // else { // document.getElementById('pubs').innerHTML = document.getElementById('pubs_by_topic').innerHTML; // document.getElementById('select2').style = 'text-decoration:underline;color:#000000'; // document.getElementById('select0').style = ''; // document.getElementById('select1').style = ''; // } } </script> </head> <body>  <center> <div id="container">  <img src="img/weichiu-mit-min.PNG" align = "right" width = 200> <div id="content"> <h1>Wei-Chiu Ma 馬惟九</h1>   Assistant Professor @ Cornell CS   <a href="mailto:wm347@cornell.edu">Email</a> / <a href="https://scholar.google.com/citations?user=SVIdh6AAAAAJ&hl=en#">Google scholar</a> / <a href="https://twitter.com/weichiuma">Twitter</a> / <a href="#misc">Misc.</a> / <a href="#pro-bono">Pro Bono</a> <h2>About Me</h2>   I am an Assistant Professor of Computer Science at <a href="http://cornell.edu">Cornell University</a>.  My research lies at the intersection of 3D/4D computer vision and robotics. I am interested in building AI systems that can understand, reconstruct, and re-simulate our dynamic world, leveraging these capabilities to enable more robust autonomous systems or advance entertainment applications.      Prior to joining Cornell, I was a Young Investigator/Postdoc at <a href="https://allenai.org">AI2</a>/<a href="https://www.washington.edu">University of Washington</a>. I received my Ph.D. from <a href="http://web.mit.edu">MIT</a>, where I worked with <a href="http://web.mit.edu/torralba/www/">Antonio Torralba</a> (aka <a href="http://bit.ly/the-great-torralba">the Great Torralba</a>) and <a href="http://www.cs.toronto.edu/~urtasun/">Raquel Urtasun</a>. Previously, I was a Senior Research Scientist at <a href="https://eng.uber.com/tag/uber-atg-toronto/">Uber ATG R&D</a> and <a href="https://waabi.ai">Waabi</a> working on self-driving vehicles. I completed my M.S. in Robotics at <a href="http://www.cs.cmu.edu/">Carnegie Mellon University (CMU)</a>, where I was advised by <a href="http://www.cs.cmu.edu/~kkitani/">Kris M. Kitani</a>. Prospective students: I am always looking for motivated and talented students! If you are interested in collaborating or joining my group as a PhD/MS/Undergrad student or intern, please read <a href="prospective.html">this</a>.     <center> <table border="0" cellpadding="0" cellspacing="0" width="900" align="center" bgcolor="#FFFFFF"> <tr> <td width="20"></td>    <td width="120" align="center" valign="middle"><img src="img/logo_mit.gif" height="50" /></td> <td width="120" align="center" valign="middle"><img src="img/waabi.gif" height="80" /></td> <td width="120" align="center" valign="middle"><img src="img/logo_uber_atg.jpg" height="70" /></td> <td width="120" align="center" valign="middle"><img src="img/logo_cmu_1.png" height="70" /></td> <td width="120" align="center" valign="middle"><img src="img/logo_uoft.jpg" height="80" /></td> <td width="120" align="center" valign="middle"><img src="img/logo_ntu.jpg" height="80" /></td> <td width="20"></td> </tr>  <tr> <td colspan = 7>   <h2>Recent News</h2> <ul>  <li>NEW Pro bono: I will be hosting pro bono office hours starting 2021. Please check out <a href="#pro-bono">here</a> for more details.</li>  <li>NEW CVPR Workshop: We will be organizing the second workshop on<a href="https://syndata4cv.github.io/" target=""> Synthetic Data for Computer Vision</a> at CVPR 2025. Stay tuned for more details!</li> <li>NEW CVPR Workshop: We will be organizing the first workshop on<a href="" target=""> Agent in Interaction, from Humans to Robots</a> at CVPR 2025. Stay tuned for more details!</li>  <li>NEW Dec. 2024: We released the 360-1M dataset! Check them out <a href="https://github.com/MattWallingford/360-1M" target="">here</a>! <li>NEW Dec. 2024: Can AI system generate 3D worlds from a single image? Yes! <a href="https://mattwallingford.github.io/ODIN/" target="">All you need is the RIGHT data!</a> <li>NEW Oct. 2024: We organized the <a href="https://3d-in-the-wild.github.io/" target="">3D Modeling, Reconstruction, and Generation in the Wild</a> workshop at ECCV 2024!</li> <li>Jul. 2024: Can Multimodal LLMs (GPT-4V) perceive the world as humans do? See our <a href="https://zeyofu.github.io/blink/" target="">ECCV paper</a> for answers!</li> <li>Apr. 2024: How can synthetic data benefit computer vision? Attend our CVPR <a href="https://syndata4cv.github.io/" target="">SynData4CV</a> workshop to find out!</li> <li>Mar. 2024: Check out our latest effort on building <a href="https://video2game.github.io/" target="">realistic, interactive, and game-engine compatible digital twins</a>!</li> <li>Sep. 2023: Two papers on in-the-wild/extreme 3D inverse graphics accepted to NeurIPS 2023!</li> <li>Sep. 2023: I am now Dr. Ma! Thank you, Antonio and Raquel, for your guidance and tremendous support! </li>  <li>Apr. 2023: Our work on <a href='https://virtual-correspondence.github.io'>extreme-view geometry</a> is featured on <a href='https://youtu.be/y2BVTW09vck?t=360'>Vox</a>! Check it out!</li> <li>Apr. 2023: Selected as a <a href='https://risingstars.linklab.virginia.edu/2023/'>Cyber-Physical Systems (CPS) rising star</a>!</li> <li>Mar. 2023: Check out our latest effort on closed-loop sensor simulation, LiDAR generation, and thermal imaging!</li> <li>Oct. 2022: Gave a talk at CMU, Columbia, and UIUC on in-the-wild 3D modeling, generation, and simulation!</li> <li>Sep. 2022: Check out our latest effort on 3D scene generation, sensor simulation, and neural fields for manipulation!</li> <li>Aug. 2022: Selected as a <a href="https://www.businesswire.com/news/home/20220922005006/en/Siebel-Scholars-Foundation-Announces-Class-of-2023">Siebel Scholar</a>!</li> <li> Apr. 2022: Gave a talk at Harvard on exploiting high-level vision for level-vision!</li> <li> Mar. 2022: Our work on extreme-view 3D reconsturction and planar neural field are accepted to CVPR 2022!</li> <li> Jul. 2021: BARF! Train your own NeRF from a collection of images without knowing camera poses!</li> <li> Mar. 2021: Check out our latest effort on simulating pedestrians in the wild at CoRL and CVPR!</li> <li> Jul. 2020: Four papers (two spotlight) accepted to ECCV 2020! Stay tuned!</li> <li> Mar. 2020: Our work on LiDAR simulation and instance segmentation are accepted to CVPR 2020!</li> <li> Oct. 2019: The source code of our real-time stereo algorithm is available now! Make sure to check out the amzaing differentiable PatchMatch module!</li> <li> Jul. 2019: Three papers accepted to ICCV 2019! Details coming soon!</li> <li> Jun. 2019: Our paper on light-weight localization is accepted to IROS 2019!</li> <li> Mar. 2019: Two papers (scene flow and road boundary extraction) accepted to CVPR 2019!</li> <li> Jul. 2018: Our paper on Unsupervised Intrinsic Decomposition is accepted to ECCV 2018!</li> <li>Mar. 2018: 3 papers accepted to CVPR 2018!</li> <li> Apr. 2017: Our book chapter on Activity Forecasting is published! Check it out!</li> <li>Mar. 2017: Our work (a game-theoretic approach to multi-agent activity forecasting) is accepted to CVPR 2017!</li> <li>Jan. 2017: Our work (self-localization for autonomous vehicles) is accepted to ICRA 2017!</li>  <li>May. 2016: I graduated from CMU --- thanks Kris for all your support in the past two years!</li> </ul> </td> </tr> <tr> <td colspan = 7>   <h2 style="display:inline;">Research Projects</h2> &nbsp (<a href="" id="select0" onclick="showPubs(0); return false;">show selected</a> / <a href="" id="select1" onclick="showPubs(1); return false;">show by date</a>)   <table border="0" cellpadding="0" cellspacing="0" width="900" align="center" bgcolor="#FFFFFF" id="pubs"></table> <script id="pubs_selected" language="text">  <tr> <td> <img border=0 src="img/odin.gif" class="publogo"> </td> <td> <div class="publication"> From an Image to a Scene: Learning to Imagine the World from a Million 360° Videos Matthew Wallingford, Anand Bhattad, Aditya Kusupati, Vivek Ramanujan, Matt Deitke, Sham Kakade, Aniruddha Kembhavi, Roozbeh Mottaghi, Wei-Chiu Ma, Ali Farhadi <a href="" target="">NeurIPS 2024</a> / <a href="https://mattwallingford.github.io/ODIN/" target="">project page</a> / <a href="https://arxiv.org/pdf/2412.07770" target="">arXiv</a> / <a href="https://huggingface.co/datasets/mwallingford/360-1M/tree/main" target="">huggingface</a> </div> </td> </tr>    <tr> <td> <img border=0 src="img/blink.png" class="publogo"> </td> <td> <div class="publication"> BLINK: Multimodal Large Language Models Can See but Not Perceive Xingyu Fu*, Yushi Hu*, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma†, Ranjay Krishna† <a href="" target="">ECCV 2024</a> / <a href="https://zeyofu.github.io/blink/" target="">project page</a> / <a href="https://arxiv.org/pdf/2404.12390" target="">paper</a> / <a href="https://huggingface.co/datasets/BLINK-Benchmark/BLINK" target="">dataset</a> / <a href="https://eval.ai/web/challenges/challenge-page/2287/overview" target="">Eval AI</a> / <a href="https://github.com/zeyofu/BLINK_Benchmark" target="">code</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/video2game.gif" class="publogo"> </td> <td> <div class="publication"> Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video Hongchi Xia, Zhi-Hao Lin, Wei-Chiu Ma, Shenlong Wang <a href="" target="">CVPR 2024</a> / <a href="https://video2game.github.io/" target="">project page</a> / <a href="https://arxiv.org/pdf/2404.09833" target="">paper</a> / <a href="https://video2game.github.io/src/garden/index.html" target="">shooting demo</a> / <a href="https://github.com/video2game/video2game" target="">code</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/unisim.mp4" class="publogo"> </td> <td> <div class="publication"> UniSim: A Neural Closed-Loop Sensor Simulator Ze Yang*, Yun Chen*, Jingkang Wang*, Sivabalan Manivasagam*, Wei-Chiu Ma, Joyce Anqi Yang, Raquel Urtasun <a href="" target="">CVPR 2023</a> / <a href="https://waabi.ai/unisim/" target="">project page</a> / <a href="https://openaccess.thecvf.com/content/CVPR2023/papers/Yang_UniSim_A_Neural_Closed-Loop_Sensor_Simulator_CVPR_2023_paper.pdf" target="">paper</a> / <a href="http://www.cs.toronto.edu/~wangjk/publications/unisim/unisim_final_v2_4k.mp4" target="">4K demo</a> / <a href="https://waabi.ai/wp-content/uploads/2023/05/UniSim-video_compressed.mp4" target="">video (8 mins)</a> Highlight presentation </div> </td> </tr>  <tr> <td> <img border=0 src="img/lidar-gen.mp4" class="publogo"> </td> <td> <div class="publication"> UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang, Raquel Urtasun <a href="" target="">CVPR 2023</a> / <a href="https://waabi.ai/ultralidar/" target="">project page</a> / <a href="https://openaccess.thecvf.com/content/CVPR2023/papers/Xiong_Learning_Compact_Representations_for_LiDAR_Completion_and_Generation_CVPR_2023_paper.pdf" target="">paper</a> / <a href="https://waabi.ai/wp-content/uploads/2023/05/UltraLidar-video.mov" target="">video (1 min)</a> </div> </td> </tr>  <tr> <td> <img border=0 src="papers/neurips22-sgam/sgam.gif" class="publogo"> </td> <td> <div class="publication"> SGAM: Building a Virtual 3D World through Simultaneous Generation and Mapping Yuan Shen, Wei-Chiu Ma, Shenlong Wang <a href="" target="">NeurIPS 2022</a> / <a href="https://yshen47.github.io/sgam/" target="">project page</a> / <a href="https://openreview.net/pdf?id=17KCLTbRymw" target="">paper</a> / <a href="https://github.com/yshen47/SGAM" target="">code</a> </div> </td> </tr>  <tr> <td> <img border=0 src="papers/corl22-cad-sim/cad-sim.gif" class="publogo"> </td> <td> <div class="publication"> CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Realistic and Controllable Sensor Simulation Jingkang Wang, Sivabalan Manivasagam, Yun Chen, Ze Yang, Ioan Andrei Bârsan, Joyce Anqi Yang, Wei-Chiu Ma, Raquel Urtasun <a href="" target="">CoRL 2022</a> / <a href="http://www.cs.toronto.edu/~wangjk/publications/cadsim.html" target="">project page</a> / <a href="https://openreview.net/pdf?id=Mp3Y5jd7rnW" target="">paper</a> / <a href="http://www.cs.toronto.edu/~wangjk/publications/cadsim/cadsim_corl_denoise.mp4" target="">video</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/vc-3.png" class="publogo"> </td> <td> <div class="publication"> Virtual Correspondence: Humans as a Cue for Extreme-View Geometry Wei-Chiu Ma, Anqi Joyce Yang, Shenlong Wang, Raquel Urtasun, Antonio Torralba <a href="" target="_new">CVPR 2022</a> / <a href="virtual-correspondence" target="_new">project page</a> / <a href="https://arxiv.org/pdf/2206.08365.pdf" target="_new">paper</a> / <a href="https://virtual-correspondence.github.io/img/vc_demo.mp4" target="_new">video (1.5 mins)</a> / <a href="https://youtu.be/W9odd2F2Bx4" target="_new">video (5 mins)</a> / <a href="https://news.mit.edu/2022/seeing-whole-from-some-parts-0617" target="_new">MIT News</a> / <a href="https://techxplore.com/news/2022-06-vision-technique-3d-2d-images.html" target="_new">TechXplore</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/neurmips.gif" class="publogo"> </td> <td> <div class="publication"> NeurMiPs: Neural Mixture of Planar Experts for View Synthesis Zhi-Hao Lin, Wei-Chiu Ma, Hao-Yu Max Hsu, Yu-Chiang Frank Wang, Shenlong Wang <a href="" target="_new">CVPR 2022</a> / <a href="https://zhihao-lin.github.io/neurmips/" target="_new">project page</a> / <a href="https://openaccess.thecvf.com/content/CVPR2022/papers/Lin_NeurMiPs_Neural_Mixture_of_Planar_Experts_for_View_Synthesis_CVPR_2022_paper.pdf" target="_new">paper</a> / <a href="https://github.com/zhihao-lin/neurmips" target="_new">code</a> / <a href="https://www.youtube.com/watch?v=PV1dCTWL5Oo" target="_new">video</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/barf.gif" class="publogo"> </td> <td> <div class="publication"> BARF: Bundle-Adjusting Neural Radiance Fields Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, Simon Lucey <a href="" target="_new">ICCV 2021</a> / <a href="https://chenhsuanlin.bitbucket.io/bundle-adjusting-NeRF/" target="_new">project page</a> / <a href="https://arxiv.org/pdf/2104.06405.pdf" target="_new">arXiv</a> / <a href="https://github.com/chenhsuanlin/bundle-adjusting-NeRF" target="_new">code</a> / <a href="https://www.youtube.com/watch?v=dCmCZs2Hpi0" target="_new">video</a> / <a href="https://read.deeplearning.ai/the-batch/issue-95#3d-scene-synthesis-for-the-real-world" target="_new">news coverage (The Batch: DeepLearning.AI)</a> Oral presentation </div> </td> </tr>  <tr> <td> <img border=0 src="img/deep-optimizer-teaser.gif" class="publogo"> </td> <td> <div class="publication"> Deep Feedback Inverse Problem Solver Wei-Chiu Ma, Shenlong Wang, Jiayuan Gu, Sivabalan Manivasagam, Antonio Torralba, Raquel Urtasun  <a href="" target="_new">ECCV 2020</a> / <a href="papers/eccv20-deep-optimizer/" target="_new">project page</a> / <a href="https://arxiv.org/pdf/2101.07719.pdf" target="_new">arXiv</a> / <a href="papers/eccv20-deep-optimizer/img/deep-feedback-inverse-problem-solver-short.mp4" target="_new">short video (1.5 mins)</a> / <a href="papers/eccv20-deep-optimizer/img/deep-feedback-inverse-problem-solver-long.mp4" target="_new">long video (10 mins)</a> Spotlight presentation </div> </td> </tr>  <tr> <td> <img border=0 src="img/lidarsim.gif" class="publogo"> </td> <td> <div class="publication"> LidarSIM: Realistic LiDAR Simulation by Leveraging the Real World Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng, Bin Yang, Shuhan Tan, Mikita Sazanovich, Wei-Chiu Ma, Raquel Urtasun  <a href="" target="_new">CVPR 2020</a> / <a href="http://openaccess.thecvf.com/content_CVPR_2020/papers/Manivasagam_LiDARsim_Realistic_LiDAR_Simulation_by_Leveraging_the_Real_World_CVPR_2020_paper.pdf" target="_new">paper</a> Oral presentation </div> </td> </tr>  <tr> <td> <a href="https://arxiv.org/pdf/1908.03274.pdf"><img border=0 src="img/light-loc.gif" class="publogo"></a> </td> <td> <div class="publication"> Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization Wei-Chiu Ma*, Ignacio Tartavull*, Ioan Andrei Bârsan*, Shenlong Wang*, Min Bai, Gellert Mattyus, Namdar Homayounfar, Shrinidhi K. Lakshmikanth, Andrei Pokrovsky, Raquel Urtasun <a href="" target="_new">IROS 2019</a> / <a href="https://arxiv.org/pdf/1908.03274.pdf" target="_new">arXiv</a> / <a href="https://www.youtube.com/watch?v=-_PvPPr7y28" target="_new">video</a> Oral presentation </div> </td> </tr>  <tr> <td> <a href="papers/cvpr19-drisf/"><img border=0 src="img/cvpr-drisf.png" class="publogo"></a> </td> <td> <div class="publication"> Deep Rigid Instance Scene Flow Wei-Chiu Ma, Shenlong Wang, Rui Hu, Yuwen Xiong, Raquel Urtasun <a href="" target="_new">CVPR 2019</a> / <a href="papers/cvpr19-drisf/" target="_new">project page</a> / <a href="https://arxiv.org/pdf/1904.08913.pdf" target="_new">arXiv</a> / <a href="papers/cvpr19-drisf/paper.pdf" target="_new">paper + supp (uncompressed)</a> / <a href="papers/cvpr19-drisf/gn_iteration.gif" target="_new">GN solver gif</a> Deep structured scene flow model that rank 1st on <a href="http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php">KITTI Scene Flow Benchmark</a>. Faster than prior art by 800 times. </div> </td> </tr>  <tr> <td> <a href="papers/iccv19-deep-pruner/deeppruner.pdf"><img border=0 src="img/deeppruner.png" class="publogo"></a> </td> <td> <div class="publication"> DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch Shivam Duggal, Shenlong Wang, Wei-Chiu Ma, Rui Hu, Raquel Urtasun <a href="" target="_new">ICCV 2019</a> / <a href="https://arxiv.org/pdf/1909.05845.pdf" target="_new">arXiv</a> / <a href="https://github.com/uber-research/DeepPruner/" target="_new">code</a> / <a href="https://github.com/uber-research/DeepPruner/tree/master/DifferentiablePatchMatch" target="_new"> differentiable PatchMatch module</a> Real-time stereo estimation (62 ms) via Differentiable PatchMatch! </div> </td> </tr>  <tr> <td> <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_Deep_Parametric_Continuous_CVPR_2018_paper.pdf"><img border=0 src="img/cont_conv.png" class="publogo"></a> </td> <td> <div class="publication"> Deep Parametric Continuous Convolutional Neural Networks Shenlong Wang*, Simon Suo*, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun <a href="" target="_new">CVPR 2018</a> / <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_Deep_Parametric_Continuous_CVPR_2018_paper.pdf" target="_new">paper</a> Spotlight presentation </div> </td> </tr>  <tr> <td> <a href="https://youtu.be/tOVWptik0qI" target="_blank"><img border=0 src="img/lost2.gif" class="publogo"></a> </td> <td> <div class="publication"> Find Your Way by Observing the Sun and Other Semantic Cues Wei-Chiu Ma, Shenlong Wang, Marcus A. Brubaker, Sanja Fidler, Raquel Urtasun <a href="" target="_new">ICRA 2017</a> / <a href="http://arxiv.org/pdf/1606.07415.pdf" target="_new">arXiv</a> / <a href="https://youtu.be/tOVWptik0qI" target="_blank">demo video</a> Oral presentation </div> </td> </tr> </script> <script id="pubs_by_date" language="text">  <tr> <td> <img border=0 src="img/svr.png" class="publogo"> </td> <td> <div class="publication"> Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering Cheng Sun, Jaesung Choe, Charles Loop, Wei-Chiu Ma, Yu-Chiang Frank Wang <a href="" target="">arXiv 2024</a> / <a href="https://arxiv.org/pdf/2412.04459" target="">arXiv</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/eval3d.gif" class="publogo"> </td> <td> <div class="publication"> Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation Shivam Duggal*, Yushi Hu*, Oscar Michel, Aniruddha Kembhavi, William T. Freeman, Noah A. Smith, Ranjay Krishna, Antonio Torralba, Ali Farhadi, Wei-Chiu Ma <a href="" target="">arXiv 2024</a> / <a href="" target="">arXiv</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/odin.gif" class="publogo"> </td> <td> <div class="publication"> From an Image to a Scene: Learning to Imagine the World from a Million 360° Videos Matthew Wallingford, Anand Bhattad, Aditya Kusupati, Vivek Ramanujan, Matt Deitke, Sham Kakade, Aniruddha Kembhavi, Roozbeh Mottaghi, Wei-Chiu Ma, Ali Farhadi <a href="" target="">NeurIPS 2024</a> / <a href="https://mattwallingford.github.io/ODIN/" target="">project page</a> / <a href="https://arxiv.org/pdf/2412.07770" target="">arXiv</a> / <a href="https://huggingface.co/datasets/mwallingford/360-1M/tree/main" target="">huggingface</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/multilingual.png" class="publogo"> </td> <td> <div class="publication"> Multilingual Diversity Improves Vision-Language Representations Thao Nguyen, Matthew Wallingford, Sebastin Santy, Wei-Chiu Ma, Sewoong Oh, Ludwig Schmidt, Pang Wei Koh, Ranjay Krishna <a href="" target="">NeurIPS 2024</a> / <a href="https://arxiv.org/pdf/2405.16915" target="">paper</a> Spotlight presentation </div> </td> </tr>  <tr> <td> <img border=0 src="img/task-me-anything.png" class="publogo"> </td> <td> <div class="publication"> Task Me Anything Jieyu Zhang, Weikai Huang, Zixian Ma, Oscar Michel, Dong He, Tanmay Gupta, Wei-Chiu Ma, Ali Farhadi, Aniruddha Kembhavi, Ranjay Krishna <a href="" target="">NeurIPS 2024 Datasets and Benchmarks</a> / <a href="https://www.task-me-anything.org/" target="">project page</a> / <a href="https://arxiv.org/pdf/2406.11775" target="">paper</a> / <a href="https://huggingface.co/spaces/zixianma/TaskMeAnything-UI" target="">huggingface</a> / <a href="https://github.com/JieyuZ2/TaskMeAnything" target="">code</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/blink.png" class="publogo"> </td> <td> <div class="publication"> BLINK: Multimodal Large Language Models Can See but Not Perceive Xingyu Fu*, Yushi Hu*, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma†, Ranjay Krishna† <a href="" target="">ECCV 2024</a> / <a href="https://zeyofu.github.io/blink/" target="">project page</a> / <a href="https://arxiv.org/pdf/2404.12390" target="">paper</a> / <a href="https://huggingface.co/datasets/BLINK-Benchmark/BLINK" target="">dataset</a> / <a href="https://eval.ai/web/challenges/challenge-page/2287/overview" target="">Eval AI</a> / <a href="https://github.com/zeyofu/BLINK_Benchmark" target="">code</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/video2game.gif" class="publogo"> </td> <td> <div class="publication"> Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video Hongchi Xia, Zhi-Hao Lin, Wei-Chiu Ma, Shenlong Wang <a href="" target="">CVPR 2024</a> / <a href="https://video2game.github.io/" target="">project page</a> / <a href="https://arxiv.org/pdf/2404.09833" target="">paper</a> / <a href="https://video2game.github.io/src/garden/index.html" target="">shooting demo</a> / <a href="https://github.com/video2game/video2game" target="">code</a> </div> </td> </tr>  <tr> <td> <img border=0 src="img/extra_nerf.gif" class="publogo"> </td> <td> <div class="publication"> ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Models Meng-Li Shih, Wei-Chiu Ma, Lorenzo Boyice, Aleksander Holynski, Forrester Cole, Brian Curless, Janne Kontkanen <a href="" target="">CVPR 2024</a> / <a href="https://arxiv.org/pdf/2406.06133" target="">arXiv</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/many-nerf.gif" class="publogo"> </td> <td> <div class="publication"> Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects Tianhang Cheng, Wei-Chiu Ma, Kaiyu Guan, Antonio Torralba, Shenlong Wang <a href="" target="">NeurIPS 2023</a> / <a href="https://tianhang-cheng.github.io/SfD-project.github.io/" target="">project page</a> / <a href="https://arxiv.org/pdf/2401.05236" target="">arXiv</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/light-sim.gif" class="publogo"> </td> <td> <div class="publication"> LightSim: Neural Lighting Simulation for Urban Scenes Ava Pun*, Gary Sun*, Jingkang Wang*, Yun Chen, Ze Yang, Sivabalan Manivasagam, Wei-Chiu Ma, Raquel Urtasun <a href="" target="">NeurIPS 2023</a> / <a href="https://arxiv.org/pdf/2312.06654" target="">arXiv</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/unisim.mp4" class="publogo"> </td> <td> <div class="publication"> UniSim: A Neural Closed-Loop Sensor Simulator Ze Yang*, Yun Chen*, Jingkang Wang*, Sivabalan Manivasagam*, Wei-Chiu Ma, Joyce Anqi Yang, Raquel Urtasun <a href="" target="">CVPR 2023</a> / <a href="https://waabi.ai/unisim/" target="">project page</a> / <a href="https://openaccess.thecvf.com/content/CVPR2023/papers/Yang_UniSim_A_Neural_Closed-Loop_Sensor_Simulator_CVPR_2023_paper.pdf" target="">paper</a> / <a href="http://www.cs.toronto.edu/~wangjk/publications/unisim/unisim_final_v2_4k.mp4" target="">4K demo</a> / <a href="https://waabi.ai/wp-content/uploads/2023/05/UniSim-video_compressed.mp4" target="">video (8 mins)</a> Highlight presentation </div> </td> </tr> <tr> <td> <img border=0 src="img/lidar-gen.mp4" class="publogo"> </td> <td> <div class="publication"> UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang, Raquel Urtasun <a href="" target="">CVPR 2023</a> / <a href="https://waabi.ai/ultralidar/" target="">project page</a> / <a href="https://openaccess.thecvf.com/content/CVPR2023/papers/Xiong_Learning_Compact_Representations_for_LiDAR_Completion_and_Generation_CVPR_2023_paper.pdf" target="">paper</a> / <a href="https://waabi.ai/wp-content/uploads/2023/05/UltraLidar-video.mov" target="">video (1 min)</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/thermal-im.png" class="publogo"> </td> <td> <div class="publication"> What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging Zitian Tang*, Wenjie Yeh*, Wei-Chiu Ma, Hang Zhao <a href="" target="">CVPR 2023</a> / <a href="https://arxiv.org/pdf/2304.13651.pdf" target="">arXiv</a> / <a href="https://github.com/ZitianTang/Thermal-IM" target="">dataset</a> </div> </td> </tr> <tr> <td> <img border=0 src="papers/neurips22-sgam/sgam.gif" class="publogo"> </td> <td> <div class="publication"> SGAM: Building a Virtual 3D World through Simultaneous Generation and Mapping Yuan Shen, Wei-Chiu Ma, Shenlong Wang <a href="" target="">NeurIPS 2022</a> / <a href="https://yshen47.github.io/sgam/" target="">project page</a> / <a href="https://openreview.net/pdf?id=17KCLTbRymw" target="">paper</a> / <a href="https://github.com/yshen47/SGAM" target="">code</a> </div> </td> </tr> <tr> <td> <img border=0 src="papers/corl22-cad-sim/cad-sim.gif" class="publogo"> </td> <td> <div class="publication"> CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Realistic and Controllable Sensor Simulation Jingkang Wang, Sivabalan Manivasagam, Yun Chen, Ze Yang, Ioan Andrei Bârsan, Joyce Anqi Yang, Wei-Chiu Ma, Raquel Urtasun <a href="" target="">CoRL 2022</a> / <a href="http://www.cs.toronto.edu/~wangjk/publications/cadsim.html" target="">project page</a> / <a href="https://openreview.net/pdf?id=Mp3Y5jd7rnW" target="">paper</a> / <a href="http://www.cs.toronto.edu/~wangjk/publications/cadsim/cadsim_corl_denoise.mp4" target="">video</a> </div> </td> </tr> <tr> <td> <img border=0 src="papers/corl22-mira/mira.gif" class="publogo"> </td> <td> <div class="publication"> MIRA: Mental Imagery for Robotic Affordances Lin Yen-Chen, Pete Florence, Andy Zeng, Jonathan T. Barron, Yilun Du, Wei-Chiu Ma, Anthony Simeonov, Alberto Rodriguez Garcia, Phillip Isola <a href="" target="">CoRL 2022</a> / <a href="http://yenchenlin.me/mira/" target="">project page</a> / <a href="https://arxiv.org/pdf/2212.06088.pdf" target="">paper</a> / <a href="https://www.youtube.com/watch?v=3GH-s9db5e0&ab_channel=MIRA" target="">video</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/vc-3.png" class="publogo"> </td> <td> <div class="publication"> Virtual Correspondence: Humans as a Cue for Extreme-View Geometry Wei-Chiu Ma, Anqi Joyce Yang, Shenlong Wang, Raquel Urtasun, Antonio Torralba <a href="" target="_new">CVPR 2022</a> / <a href="virtual-correspondence" target="_new">project page</a> / <a href="https://arxiv.org/pdf/2206.08365.pdf" target="_new">paper</a> / <a href="https://virtual-correspondence.github.io/img/vc_demo.mp4" target="_new">video (1.5 mins)</a> / <a href="https://youtu.be/W9odd2F2Bx4" target="_new">video (5 mins)</a> / <a href="https://news.mit.edu/2022/seeing-whole-from-some-parts-0617" target="_new">MIT News</a> / <a href="https://techxplore.com/news/2022-06-vision-technique-3d-2d-images.html" target="_new">TechXplore</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/neurmips.gif" class="publogo"> </td> <td> <div class="publication"> NeurMiPs: Neural Mixture of Planar Experts for View Synthesis Zhi-Hao Lin, Wei-Chiu Ma, Hao-Yu Max Hsu, Yu-Chiang Frank Wang, Shenlong Wang <a href="" target="_new">CVPR 2022</a> / <a href="https://zhihao-lin.github.io/neurmips/" target="_new">project page</a> / <a href="https://openaccess.thecvf.com/content/CVPR2022/papers/Lin_NeurMiPs_Neural_Mixture_of_Planar_Experts_for_View_Synthesis_CVPR_2022_paper.pdf" target="_new">paper</a> / <a href="https://github.com/zhihao-lin/neurmips" target="_new">code</a> / <a href="https://www.youtube.com/watch?v=PV1dCTWL5Oo" target="_new">video</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/sdf_secrets.png" class="publogo"> </td> <td> <div class="publication"> Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wild Shivam Duggal*, Zihao Wang*, Wei-Chiu Ma, Sivabalan Manivasagam, Justin Liang, Shenlong Wang, Raquel Urtasun <a href="" target="_new">WACV 2022</a> / <a href="https://arxiv.org/pdf/2101.06860.pdf" target="_new">arXiv</a> / <a href="https://www.youtube.com/watch?v=IRygme5J-Ng" target="_new">video</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/barf.gif" class="publogo"> </td> <td> <div class="publication"> BARF: Bundle-Adjusting Neural Radiance Fields Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, Simon Lucey <a href="" target="_new">ICCV 2021</a> / <a href="https://chenhsuanlin.bitbucket.io/bundle-adjusting-NeRF/" target="_new">project page</a> / <a href="https://arxiv.org/pdf/2104.06405.pdf" target="_new">arXiv</a> / <a href="https://github.com/chenhsuanlin/bundle-adjusting-NeRF" target="_new">code</a> / <a href="https://www.youtube.com/watch?v=dCmCZs2Hpi0" target="_new">video</a> / <a href="https://read.deeplearning.ai/the-batch/issue-95#3d-scene-synthesis-for-the-real-world" target="_new">news coverage (The Batch: DeepLearning.AI)</a> Oral presentation </div> </td> </tr> <tr> <td> <img border=0 src="img/s3-skinning.png" class="publogo"> </td> <td> <div class="publication"> S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling Ze Yang, Shenlong Wang, Sivabalan Manivasagam, Zeng Huang, Wei-Chiu Ma, Xinchen Yan, Ersin Yumer, Raquel Urtasun <a href="" target="_new">CVPR 2021</a> / <a href="https://arxiv.org/pdf/2101.06571.pdf" target="_new">arXiv</a> / <a href="https://www.youtube.com/watch?v=oCpFJZrVDCs" target="_new">video (5 mins)</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/lime.png" class="publogo"> </td> <td> <div class="publication"> Recovering and Simulating Pedestrians in the Wild Ze Yang, Sivabalan Manivasagam, Ming Liang, Bin Yang, Wei-Chiu Ma, Raquel Urtasun <a href="" target="_new">CoRL 2020</a> / <a href="https://arxiv.org/abs/2011.08106" target="_new">arXiv</a> / <a href="https://www.youtube.com/watch?v=iBsIoCJ9I1s" target="_new">video</a> Spotlight presentation </div> </td> </tr> <tr> <td> <img border=0 src="img/deep-optimizer-teaser.gif" class="publogo"> </td> <td> <div class="publication"> Deep Feedback Inverse Problem Solver Wei-Chiu Ma, Shenlong Wang, Jiayuan Gu, Sivabalan Manivasagam, Antonio Torralba, Raquel Urtasun  <a href="" target="_new">ECCV 2020</a> / <a href="papers/eccv20-deep-optimizer/" target="_new">project page</a> / <a href="https://arxiv.org/pdf/2101.07719.pdf" target="_new">arXiv</a> / <a href="papers/eccv20-deep-optimizer/img/deep-feedback-inverse-problem-solver-short.mp4" target="_new">short video (1.5 mins)</a> / <a href="papers/eccv20-deep-optimizer/img/deep-feedback-inverse-problem-solver-long.mp4" target="_new">long video (10 mins)</a> Spotlight presentation </div> </td> </tr> <tr> <td> <img border=0 src="img/shape-comp.gif" class="publogo"> </td> <td> <div class="publication"> Weakly-supervised 3D Shape Completion in the Wild Jiayuan Gu, Wei-Chiu Ma, Sivabalan Manivasagam, Wenyuan Zeng, Zihao Wang, Yuwen Xiong, Hao Su, Raquel Urtasun  <a href="" target="_new">ECCV 2020</a> / <a href="https://arxiv.org/pdf/2008.09110.pdf" target="_new">arXiv</a> Spotlight presentation </div> </td> </tr> <tr> <td> <img border=0 src="img/levelset-rcnn.png" class="publogo"> </td> <td> <div class="publication"> LevelSet R-CNN: A Deep Variational Method for Instance Segmentation Namdar Homayounfar*, Yuwen Xiong*, Justin Liang*, Wei-Chiu Ma, Raquel Urtasun <a href="" target="_new">ECCV 2020</a> / <a href="https://arxiv.org/pdf/2007.15629.pdf" target="_new">arXiv</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/video-compression.gif" class="publogo"> </td> <td> <div class="publication"> Conditional Entropy Coding for Efficient Video Compression Jerry Junkai Liu, Shenlong Wang, Wei-Chiu Ma, Meet Shah, Rui Hu, Pranaab Dhawan, Raquel Urtasun <a href="" target="_new">ECCV 2020</a> / <a href="https://arxiv.org/pdf/2008.09180.pdf" target="_new">arXiv</a> </div> </td> </tr> <tr> <td> <a href="https://arxiv.org/pdf/1912.02801.pdf"><img border=0 src="img/polytransform-new" class="publogo"></a> </td> <td> <div class="publication"> PolyTransform: Deep Polygon Transformer for Instance Segmentation Justin Liang, Namdar Homayounfar, Wei-Chiu Ma, Yuwen Xiong, Rui Hu, Raquel Urtasun <a href="" target="_new">CVPR 2020</a> / <a href="https://arxiv.org/pdf/1912.02801.pdf" target="_new">arXiv</a> / <a href="http://justin-liang.com/papers/Liang_PolyTransform_Supp.pdf" target="_new">supp</a> / <a href="http://justin-liang.com/papers/Liang_PolyTransform_Video.zip" target="_new">video</a> / <a href="" target="_new">code (coming soon!)</a> State-of-the-art performance on Cityscapes! Consistent improvements across all backbones! </div> </td> </tr> <tr> <td> <img border=0 src="img/lidarsim.gif" class="publogo"> </td> <td> <div class="publication"> LidarSIM: Realistic LiDAR Simulation by Leveraging the Real World Sivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng, Bin Yang, Shuhan Tan, Mikita Sazanovich, Wei-Chiu Ma, Raquel Urtasun  <a href="" target="_new">CVPR 2020</a> / <a href="http://openaccess.thecvf.com/content_CVPR_2020/papers/Manivasagam_LiDARsim_Realistic_LiDAR_Simulation_by_Leveraging_the_Real_World_CVPR_2020_paper.pdf" target="_new">paper</a> Oral presentation </div> </td> </tr> <tr> <td> <a href="https://arxiv.org/pdf/1908.03274.pdf"><img border=0 src="img/light-loc.gif" class="publogo"></a> </td> <td> <div class="publication"> Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization Wei-Chiu Ma*, Ignacio Tartavull*, Ioan Andrei Bârsan*, Shenlong Wang*, Min Bai, Gellert Mattyus, Namdar Homayounfar, Shrinidhi K. Lakshmikanth, Andrei Pokrovsky, Raquel Urtasun <a href="" target="_new">IROS 2019</a> / <a href="https://arxiv.org/pdf/1908.03274.pdf" target="_new">arXiv</a> / <a href="https://www.youtube.com/watch?v=-_PvPPr7y28" target="_new">video</a> Oral presentation </div> </td> </tr> <tr> <td> <a href="papers/cvpr19-drisf/"><img border=0 src="img/cvpr-drisf.png" class="publogo"></a> </td> <td> <div class="publication"> Deep Rigid Instance Scene Flow Wei-Chiu Ma, Shenlong Wang, Rui Hu, Yuwen Xiong, Raquel Urtasun <a href="" target="_new">CVPR 2019</a> / <a href="papers/cvpr19-drisf/" target="_new">project page</a> / <a href="https://arxiv.org/pdf/1904.08913.pdf" target="_new">arXiv</a> / <a href="papers/cvpr19-drisf/paper.pdf" target="_new">paper + supp (uncompressed)</a> / <a href="papers/cvpr19-drisf/gn_iteration.gif" target="_new">GN solver gif</a> Deep structured scene flow model that rank 1st on <a href="http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php">KITTI Scene Flow Benchmark</a>. Faster than prior art by 800 times. </div> </td> </tr> <tr> <td> <a href=""><img border=0 src="img/cvpr-road-boundary.png" class="publogo"></a> </td> <td> <div class="publication"> Convolutional Recurrent Network for Road Boundary Extraction Justin Liang*, Namdar Homayounfar*, Wei-Chiu Ma, Shenlong Wang, Raquel Urtasun <a href="" target="_new">CVPR 2019</a> / <a href="https://nhoma.github.io/papers/road_cvpr19.pdf" target="_new">paper</a> / <a href="https://nhoma.github.io/papers/road_cvpr19_supp.pdf" target="_new">supp</a> </div> </td> </tr> <tr> <td> <a href="papers/iccv19-deep-pruner/deeppruner.pdf"><img border=0 src="img/deeppruner.png" class="publogo"></a> </td> <td> <div class="publication"> DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch Shivam Duggal, Shenlong Wang, Wei-Chiu Ma, Rui Hu, Raquel Urtasun <a href="" target="_new">ICCV 2019</a> / <a href="https://arxiv.org/pdf/1909.05845.pdf" target="_new">arXiv</a> / <a href="https://github.com/uber-research/DeepPruner/" target="_new">code</a> / <a href="https://github.com/uber-research/DeepPruner/tree/master/DifferentiablePatchMatch" target="_new"> differentiable PatchMatch module</a> Real-time stereo estimation (62 ms) via Differentiable PatchMatch! </div> </td> </tr> <tr> <td> <a href="http://www.mit.edu/~hangzhao/videos/SoM_supp.mp4"><img border=0 src="img/sound_of_motions" class="publogo"></a> </td> <td> <div class="publication"> The Sound of Motions Hang Zhao, Chuang Gan, Wei-Chiu Ma, Antonio Torralba <a href="" target="_new">ICCV 2019</a> / <a href="https://arxiv.org/pdf/1904.05979.pdf" target="_new">arXiv</a> / <a href="http://www.mit.edu/~hangzhao/videos/SoM_supp.mp4" target="_new">demon video</a> </div> </td> </tr> <tr> <td> <a href=""><img border=0 src="img/iccv19-dag.jpg" class="publogo"></a> </td> <td> <div class="publication"> DAG-Mapper: Learning to Map by Discovering Lane Topology Namdar Homayounfar, Wei-Chiu Ma, Justin Liang, Xinyu Wu, Jack Fan, Raquel Urtasun <a href="" target="_new">ICCV 2019</a> / <a href="https://nhoma.github.io/papers/dagmapper_iccv19.pdf" target="_new">paper</a> / <a href="https://nhoma.github.io/papers/dagmapper_iccv19_supp.pdf" target="_new">supp</a> </div> </td> </tr> <tr> <td> <a href="papers/eccv18-intrinsics/top.pdf"><img border=0 src="img/cover_intrinsic.png" class="publogo"></a> </td> <td> <div class="publication"> Single Image Intrinsic Decomposition without a Single Intrinsic Image Wei-Chiu Ma, Hang Chu, Bolei Zhou, Raquel Urtasun, Antonio Torralba <a href="" target="_new">ECCV 2018</a> / <a href="papers/eccv18-intrinsics/top.pdf" target="_new">paper</a> </div> </td> </tr> <tr> <td> <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_Deep_Parametric_Continuous_CVPR_2018_paper.pdf"><img border=0 src="img/cont_conv.png" class="publogo"></a> </td> <td> <div class="publication"> Deep Parametric Continuous Convolutional Neural Networks Shenlong Wang*, Simon Suo*, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun <a href="" target="_new">CVPR 2018</a> / <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_Deep_Parametric_Continuous_CVPR_2018_paper.pdf" target="_new">paper</a> Spotlight presentation </div> </td> </tr> <tr> <td> <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Chu_SurfConv_Bridging_3D_CVPR_2018_paper.pdf"><img border=0 src="img/surf_conv.png" class="publogo"></a> </td> <td> <div class="publication"> SurfConv: Bridging 3D and 2D Convolution for RGBD Images Hang Chu, Wei-Chiu Ma, Kaustav Kundu, Raquel Urtasun, and Sanja Fidler <a href="" target="_new">CVPR 2018</a> / <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Chu_SurfConv_Bridging_3D_CVPR_2018_paper.pdf" target="_new">paper</a> / <a href="https://github.com/chuhang/SurfConv" target="_new">code</a> </div> </td> </tr> <tr> <td> <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Homayounfar_Hierarchical_Recurrent_Attention_CVPR_2018_paper.pdf"><img border=0 src="img/snake.png" class="publogo"></a> </td> <td> <div class="publication"> Hierarchical Recurrent Attention Networks for Structured Online Maps Namdar Homayounfar, Wei-Chiu Ma, Shrinidhi K. Lakshmikanth, Raquel Urtasun <a href="" target="_new">CVPR 2018</a> / <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Homayounfar_Hierarchical_Recurrent_Attention_CVPR_2018_paper.pdf" target="_new">paper</a> / <a href="https://nhoma.github.io/papers/hran_cvpr18_supp.pdf" target="_new">supp</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/book1.png" class="publogo"> </td> <td> <div class="publication"> Activity Forecasting: An Invitation to Predictive Perception Kris M. Kitani, De-An Huang, Wei-Chiu Ma <a href="" target="_new">Group and Crowd Behavior for Computer Vision. Chapter 12, 2017</a> / <a href="http://www.sciencedirect.com/science/article/pii/B978012809276700014X" target="_new">link</a> </div> </td> </tr> <tr> <td> <a href="https://youtu.be/tOVWptik0qI" target="_blank"><img border=0 src="img/lost2.gif" class="publogo"></a> </td> <td> <div class="publication"> Find Your Way by Observing the Sun and Other Semantic Cues Wei-Chiu Ma, Shenlong Wang, Marcus A. Brubaker, Sanja Fidler, Raquel Urtasun <a href="" target="_new">ICRA 2017</a> / <a href="http://arxiv.org/pdf/1606.07415.pdf" target="_new">arXiv</a> / <a href="https://youtu.be/tOVWptik0qI" target="_blank">demo video</a> Oral presentation </div> </td> </tr> <tr> <td> <a href="http://arxiv.org/pdf/1604.01431.pdf"><img border=0 src="img/FPIOC.gif" class="publogo"></a> </td> <td> <div class="publication"> Forecasting Interactive Dynamics of Pedestrians with Fictitious Play Wei-Chiu Ma, De-An Huang, Namhoon Lee, Kris M. Kitani <a href="" target="_new">CVPR 2017</a> / <a href="http://arxiv.org/pdf/1604.01431.pdf" target="_new">arXiv</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/CVPR2015.png" class="publogo"> </td> <td> <div class="publication"> How Do We Use Our Hands? Discovering a Diverse Set of Common Grasps De-An Huang, Wei-Chiu. Ma*, Minghuan Ma*, K. M. Kitani <a href="" target="_new">CVPR 2015</a> / <a href="https://scholar.google.com/citations?view_op=view_citation&hl=en&user=yv3sH74AAAAJ&cstart=20&sortby=pubdate&citation_for_view=yv3sH74AAAAJ:HeT0ZceujKMC" target="_new">paper</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/Hand-Object-Interaction.png" class="publogo"> </td> <td> <div class="publication"> Recognizing Hand-Object Interactions in Wearable Camera Videos Tatsuya Ishihara, Kris M. Kitani, Wei-Chiu Ma, Hironobu Takagi, Chieko Asakawa <a href="" target="_new">ICIP 2015</a> / <a href="http://www.cs.cmu.edu/~kkitani/pdf/IKMTA-ICIP2015.pdf" target="_new">paper</a> </div> </td> </tr> <tr> <td> <img border=0 src="img/TrafficGA.png" class="publogo"> </td> <td> <div class="publication"> Novel traffic signal timing adjustment strategy based on Genetic Algorithm Hsiao-Yu Tung*, Wei-Chiu Ma*, Tian-Li Yu <a href="" target="_new">CEC 2014</a> / <a href="http://www.andrew.cmu.edu/user/weichium/publications/TrafficGA.pdf" target="_new">paper</a> Oral presentation </div> </td> </tr> <tr> <td> <a href="https://www.youtube.com/embed/TUwk1DSgl1Q"><img border=0 src="img/TDTOS-concept.png" class="publogo"></a> </td> <td> <div class="publication"> TDTOS: T-Shirt Design and Try On System Chen-Yu Hsu*, Chi-Hsien Yen*, Wei-Chiu Ma*, Shao-Yi Chien <a href="" target="_new">Asia-Pacific Workshop on FPGA Applications 2012</a> /  <a href="https://www.youtube.com/embed/TUwk1DSgl1Q" target="_new">demo video</a> Oral presentation Best Application Award </div> </td> </tr> </script> </table> </div> </td> </tr> <tr> <td colspan = 7>   <h2 id="pro-bono">Pro bono office hour</h2> Inspired by <a href="https://kyunghyuncho.me/office-hour-request/">Prof. Kyunghyun Cho</a> and <a href="https://krrish94.github.io/">Krishna Murthy</a>, starting January 2021, I have decided to commit 1~2 hours every week to provide guidance, suggestions, and/or mentorships for students from underrepresented groups or whoever is in need. Please fill in this <a href="https://docs.google.com/forms/d/1uYA95BMyFjnvY2-XDbwSAlTz79Ip55cytzrNoROS9ko/edit">form</a> if you are interested. Need more (diverse) opinions? Consider talking to people with different expertise or from different background: <a href="https://www.tongzhouwang.info">Tongzhou Wang (ML/RL)</a>, <a href="https://alexander-h-liu.github.io">Alexander Haojan Liu (Speech/NLP)</a>, <a href="https://zhijianliu.com">Zhijian Liu (ML)</a>, <a href="https://www.zyrianov.org">Vlas Zyrianov (CV)</a>, <a href="https://yshen47.github.io">Yuan Shen (CV)</a>, <a href="https://www.cs.toronto.edu/~jungao/#pro-bono">Jun Gao (CV/Graphics)</a>. </td> </tr> <tr> <td colspan = 7> <h2 id="misc">Misc.</h2> <ul> <li> I enjoy playing/watching all kinds of sports, including but not limited to badminton, basketball, baseball, tennis, running, etc. My favorite sports players and my favorite games (click their names!) are: <ul> <li><a href="https://www.youtube.com/watch?v=GTJwoWHMEw0">Kobe Bryant</a> (basketball) </li> <li> <a href="https://www.youtube.com/watch?v=rctOtFOXco8">Roger Federer</a> (tennis)</li> <li> <a href="https://www.youtube.com/watch?v=Hf-n1yfd8II">Lee Chong Wei</a> (badminton)</li> </ul> </li> <li> I like chinese pop music and my favorite singer is <a href="https://en.wikipedia.org/wiki/Jay_Chou">Jay Chou</a>.  </li> <li> A few personal favorite papers/books/courses can be found <a href="misc/favorite-books-papers.html">here</a> (unordered and unfinished).</li> </ul> </td> </tr>  </table> </center> </div> </div> <center><a href="http://accessibility.mit.edu/">Accessibility</a></center> </center>  <script type="text/javascript"> var sc_project=11104199; var sc_invisible=1; var sc_security="50dd2089"; var scJsHost = (("https:" == document.location.protocol) ? "https://secure." : "http://www."); document.write("<sc"+"ript type='text/javascript' src='" + scJsHost+ "statcounter.com/counter/counter.js'></"+"script>"); </script> <noscript><div class="statcounter"><a title="free hit counter" href="http://statcounter.com/" target="_blank"><img class="statcounter" src="//c.statcounter.com/11104199/0/50dd2089/1/" alt="free hit counter"></a></div></noscript>  <script>showPubs(0);</script> </body> </html>

CINXE.COM

Wei-Chiu Ma, Cornell