NeRV

<!DOCTYPE html> <html> <head lang="en"> <meta charset="UTF-8"> <meta http-equiv="x-ua-compatible" content="ie=edge"> <title>NeRV</title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1">   <link rel="icon" type="image/png" href="icon.png">  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css"> <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css"> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.css"> <link rel="stylesheet" href="app.css"> <link rel="stylesheet" href="bootstrap.min.css"> <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script> <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.8.0/codemirror.min.js"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/1.5.3/clipboard.min.js"></script> <script src="app.js"></script> </head> <body> <div class="container" id="main"> <div class="row"> <h1 class="col-md-12 text-center"> NeRV: Neural Reflectance and Visibility Fields </br> for Relighting and View Synthesis </br> <small> CVPR 2021 </small> </h1> </div> <div class="row"> <div class="col-md-12 text-center"> <ul class="list-inline"> <li> <a href="https://pratulsrinivasan.github.io/"> Pratul P. Srinivasan </a> </br>Google Research </li> <li> <a href="https://boyangdeng.com/"> Boyang Deng </a> </br>Google Research </li> <li> <a href="https://people.csail.mit.edu/xiuming/"> Xiuming Zhang </a> </br>MIT </li> <li> <a href="https://www.matthewtancik.com/"> Matthew Tancik </a> </br>UC Berkeley </li> <li> <a href="https://bmild.github.io/"> Ben Mildenhall </a> </br>UC Berkeley </li> <li> <a href="https://jonbarron.info/"> Jonathan T. Barron </a> </br>Google Research </li> </ul> </div> </div> <div class="row"> <div class="col-md-8 col-md-offset-2 text-center"> <ul class="nav nav-pills nav-justified"> <li> <a href="https://arxiv.org/abs/2012.03927"> <image src="paper.png" height="120px"><br> <h4><strong>Paper</strong></h4> </a> </li> <li> <a href="https://www.youtube.com/watch?v=4XyDdvhhjVo"> <image src="youtube_icon_dark.png" height="120px"><br> <h4><strong>Technical Video</strong></h4> </a> </li> <li> <a href=""> <image src="github_pad.png" height="120px"><br> <h4><strong>Code [Coming Soon]</strong></h4> </a> </li> </ul> </div> </div> <div class="row"> <div class="col-md-8 col-md-offset-2"> <h3> Example Results: Relighting and View Synthesis </h3> <video id="v0" width="100%" autoplay loop muted controls> <source src="nerv_results.mp4" type="video/mp4" /> </video> </div> </div> <div class="row"> <div class="col-md-8 col-md-offset-2"> <h3> Example Results: Material Editing </h3> <video id="v0" width="100%" autoplay loop muted controls> <source src="nerv_results_matedit.mp4" type="video/mp4" /> </video> </div> </div>  <div class="row"> <div class="col-md-8 col-md-offset-2"> <h3> Abstract </h3> <image src="hotdog.png" class="img-responsive" alt="overview"><br> <p class="text-justify"> We present a method that takes as input a set of images of a scene illuminated by unconstrained known lighting, and produces as output a 3D representation that can be rendered from novel viewpoints under arbitrary lighting conditions. Our method represents the scene as a continuous volumetric function parameterized as MLPs whose inputs are a 3D location and whose outputs are the following scene properties at that input location: volume density, surface normal, material parameters, distance to the first surface intersection in any direction, and visibility of the external environment in any direction. Together, these allow us to render novel views of the object under arbitrary lighting, including indirect illumination effects. The predicted visibility and surface intersection fields are critical to our model's ability to simulate direct and indirect illumination during training, because the brute-force techniques used by prior work are intractable for lighting conditions outside of controlled setups with a single light. Our method outperforms alternative approaches for recovering relightable 3D scene representations, and performs well in complex lighting settings that have posed a significant challenge to prior work. </p> </div> </div>  <div class="row"> <div class="col-md-8 col-md-offset-2"> <h3> Technical Video </h3> <div class="text-center"> <div style="position:relative;padding-top:56.25%;">  <iframe src="https://www.youtube.com/embed/4XyDdvhhjVo" allowfullscreen style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe> </div> </div> </div> </div> <div class="row"> <div class="col-md-8 col-md-offset-2"> <h3> BibTeX </h3> <div class="form-group col-md-10 col-md-offset-1"> <textarea id="bibtex" class="form-control" readonly> @inproceedings{nerv2021, title={NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis}, author={Pratul P. Srinivasan and Boyang Deng and Xiuming Zhang and Matthew Tancik and Ben Mildenhall and Jonathan T. Barron}, booktitle={CVPR}, year={2021}} </textarea> </div> </div> </div> <div class="row"> <div class="col-md-8 col-md-offset-2"> <h3> Acknowledgements </h3> Our implementation is based on <a href="https://github.com/google-research/google-research/tree/master/jaxnerf">Boyang Deng's awesome JAX implementation of NeRF</a>. <br> The website template was borrowed from <a href="http://mgharbi.com/">Micha毛l Gharbi</a>. </p> </div> </div> </div> </body> </html>

CINXE.COM

NeRV