Image Collation

<html class="gr__xishen_com"> <head> <title>Image Collation</title> <meta name="viewport" content="width=device-width,initial-scale=1,user-scalable=yes"> <meta name="description" content="Image Collation: Matching illustrations in manuscripts"> <link href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-rwoIResjU2yc3z8GV/NPeZWAv56rSmLldC3R/AZzGRnGxQQKnKkoFVhFQhNUwEyJ" crossorigin="anonymous"> <script src="https://code.jquery.com/jquery-3.2.1.min.js" integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4=" crossorigin="anonymous"></script> <link href="https://fonts.googleapis.com/css?family=Open+Sans:300,400,700" rel="stylesheet"> <link href="style.css" rel="stylesheet"> <meta name="twitter:card" content="summary_large_image"> <meta name="twitter:title" content="Image Collation"> <meta name="twitter:description" content="Image Collation: Matching illustrations in manuscripts">  <link rel="stylesheet" id="coToolbarStyle" href="chrome-extension://cjabmdjcfcfdmffimndhafhblfmpjdpe/toolbar/styles/placeholder.css" type="text/css"> <script type="text/javascript" id="cosymantecbfw_removeToolbar"> (function() { var toolbarElement = {}, parent = {}, interval = 0, retryCount = 0, isRemoved = false; if (window.location.protocol === 'file:') { interval = window.setInterval(function() { toolbarElement = document.getElementById('coFrameDiv'); if (toolbarElement) { parent = toolbarElement.parentNode; if (parent) { parent.removeChild(toolbarE truncated 716 bytes... </script> </head> <body class="nd-docs" data-gr-c-s-loaded="true"> <div id="pageheader" class="container-fluid" style="text-align: center; background-color: #37a2a8; padding: 2rem 15px; margin-bottom: 1.5rem; color: #e4f1fe;"> <p></p> <h2>Image Collation: Matching illustrations in manuscripts</h2> <p><a href="https://icdar2021.org/">ICDAR 2021</a></p> <p> <a href="https://github.com/Rykoua">Ryad Kaoua</a><sup>1</sup>            <a href="https://imagine-lab.enpc.fr/~xshen">Xi Shen</a><sup>1</sup>            Alexandra Durr<sup>2</sup>            <a href="https://www.orient-mediterranee.com/spip.php?article487&lang=fr">Stavros Lazaris</a><sup>3</sup>            <a href="https://davidpicard.github.io/">David Picard</a><sup>1</sup>            <a href="http://imagine.enpc.fr/~aubrym/">Mathieu Aubry</a><sup>1</sup> </p> <p> <sup>1</sup><a href="http://imagine.enpc.fr/">LIGM (UMR 8049) - Ecole des Ponts, UPE</a>             <sup>2</sup><a href="https://www.uvsq.fr/">Université de Versailles-Saint-Quentin-en-Yvelines </a>            <sup>3</sup><a href="https://www.orient-mediterranee.com/spip.php?article487&lang=fr">CNRS</a> </p> <p> <a href="https://github.com/Rykoua/ImageCollation">Code</a>     <a href="https://github.com/Rykoua/ImageCollation/tree/main/ManuscriptDownloader">Dataset</a>     <a href="https://arxiv.org/abs/2108.08109">arXiv</a>     </p> </div> <div class="container"> <div class="row"> <div class="col"> <div class="container" style="margin-bottom: 10px; text-align: center;"> <tr> <td><a href="teaser/8.jpg"><img src="teaser/8.jpg" style="width: 150px; height: 200px;" alt="teaser"></a></td> <td><a href="teaser/2.jpg"><img src="teaser/2.jpg" style="width: 150px; height: 200px;" alt="teaser"></a></td> <td><a href="teaser/3.jpg"><img src="teaser/3.jpg" style="width: 150px; height: 200px;" alt="teaser"></a></td> <td><a href="teaser/5.jpg"><img src="teaser/5.jpg" style="width: 150px; height: 200px;" alt="teaser"></a></td> <td><a href="teaser/6.jpg"><img src="teaser/6.jpg" style="width: 150px; height: 200px;" alt="teaser"></a></td> </tr> <li> Illustrations of different plants from the same manuscript </li> <tr> <td><a href="teaser/c.jpg"><img src="teaser/c.jpg" style="width: 250px; height: 200px;" alt="teaser"></a></td> <td><a href="teaser/b.jpg"><img src="teaser/b.jpg" style="width: 250px; height: 200px;" alt="teaser"></a></td> <td><a href="teaser/a.jpg"><img src="teaser/a.jpg" style="width: 250px; height: 200px;" alt="teaser"></a></td> </tr> <li> Same illustration in three manuscripts with different styles </li> </div> <h2>Abstract</h2> <p> Illustrations are an essential transmission instrument. For an historian, the first step in studying their evolution in a corpus of similar manuscripts is to identify which ones correspond to each other. This image collation task is daunting for manuscripts separated by many lost copies, spreading over centuries, which might have been completely re-organized and greatly modified to adapt to novel knowledge or belief and include hundreds of illustrations. Our contributions in this paper are threefold. First, we introduce the task of illustration collation and a large annotated public dataset to evaluate solutions, including 6 manuscripts of 2 different texts with more than 2 000 illustrations and 1 200 annotated correspondences. Second, we analyze state of the art similarity measures for this task and show that they succeed in simple cases but struggle for large manuscripts when the illustrations have undergone very significant changes and are discriminated only by fine details. Finally, we show clear evidence that significant performance boosts can be expected by exploiting cycle-consistent correspondences. </p> <h2>1 min Video</h2> <center> <p align="center"> <table> <tr> <td><iframe width="800" height="400" style="max-width: 800px;" src="https://www.youtube.com/embed/yXe7JHSJDUs" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> </td> </tr> </table> </p> </center> <p> <h2>Dataset </h2> <br /> <br /> <li> The manuscripts in our dataset come from three different texts, have diverse number of illustrations and come from diverse. In total, it includes more than 2 000 illustrations and 1 300 annotated correspondences. </li> <li> To download the dataset, please refer to <a href="https://github.com/Rykoua/ImageCollation/tree/main/ManuscriptDownloader"> ManuscriptDownloader </a>. </li> <div class="container" style="margin-bottom: 10px; text-align: center;"> <img id="table" src="annotation.JPG" style="width: 100%; max-width: 600px;"> </div> </p> <p> <h2>Visual Results</h2> <br /> <br /> <ul> <center> <div class="container" style="margin-bottom: 10px; text-align: center;"> <tr> <td><a href="qual/r0.jpg"><img src="qual/r0.jpg" style="width: 120px; height: 150px; border: 5px solid #0000FF;" alt="teaser"></a></td> <td><a href="qual/r1.jpg"><img src="qual/r1.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/r2.jpg"><img src="qual/r2.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/r3.jpg"><img src="qual/r3.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/r4.jpg"><img src="qual/r4.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/r5.jpg"><img src="qual/r5.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <li><font color="#0000FF">Query</font> and 5 nearest neighbors according to our similarity score</li> </tr> <tr> <td><a href="qual/r0.jpg"><img src="qual/r0.jpg" style="width: 120px; height: 150px; border: 5px solid #0000FF;" alt="teaser"></a></td> <td><a href="qual/n1.jpg"><img src="qual/n1.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/n2.jpg"><img src="qual/n2.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/n3.jpg"><img src="qual/n3.jpg" style="width: 120px; height: 150px; border: 5px solid #00FF00;" alt="teaser"></a></td> <td><a href="qual/n4.jpg"><img src="qual/n4.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/n5.jpg"><img src="qual/n5.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <li><font color="#0000FF">Query</font> and 5 nearest neighbors (correct in <font color="#00FF00">Green</font>) according to our normalized score</li> </tr> <tr> <td><a href="qual/r0.jpg"><img src="qual/r0.jpg" style="width: 120px; height: 150px; border: 5px solid #0000FF;" alt="teaser"></a></td> <td><a href="qual/p1.jpg"><img src="qual/p1.jpg" style="width: 120px; height: 150px; border: 5px solid #00FF00;" alt="teaser"></a></td> <td><a href="qual/p2.jpg"><img src="qual/p2.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/p3.jpg"><img src="qual/p3.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/p4.jpg"><img src="qual/p4.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <td><a href="qual/p5.jpg"><img src="qual/p5.jpg" style="width: 120px; height: 150px;" alt="teaser"></a></td> <li><font color="#0000FF">Query</font> and 5 nearest neighbors (correct in <font color="#00FF00">Green</font>) after information propagation</li> </tr> </div> </center> <br /> <br /> <h2>Code and Paper</h2> <div class="row"> <div class="col text-center"> <p> <a href="https://github.com/Rykoua/ImageCollation" class="d-inline-block p-3 align-top"><img height="150" width="117" src="github.JPG" style="border:1px solid" data-nothumb=""><br>Code</a> <a href="https://arxiv.org/abs/2108.08109" class="d-inline-block p-3 align-top"><img height="150" width="" src="icdar_paper.JPG" style="border:1px solid" data-nothumb=""><br>Paper<br></a> <a href="http://imagine.enpc.fr/~shenx/ImageCollation/ICDAR_ImageCollation_Slides.pdf" class="d-inline-block p-3 align-top"><img height="150" width="260" src="icdar_slides.JPG" style="border:1px solid" data-nothumb=""><br>Slides<br></a> <a href="http://imagine.enpc.fr/~shenx/ImageCollation/icdar21.pdf" class="d-inline-block p-3 align-top"><img height="150" width="300" src="icdar_poster.JPG" style="border:1px solid" data-nothumb=""><br>Poster<br></a> </p> </div> </div> <p> To cite our paper, </p> <div class="card"> <div class="card-block"> <pre class="card-text clickselect">@inproceedings{kaoua2021imagecollation, title={Image Collation: Matching illustrations in manuscripts}, author={Kaoua, Ryad and Shen, Xi and Durr, Alexandra and Lazaris, Stavros and Picard, David and Aubry, Mathieu}, booktitle={International Conference on Document Analysis and Recognition (ICDAR)}, year={2021} } </pre> </div> </div> <br /> <br /> <br /> <br /> <h2>Acknowledgment</h2> <p> This work was supported in part by ANR project <a href="https://enherit.enpc.fr/">EnHerit</a> ANR-17-CE23-0008, project Rapid Tabasco, and gifts from Adobe. We thank <a href="https://webusers.imj-prg.fr/~alexandre.guilbaud/">Alexandre Guilbaud </a>for fruitful discussions.</p> <br /> <br />  <br /> <br /> <p></p> <p></p> <p></p> </div> </div> </div>  </body> <div id="coFrameDiv" style="height:0px;display:none;"><iframe id="coToolbarFrame" src="chrome-extension://cjabmdjcfcfdmffimndhafhblfmpjdpe/toolbar/placeholder.html" style="height:0px;width:100%;display:none;"></iframe></div> </html>

CINXE.COM

Image Collation