Detecting Syntactic Change Using a Neural Part-of-Speech Tagger

<!doctype html><html lang=en-us><head><meta charset=utf-8><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1,shrink-to-fit=no"><title>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger - ACL Anthology</title><meta name=generator content="Hugo 0.118.2"><link href=/aclicon.ico rel="shortcut icon" type=image/x-icon><link rel=stylesheet href=/css/main.min.b1d14a9a8f6bb9c608ca4de9aad72a6e06945119f97951f2908522dc220e6277.css media=screen><link rel=stylesheet href=https://use.fontawesome.com/releases/v5.7.2/css/all.css integrity=sha384-fnmOCqbTlWIlj8LyTjo7mOUStjsKC4pOpQbqyi7RrhN7udi9RwhKkMHpvLbHG9Sr crossorigin=anonymous><link rel=stylesheet href=/css/academicons.min.css><meta content="Detecting Syntactic Change Using a Neural Part-of-Speech Tagger" name=citation_title><meta content="William Merrill" name=citation_author><meta content="Gigi Stark" name=citation_author><meta content="Robert Frank" name=citation_author><meta content="Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change" name=citation_conference_title><meta content="2019/8" name=citation_publication_date><meta content="https://aclanthology.org/W19-4721.pdf" name=citation_pdf_url><meta content="167" name=citation_firstpage><meta content="174" name=citation_lastpage><meta content="10.18653/v1/W19-4721" name=citation_doi><meta property="og:title" content="Detecting Syntactic Change Using a Neural Part-of-Speech Tagger"><meta property="og:image" content="https://aclanthology.org/thumb/W19-4721.jpg"><meta property="og:image:alt" content="First page of paper PDF."><meta property="og:type" content="article"><meta property="og:site_name" content="ACL Anthology"><meta property="og:url" content="https://aclanthology.org/W19-4721"><meta property="og:description" content="William Merrill, Gigi Stark, Robert Frank. Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change. 2019."><link rel=canonical href=https://aclanthology.org/W19-4721></head><body><nav class="navbar navbar-expand-sm navbar-light bg-light bg-gradient-light shadow-sm py-0 mb-3 mb-md-4 mb-xl-5"><div id=navbar-container class=container><a class=navbar-brand href=/><img src=/images/acl-logo.svg width=56 alt="ACL Logo"> <span class="d-inline pl-2">ACL Anthology</span></a> <button class=navbar-toggler type=button data-toggle=collapse data-target=#navbarSupportedContent aria-controls=navbarSupportedContent aria-expanded=false aria-label="Toggle navigation"> <span class=navbar-toggler-icon></span></button><div class="collapse navbar-collapse" id=navbarSupportedContent><ul class="navbar-nav flex-grow-1 pr-md-2"><li class=nav-item><a class=nav-link href=/posts/>News<span class=sr-only>(current)</span></a></li><li class=nav-item><a class=nav-link href=/faq/>FAQ<span class=sr-only>(current)</span></a></li><li class=nav-item><a class=nav-link href=/info/corrections/>Corrections<span class=sr-only>(current)</span></a></li><li class=nav-item><a class=nav-link href=/info/contrib/>Submissions<span class=sr-only>(current)</span></a></li><li class=nav-item><a class=nav-link href=https://github.com/acl-org/acl-anthology/><i class="fab fa-github pr-1"></i>Github</a></li></ul><form class="form-inline my-2 my-lg-0 flex-nowrap" action=/search/? method=get><input id=acl-search-box class="form-control mr-sm-2" name=q type=search placeholder=Search... aria-label=Search> <button class="btn btn-outline-primary" type=submit><i class="fas fa-search"></i></button></form></div></div></nav><div id=main-container class=container><section id=main><div><h2 id=title><a href=https://aclanthology.org/W19-4721.pdf>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger</a></h2><p class=lead><a href=/people/w/william-merrill/>William Merrill</a>, <a href=/people/g/gigi-stark/>Gigi Stark</a>, <a href=/people/r/robert-frank/>Robert Frank</a></p></div><hr><div class="row acl-paper-details"><div class="col col-lg-10 order-2"><div class="card bg-light mb-2 mb-lg-3"><div class="card-body acl-abstract"><h5 class=card-title>Abstract</h5><span>We train a diachronic long short-term memory (LSTM) part-of-speech tagger on a large corpus of American English from the 19th, 20th, and 21st centuries. We analyze the tagger’s ability to implicitly learn temporal structure between years, and the extent to which this knowledge can be transferred to date new sentences. The learned year embeddings show a strong linear correlation between their first principal component and time. We show that temporal information encoded in the model can be used to predict novel sentences’ years of composition relatively well. Comparisons to a feedforward baseline suggest that the temporal change learned by the LSTM is syntactic rather than purely lexical. Thus, our results suggest that our tagger is implicitly learning to model syntactic change in American English over the course of the 19th, 20th, and early 21st centuries.</span></div></div><dl><dt>Anthology ID:</dt><dd>W19-4721</dd><dt>Volume:</dt><dd><a href=/volumes/W19-47/>Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change</a></dd><dt>Month:</dt><dd>August</dd><dt>Year:</dt><dd>2019</dd><dt>Address:</dt><dd>Florence, Italy</dd><dt>Editors:</dt><dd><a href=/people/n/nina-tahmasebi/>Nina Tahmasebi</a>, <a href=/people/l/lars-borin/>Lars Borin</a>, <a href=/people/a/adam-jatowt/>Adam Jatowt</a>, <a href=/people/y/yang-xu/>Yang Xu</a></dd><dt>Venue:</dt><dd><a href=/venues/lchange/>LChange</a></dd><dt>SIG:</dt><dd></dd><dt>Publisher:</dt><dd>Association for Computational Linguistics</dd><dt>Note:</dt><dd></dd><dt>Pages:</dt><dd>167–174</dd><dt>Language:</dt><dd></dd><dt>URL:</dt><dd><a href=https://aclanthology.org/W19-4721>https://aclanthology.org/W19-4721</a></dd><dt>DOI:</dt><dd><a href=https://doi.org/10.18653/v1/W19-4721 title="To the current version of the paper by DOI">10.18653/v1/W19-4721</a></dd><dt class=acl-button-row>Bibkey:</dt><dd class=acl-button-row><button type=button class="btn btn-clipboard-outside btn-secondary btn-sm d-none" data-clipboard-target=#citePaperBibkey><i class="far fa-clipboard"></i><span id=citePaperBibkey class="pl-2 text-monospace">merrill-etal-2019-detecting</span></button></dd><dt>Cite (ACL):</dt><dd><span id=citeACL>William Merrill, Gigi Stark, and Robert Frank. 2019. <a href=https://aclanthology.org/W19-4721>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger</a>. In <i>Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change</i>, pages 167–174, Florence, Italy. Association for Computational Linguistics.</span><button type=button class="btn btn-clipboard btn-secondary btn-sm d-none ml-2" data-clipboard-target=#citeACL><i class="far fa-clipboard"></i></button></dd><dt>Cite (Informal):</dt><dd><span id=citeRichText><a href=https://aclanthology.org/W19-4721>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger</a> (Merrill et al., LChange 2019)</span><button type=button class="btn btn-clipboard btn-secondary btn-sm d-none ml-2" data-clipboard-target=#citeRichText><i class="far fa-clipboard"></i></button></dd><dt class=acl-button-row>Copy Citation:</dt><dd class=acl-button-row><button type=button class="btn btn-clipboard-outside btn-secondary btn-sm d-none" data-clipboard-target=#citeBibtexContent><i class="far fa-clipboard pr-2"></i>BibTeX</button> <button type=button class="btn btn-clipboard-outside btn-secondary btn-sm d-none" data-clipboard-target=#citeMarkdownContent><i class="far fa-clipboard pr-2"></i>Markdown</button> <button type=button class="btn btn-clipboard-outside btn-secondary btn-sm d-none" data-clipboard-target=#citeModsContent><i class="far fa-clipboard pr-2"></i>MODS XML</button> <button type=button class="btn btn-clipboard-outside btn-secondary btn-sm d-none" data-clipboard-target=#citeEndnoteContent><i class="far fa-clipboard pr-2"></i>Endnote</button> <button type=button class="btn btn-secondary btn-sm" data-toggle=modal data-target=#citeModal>More options…</button></dd><dt>PDF:</dt><dd><a href=https://aclanthology.org/W19-4721.pdf>https://aclanthology.org/W19-4721.pdf</a></dd><dt>Code</dt><dd><a href=https://github.com/viking-sudo-rm/DiachronicPOSTagger><i class="fab fa-github"></i> viking-sudo-rm/DiachronicPOSTagger</a></dd></dl></div><div class=acl-paper-link-block><a class="btn btn-primary" href=https://aclanthology.org/W19-4721.pdf title="Open PDF of 'Detecting Syntactic Change Using a Neural Part-of-Speech Tagger'"><i class="far fa-file-pdf"></i><span class=pl-2>PDF</span></a> <a class="btn btn-secondary" title="Open dialog for exporting citations" data-toggle=modal data-target=#citeModal href=#><i class="fas fa-quote-left"></i><span class=pl-2>Cite</span></a> <a class="btn btn-secondary" href="https://www.semanticscholar.org/search?q=Detecting+Syntactic+Change+Using+a+Neural+Part-of-Speech+Tagger" title="Search for 'Detecting Syntactic Change Using a Neural Part-of-Speech Tagger' on Semantic Scholar"><i class="ai ai-semantic-scholar"></i><span class="pl-sm-2 d-none d-sm-inline">Search</span></a> <a class="btn btn-secondary d-flex flex-wrap justify-content-center" href="https://paperswithcode.com/paper/?acl=W19-4721" title="Code for 'Detecting Syntactic Change Using a Neural Part-of-Speech Tagger' on Papers with Code"><svg xmlns="http://www.w3.org/2000/svg" class="pwc-icon-big" viewBox="0 0 512 512"><path stroke="#4d8093" fill="#4d8093" d="M88 128h48v256H88zm144 0h48v256h-48zm-72 16h48v224h-48zm144 0h48v224h-48zm72-16h48v256h-48z"/><path stroke="#4d8093" fill="#4d8093" d="M104 104V56H16v4e2h88v-48H64V104zM408 56v48h40v304h-40v48h88V56z"/></svg><span class="pl-sm-2 d-none d-sm-inline">Code</span></a></div></div><hr><div class="modal fade" id=citeModal tabindex=-1 role=dialog aria-labelledby=citeModalLabel aria-hidden=true><div class="modal-dialog modal-lg" role=document><div class=modal-content><div class=modal-header><h5 class=modal-title id=citeModalLabel>Export citation</h5><button class=close data-dismiss=modal aria-label=Close> <span aria-hidden=true>×</span></button></div><div class=modal-body><ul class="nav nav-tabs mb-2" id=citeFormats role=tablist><li class=nav-item><a class="nav-link active" data-toggle=list href=#citeBibtex role=tab aria-controls=citeBibtex aria-selected=true>BibTeX</a></li><li class=nav-item><a class=nav-link data-toggle=list href=#citeMods role=tab aria-controls=citeMods aria-selected=false>MODS XML</a></li><li class=nav-item><a class=nav-link data-toggle=list href=#citeEndnote role=tab aria-controls=citeEndnote aria-selected=false>Endnote</a></li><li class=nav-item><a class=nav-link data-toggle=list href=#citeMarkdown role=tab aria-controls=citeMarkdown aria-selected=false>Preformatted</a></li></ul><div class=tab-content id=citeFormatsContent><div class="tab-pane active" id=citeBibtex role=tabpanel><pre id=citeBibtexContent class="bg-light border p-2" style=max-height:50vh>@inproceedings{merrill-etal-2019-detecting, title = "Detecting Syntactic Change Using a Neural Part-of-Speech Tagger", author = "Merrill, William and Stark, Gigi and Frank, Robert", editor = "Tahmasebi, Nina and Borin, Lars and Jatowt, Adam and Xu, Yang", booktitle = "Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change", month = aug, year = "2019", address = "Florence, Italy", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/W19-4721", doi = "10.18653/v1/W19-4721", pages = "167--174", abstract = "We train a diachronic long short-term memory (LSTM) part-of-speech tagger on a large corpus of American English from the 19th, 20th, and 21st centuries. We analyze the tagger{'}s ability to implicitly learn temporal structure between years, and the extent to which this knowledge can be transferred to date new sentences. The learned year embeddings show a strong linear correlation between their first principal component and time. We show that temporal information encoded in the model can be used to predict novel sentences{'} years of composition relatively well. Comparisons to a feedforward baseline suggest that the temporal change learned by the LSTM is syntactic rather than purely lexical. Thus, our results suggest that our tagger is implicitly learning to model syntactic change in American English over the course of the 19th, 20th, and early 21st centuries.", } </pre><div class="modal-footer pb-1"><a class="btn btn-secondary" href=/W19-4721.bib><i class="fas fa-download pr-2"></i>Download as File</a> <button class="btn btn-clipboard btn-primary d-none" data-clipboard-target=#citeBibtexContent><i class="far fa-clipboard pr-2"></i>Copy to Clipboard</button></div></div><div class=tab-pane id=citeMods role=tabpanel><pre id=citeModsContent class="bg-light border p-2" style=max-height:50vh><?xml version="1.0" encoding="UTF-8"?> <modsCollection xmlns="http://www.loc.gov/mods/v3"> <mods ID="merrill-etal-2019-detecting"> <titleInfo> <title>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger</title> </titleInfo> <name type="personal"> <namePart type="given">William</namePart> <namePart type="family">Merrill</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Gigi</namePart> <namePart type="family">Stark</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Robert</namePart> <namePart type="family">Frank</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <originInfo> <dateIssued>2019-08</dateIssued> </originInfo> <typeOfResource>text</typeOfResource> <relatedItem type="host"> <titleInfo> <title>Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change</title> </titleInfo> <name type="personal"> <namePart type="given">Nina</namePart> <namePart type="family">Tahmasebi</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Lars</namePart> <namePart type="family">Borin</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Adam</namePart> <namePart type="family">Jatowt</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Yang</namePart> <namePart type="family">Xu</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <originInfo> <publisher>Association for Computational Linguistics</publisher> <place> <placeTerm type="text">Florence, Italy</placeTerm> </place> </originInfo> <genre authority="marcgt">conference publication</genre> </relatedItem> <abstract>We train a diachronic long short-term memory (LSTM) part-of-speech tagger on a large corpus of American English from the 19th, 20th, and 21st centuries. We analyze the tagger’s ability to implicitly learn temporal structure between years, and the extent to which this knowledge can be transferred to date new sentences. The learned year embeddings show a strong linear correlation between their first principal component and time. We show that temporal information encoded in the model can be used to predict novel sentences’ years of composition relatively well. Comparisons to a feedforward baseline suggest that the temporal change learned by the LSTM is syntactic rather than purely lexical. Thus, our results suggest that our tagger is implicitly learning to model syntactic change in American English over the course of the 19th, 20th, and early 21st centuries.</abstract> <identifier type="citekey">merrill-etal-2019-detecting</identifier> <identifier type="doi">10.18653/v1/W19-4721</identifier> <location> <url>https://aclanthology.org/W19-4721</url> </location> <part> <date>2019-08</date> <extent unit="page"> <start>167</start> <end>174</end> </extent> </part> </mods> </modsCollection> </pre><div class="modal-footer pb-1"><a class="btn btn-secondary" href=/W19-4721.xml><i class="fas fa-download pr-2"></i>Download as File</a> <button class="btn btn-clipboard btn-primary d-none" data-clipboard-target=#citeModsContent><i class="far fa-clipboard pr-2"></i>Copy to Clipboard</button></div></div><div class=tab-pane id=citeEndnote role=tabpanel><pre id=citeEndnoteContent class="bg-light border p-2" style=max-height:50vh>%0 Conference Proceedings %T Detecting Syntactic Change Using a Neural Part-of-Speech Tagger %A Merrill, William %A Stark, Gigi %A Frank, Robert %Y Tahmasebi, Nina %Y Borin, Lars %Y Jatowt, Adam %Y Xu, Yang %S Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change %D 2019 %8 August %I Association for Computational Linguistics %C Florence, Italy %F merrill-etal-2019-detecting %X We train a diachronic long short-term memory (LSTM) part-of-speech tagger on a large corpus of American English from the 19th, 20th, and 21st centuries. We analyze the tagger’s ability to implicitly learn temporal structure between years, and the extent to which this knowledge can be transferred to date new sentences. The learned year embeddings show a strong linear correlation between their first principal component and time. We show that temporal information encoded in the model can be used to predict novel sentences’ years of composition relatively well. Comparisons to a feedforward baseline suggest that the temporal change learned by the LSTM is syntactic rather than purely lexical. Thus, our results suggest that our tagger is implicitly learning to model syntactic change in American English over the course of the 19th, 20th, and early 21st centuries. %R 10.18653/v1/W19-4721 %U https://aclanthology.org/W19-4721 %U https://doi.org/10.18653/v1/W19-4721 %P 167-174 </pre><div class="modal-footer pb-1"><a class="btn btn-secondary" href=/W19-4721.endf><i class="fas fa-download pr-2"></i>Download as File</a> <button class="btn btn-clipboard btn-primary d-none" data-clipboard-target=#citeEndnoteContent><i class="far fa-clipboard pr-2"></i>Copy to Clipboard</button></div></div><div class=tab-pane id=citeMarkdown role=tabpanel><h5>Markdown (Informal)</h5><p id=citeMarkdownContent class="text-monospace small bg-light border p-2">[Detecting Syntactic Change Using a Neural Part-of-Speech Tagger](https://aclanthology.org/W19-4721) (Merrill et al., LChange 2019)</p><ul class=mt-2><li><a href=https://aclanthology.org/W19-4721>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger</a> (Merrill et al., LChange 2019)</li></ul><h5>ACL</h5><ul class=mt-2><li id=citeACLstyleContent>William Merrill, Gigi Stark, and Robert Frank. 2019. <a href=https://aclanthology.org/W19-4721>Detecting Syntactic Change Using a Neural Part-of-Speech Tagger</a>. In <i>Proceedings of the 1st International Workshop on Computational Approaches to Historical Language Change</i>, pages 167–174, Florence, Italy. Association for Computational Linguistics.</li></ul><div class="modal-footer pb-1"><button type=button class="btn btn-clipboard btn-primary d-none" data-clipboard-target=#citeMarkdownContent><i class="far fa-clipboard pr-2"></i>Copy Markdown to Clipboard</button> <button type=button class="btn btn-clipboard btn-primary d-none" data-clipboard-target=#citeACLstyleContent><i class="far fa-clipboard pr-2"></i>Copy ACL to Clipboard</button></div></div></div></div></div></div></div></section></div><footer class="bg-gradient-light py-2 py-xl-3 mt-3 mt-md-4 mt-xl-5"><div class=container><p class="text-muted small px-1"><span class="float-right mt-2 ml-2"><a rel=license href=http://creativecommons.org/licenses/by/4.0/><img alt="Creative Commons License" style=border-width:0 src=https://i.creativecommons.org/l/by/4.0/88x31.png></a></span> ACL materials are Copyright © 1963–2024 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the <a href=https://creativecommons.org/licenses/by-nc-sa/3.0/>Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License</a>. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a <a href=https://creativecommons.org/licenses/by/4.0/>Creative Commons Attribution 4.0 International License</a>.</p><p class="text-muted small px-1">The ACL Anthology is managed and built by the <a href=/info/credits/>ACL Anthology team</a> of volunteers.</p><p class="text-muted small px-1"><i>Site last built on 24 November 2024 at 17:31 UTC with <a href=https://github.com/acl-org/acl-anthology/tree/5f12c57adb4bba314cd6a9a6fdec63d6c3848719>commit 5f12c57</a>.</i></p></div></footer><script src=https://code.jquery.com/jquery-3.3.1.slim.min.js integrity=sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo crossorigin=anonymous></script> <script src=https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.6/umd/popper.min.js integrity=sha384-wHAiFfRlMFy6i5SRaxvfOCifBUQy1xHdJ/yoi7FRNXMRBu5WHdZYu1hA6ZOblgut crossorigin=anonymous></script> <script src=https://stackpath.bootstrapcdn.com/bootstrap/4.2.1/js/bootstrap.min.js integrity=sha384-B0UglyR+jN6CkvvICOB2joaf5I4l3gm9GU6Hc1og6Ls7i6U/mkkaduKaBhlAXv9k crossorigin=anonymous></script> <script>$(function(){$('[data-toggle="tooltip"]').tooltip(),$("#toggle-all-abstracts")&&($("#toggle-all-abstracts").click(function(){var e=$("#toggle-all-abstracts");e.attr("disabled",!0),e.attr("data-toggle-state")=="hide"?($(".abstract-collapse").collapse("show"),e.attr("data-toggle-state","show")):($(".abstract-collapse").collapse("hide"),e.attr("data-toggle-state","hide")),e.attr("disabled",!1)}),$("#toggle-all-abstracts").attr("disabled",!1))})</script><script src=/js/clipboard.min.js></script> <script>$(document).ready(function(){if(ClipboardJS.isSupported()){success_fn=function(e){var t=$(e.trigger);t.toggleClass("btn-success"),t.children("i").toggleClass("far fa-clipboard fas fa-clipboard-check"),e.clearSelection(),setTimeout(function(){t.toggleClass("btn-success"),t.children("i").toggleClass("far fa-clipboard fas fa-clipboard-check")},2e3)};var e,t=new ClipboardJS(".btn-clipboard");t.on("success",success_fn),$(".btn-clipboard").removeClass("d-none"),e=new ClipboardJS(".btn-clipboard-outside",{text:function(e){var t=e.getAttribute("data-clipboard-target");return $(t).text()}}),e.on("success",success_fn),$(".btn-clipboard-outside").removeClass("d-none")}})</script></body></html>

CINXE.COM

Detecting Syntactic Change Using a Neural Part-of-Speech Tagger - ACL Anthology