CINXE.COM

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">     <html xml:lang="en" lang="en" dir="ltr" >    <head profile="http://www.w3.org/1999/xhtml/vocab"> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <link rel=”apple-touch-icon-precomposed” href=”/sites/default/themes/bmj/the_bmj/img/icon.png”/> <meta http-equiv="X-UA-Compatible" content="IE=edge">  <link rel="dns-prefetch" href="//public.flourish.studio" /> <link rel="dns-prefetch" href="//static.polldaddy.com" /> <link rel="dns-prefetch" href="//ajax.googleapis.com" /> <link rel="dns-prefetch" href="//crossmark-cdn.crossref.org" /> <link rel="dns-prefetch" href="//d1bxh8uas1mnw7.cloudfront.net" /> <link rel="dns-prefetch" href="//www.google.com" /> <link rel="dns-prefetch" href="//scholar.google.com" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="shortcut icon" href="https://www.bmj.com/sites/default/themes/bmj/the_bmj/favicon.ico" type="image/vnd.microsoft.icon" /> <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet" media="all" type="text/css" /> <link rel="alternate" type="application/pdf" title="Full Text (PDF)" href="/content/387/bmj-2024-081948.full.pdf" /> <link rel="alternate" type="text/plain" title="Full Text (Plain)" href="/content/387/bmj-2024-081948.full.txt" /> <script>window.dataLayer = window.dataLayer || [];window.dataLayer.push({"page":{"hwTitle":"Age against the machine\u2014susceptibility of large language models to cognitive impairment: cross sectional analysis","hwUrl":"https://www.bmj.com/content/387/bmj-2024-081948","hwPath":"/content/387/bmj-2024-081948/rapid-responses","hwType":"journal page"},"user":{"hwMemberID":"","hwLoggedIntoInstitutionID":"","hwLoggedIntoInstitution":"","hwAccessUsageTerms":"guest","hwRegisteredUser":"no","hwHasFullTextAccess":"yes","hwEntitled":"no","mail":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"},"content":{"hwIssueDate":"2024-12-20","hwIssueDateTime":"2024-12-20 06:41:07","hwPublicationDate":"2024-12-20","hwPublicationDateTime":"2024-12-20 06:41:07","hwNlmArticleType":"research-article","hwDOI":"10.1136/bmj-2024-081948","hwAuthors":"Roy Dayan,Benjamin Uliel,Gal Koplewitz","hwContributors":"","hwIsEarlyRelease":"no","hwEissn":"1756-1833","hwPissn":"","hwVolume":"387","hwIssue":"","hwArticleType":"Research","hwTabView":"rapid-responses","hwViewType":"full","hwAccessType":"OpenAccess","hwOpenAccess":"yes","hwFreeAccess":"yes","hwTaxonomy":"Christmas","hwCorpusCode":"bmj","hwType":"article","hwSubtitle":null,"hwOverline":null,"hwPisaId":"bmj;387/dec18_11/e081948","hwFirstPage":null,"hwLastPage":null,"hwIsLatestVersion":"yes","hwIsCurrentIssue":"no","hwIsOpenIssue":"no","hwHasFullText":"yes","hwHasPDF":"yes","hwParentTitle":"BMJ: 387","hwElectronicPubDate":"2024-12-20","hwElectronicPubDateTime":"2024-12-20 06:41:07"}}); //--><!]]></script> <meta name="type" content="article" /> <meta name="category" content="research-article" /> <meta name="HW.identifier" content="/bmj/387/bmj-2024-081948.atom" /> <meta name="HW.pisa" content="bmj;387/dec18_11/e081948" /> <meta name="DC.Format" content="text/html" /> <meta name="DC.Language" content="en" /> <meta name="DC.Title" content="Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis" /> <meta name="DC.Identifier" content="10.1136/bmj-2024-081948" /> <meta name="DC.Date" content="2024-12-20" /> <meta name="DC.Publisher" content="British Medical Journal Publishing Group" /> <meta name="DC.Rights" content="© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.. http://creativecommons.org/licenses/by-nc/4.0/This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/." /> <meta name="DC.AccessRights" content="open-access" /> <meta name="DC.Relation" content="10.1136/bmj-2022-072826" /> <meta name="DC.Relation" content="10.1136/bmj-2023-078538" /> <meta name="DC.Relation" content="10.1136/bmj.k4563" /> <meta name="DC.Relation" content="10.1136/bmj.r27" /> <meta name="DC.Description" content="Objective To evaluate the cognitive abilities of the leading large language models and identify their susceptibility to cognitive impairment, using the Montreal Cognitive Assessment (MoCA) and additional tests. Design Cross sectional analysis. Setting Online interaction with large language models via text based prompts. Participants Publicly available large language models, or “chatbots”: ChatGPT versions 4 and 4o (developed by OpenAI), Claude 3.5 “Sonnet” (developed by Anthropic), and Gemini versions 1 and 1.5 (developed by Alphabet). Assessments The MoCA test (version 8.1) was administered to the leading large language models with instructions identical to those given to human patients. Scoring followed official guidelines and was evaluated by a practising neurologist. Additional assessments included the Navon figure, cookie theft picture, Poppelreuter figure, and Stroop test. Main outcome measures MoCA scores, performance in visuospatial/executive tasks, and Stroop test results. Results ChatGPT 4o achieved the highest score on the MoCA test (26/30), followed by ChatGPT 4 and Claude (25/30), with Gemini 1.0 scoring lowest (16/30). All large language models showed poor performance in visuospatial/executive tasks. Gemini models failed at the delayed recall task. Only ChatGPT 4o succeeded in the incongruent stage of the Stroop test. Conclusions With the exception of ChatGPT 4o, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment. Moreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence. No additional data available." /> <meta name="DC.Contributor" content="Roy Dayan" /> <meta name="DC.Contributor" content="Benjamin Uliel" /> <meta name="DC.Contributor" content="Gal Koplewitz" /> <meta name="article:published_time" content="2024-12-20" /> <meta name="article:section" content="Research" /> <meta name="citation_title" content="Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis" /> <meta name="citation_abstract" lang="en" content="<h3>Abstract</h3> <h3>Objective</h3> <p>To evaluate the cognitive abilities of the leading large language models and identify their susceptibility to cognitive impairment, using the Montreal Cognitive Assessment (MoCA) and additional tests.</p><h3>Design</h3> <p>Cross sectional analysis.</p><h3>Setting</h3> <p>Online interaction with large language models via text based prompts.</p><h3>Participants</h3> <p>Publicly available large language models, or “chatbots”: ChatGPT versions 4 and 4o (developed by OpenAI), Claude 3.5 “Sonnet” (developed by Anthropic), and Gemini versions 1 and 1.5 (developed by Alphabet).</p><h3>Assessments</h3> <p>The MoCA test (version 8.1) was administered to the leading large language models with instructions identical to those given to human patients. Scoring followed official guidelines and was evaluated by a practising neurologist. Additional assessments included the Navon figure, cookie theft picture, Poppelreuter figure, and Stroop test.</p><h3>Main outcome measures</h3> <p>MoCA scores, performance in visuospatial/executive tasks, and Stroop test results.</p><h3>Results</h3> <p>ChatGPT 4o achieved the highest score on the MoCA test (26/30), followed by ChatGPT 4 and Claude (25/30), with Gemini 1.0 scoring lowest (16/30). All large language models showed poor performance in visuospatial/executive tasks. Gemini models failed at the delayed recall task. Only ChatGPT 4o succeeded in the incongruent stage of the Stroop test.</p><h3>Conclusions</h3> <p>With the exception of ChatGPT 4o, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment. Moreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence.</p>" /> <meta name="citation_journal_title" content="BMJ" /> <meta name="citation_publisher" content="British Medical Journal Publishing Group" /> <meta name="citation_publication_date" content="2024/12/20" /> <meta name="citation_mjid" content="bmj;387/dec18_11/e081948" /> <meta name="citation_id" content="387/dec18_11/e081948" /> <meta name="citation_public_url" content="https://www.bmj.com/content/387/bmj-2024-081948" /> <meta name="citation_abstract_html_url" content="https://www.bmj.com/content/387/bmj-2024-081948.abstract" /> <meta name="citation_full_html_url" content="https://www.bmj.com/content/387/bmj-2024-081948.full" /> <meta name="citation_pdf_url" content="https://www.bmj.com/content/bmj/387/bmj-2024-081948.full.pdf" /> <meta name="citation_issn" content="1756-1833" /> <meta name="citation_journal_abbrev" content="BMJ" /> <meta name="citation_doi" content="10.1136/bmj-2024-081948" /> <meta name="citation_pmid" content="39706600" /> <meta name="citation_volume" content="387" /> <meta name="citation_article_type" content="Research Article" /> <meta name="citation_section" content="Research" /> <meta name="citation_access" content="all" /> <meta name="citation_author" content="Roy Dayan" /> <meta name="citation_author_institution" content="Department of Neurology, Hadassah Medical Center, Jerusalem, Israel" /> <meta name="citation_author_institution" content="Faculty of Medicine, Hebrew University, Jerusalem, Israel" /> <meta name="citation_author_orcid" content="https://orcid.org/0000-0003-3061-3740" /> <meta name="citation_author" content="Benjamin Uliel" /> <meta name="citation_author_institution" content="Department of Neurology, Hadassah Medical Center, Jerusalem, Israel" /> <meta name="citation_author_institution" content="Faculty of Medicine, Hebrew University, Jerusalem, Israel" /> <meta name="citation_author" content="Gal Koplewitz" /> <meta name="citation_author_institution" content="QuantumBlack Analytics, London, UK" /> <meta name="citation_author_institution" content="Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel" /> <meta name="citation_author_orcid" content="https://orcid.org/0000-0003-4906-0779" /> <meta name="citation_reference" content="citation_journal_title=N Engl J Med;citation_author=CJ. Haug;citation_author=JM. Drazen;citation_title=Artificial Intelligence and Machine Learning in Clinical Medicine, 2023;citation_pages=1201-8;citation_volume=388;citation_year=2023;citation_pmid=36988595;citation_doi=10.1056/NEJMra2302038" /> <meta name="citation_reference" content="citation_journal_title=BMJ;citation_author=J. Goldhahn;citation_author=V. Rampton;citation_author=GA. Spinas;citation_title=Could artificial intelligence make doctors obsolete?;citation_volume=363;citation_year=2018;citation_pmid=30404897;citation_doi=10.1136/bmj.k4563" /> <meta name="citation_reference" content="citation_journal_title=Sci Rep;citation_author=WH. Walters;citation_author=EI. Wilder;citation_title=Fabrication and errors in the bibliographic citations generated by ChatGPT;citation_pages=14045;citation_volume=13;citation_year=2023;citation_pmid=37679503;citation_doi=10.1038/s41598-023-41032-5" /> <meta name="citation_reference" content="citation_journal_title=NEJM AI;citation_author=U. Katz;citation_author=E. Cohen;citation_author=E. Shachar;citation_title=GPT versus Resident Physicians — A Benchmark Based on Official Board Scores;citation_volume=1;citation_year=2024;citation_issue=5;citation_doi=10.1056/AIdbp2300192" /> <meta name="citation_reference" content="citation_journal_title=Eur Heart J Digit Health;citation_author=I. Skalidis;citation_author=A. Cagnina;citation_author=W. Luangphiphat;citation_title=ChatGPT takes on the European Exam in Core Cardiology: an artificial intelligence success story?;citation_pages=279-81;citation_volume=4;citation_year=2023;citation_pmid=37265864;citation_doi=10.1093/ehjdh/ztad029" /> <meta name="citation_reference" content="citation_journal_title=Am J Med Sci;citation_author=A. Gencer;citation_author=S. Aydin;citation_title=Can ChatGPT pass the thoracic surgery exam?;citation_volume=366;citation_year=2023;citation_pmid=37549788;citation_doi=10.1016/j.amjms.2023.08.001" /> <meta name="citation_reference" content="citation_journal_title=Front Med (Lausanne);citation_author=M. Riedel;citation_author=K. Kaefinger;citation_author=A. Stuehrenberg;citation_title=ChatGPT’s performance in German OB/GYN exams - paving the way for AI-enhanced medical education and clinical practice;citation_pages=1296615;citation_volume=10;citation_year=2023;citation_pmid=38155661;citation_doi=10.3389/fmed.2023.1296615" /> <meta name="citation_reference" content="citation_journal_title=BMJ Neurol Open;citation_author=TC. Chen;citation_author=E. Multala;citation_author=P. Kearns;citation_title=Assessment of ChatGPT’s performance on neurology written board examination questions;citation_volume=5;citation_year=2023;citation_pmid=37936648;citation_doi=10.1136/bmjno-2023-000530" /> <meta name="citation_reference" content="citation_journal_title=BMJ;citation_author=SC. Shelmerdine;citation_author=H. Martin;citation_author=K. Shirodkar;citation_author=S. Shamshuddin;citation_author=JR. Weir-McCall;citation_title=Can artificial intelligence pass the Fellowship of the Royal College of Radiologists examination? Multi-reader diagnostic accuracy study;citation_volume=379;citation_year=2022;citation_pmid=36543352;citation_doi=10.1136/bmj-2022-072826" /> <meta name="citation_reference" content="citation_journal_title=Iran Endod J;citation_author=M. Farajollahi;citation_author=A. Modaberi;citation_title=Can ChatGPT pass the “Iranian Endodontics Specialist Board” exam?;citation_pages=192;citation_volume=18;citation_year=2023;citation_pmid=37431530" /> <meta name="citation_reference" content="citation_journal_title=J Chin Med Assoc;citation_author=TL. Weng;citation_author=YM. Wang;citation_author=S. Chang;citation_author=TJ. Chen;citation_author=SJ. Hwang;citation_title=ChatGPT failed Taiwan’s Family Medicine Board Exam;citation_pages=762-6;citation_volume=86;citation_year=2023;citation_pmid=37294147;citation_doi=10.1097/JCMA.0000000000000928" /> <meta name="citation_reference" content="citation_journal_title=J Shoulder Elbow Surg;citation_author=B. Fiedler;citation_author=EN. Azua;citation_author=T. Phillips;citation_author=AS. Ahmed;citation_title=ChatGPT performance on the American Shoulder and Elbow Surgeons maintenance of certification exam;citation_pages=1888-93;citation_volume=33;citation_year=2024;citation_pmid=38580067;citation_doi=10.1016/j.jse.2024.02.029" /> <meta name="citation_reference" content="Baker P. Biden Drops Out of Race, Scrambling the Campaign for the White House. 2024. https://www.nytimes.com/2024/07/21/us/politics/biden-drops-out.html." /> <meta name="citation_reference" content="Baker P. ‘Person. Woman. Man. Camera. TV.’ Didn’t Mean What Trump Hoped It Did. 2020. https://www.nytimes.com/2020/07/23/us/politics/person-woman-man-camera-tv-trump.html." /> <meta name="citation_reference" content="citation_journal_title=Nat Med;citation_author=P. Hager;citation_author=F. Jungmann;citation_author=R. Holland;citation_title=Evaluation and mitigation of the limitations of large language models in clinical decision-making;citation_volume=30;citation_year=2024;citation_pmid=38965432;citation_doi=10.1038/s41591-024-03097-1" /> <meta name="citation_reference" content="citation_journal_title=BMJ;citation_author=BD. Menz;citation_author=NM. Kuderer;citation_author=S. Bacchi;citation_title=Current safeguards, risk mitigation, and transparency measures of large language models against the generation of health disinformation: repeated cross sectional analysis;citation_volume=384;citation_year=2024;citation_pmid=38508682;citation_doi=10.1136/bmj-2023-078538" /> <meta name="citation_reference" content="citation_journal_title=Sci Rep;citation_author=S. Bandyopadhyay;citation_author=J. Wittmayer;citation_author=DJ. Libon;citation_author=P. Tighe;citation_author=C. Price;citation_author=P. Rashidi;citation_title=Explainable semi-supervised deep learning shows that dementia is associated with small, avocado-shaped clocks with irregularly placed hands;citation_pages=7384;citation_volume=13;citation_year=2023;citation_pmid=37149670;citation_doi=10.1038/s41598-023-34518-9" /> <meta name="citation_reference" content="citation_journal_title=Journal of the American Geriatrics Society;citation_journal_abbrev=J Am Geriatr Soc;citation_author=ZS. Nasreddine;citation_author=NA. Phillips;citation_author=V. Bédirian;citation_title=The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment.;citation_pages=695-699;citation_volume=53;citation_year=2005;citation_issue=4;citation_pmid=15817019;citation_doi=10.1111/j.1532-5415.2005.53221.x" /> <meta name="citation_reference" content="citation_journal_title=J Vis Impair Blind;citation_author=W. Wittich;citation_author=N. Phillips;citation_author=ZS. Nasreddine;citation_author=H. Chertkow;citation_title=Sensitivity and Specificity of the Montreal Cognitive Assessment Modified for Individuals who are Visually Impaired;citation_pages=360-8;citation_volume=104;citation_year=2010;citation_doi=10.1177/0145482X1010400606" /> <meta name="citation_reference" content="citation_journal_title=Cogn Psychol;citation_author=D. Navon;citation_title=Forest before trees: The precedence of global features in visual perception;citation_pages=353-83;citation_volume=9;citation_year=1977;citation_doi=10.1016/0010-0285(77)90012-3" /> <meta name="citation_reference" content="citation_title=Die psychischen Schädigungen durch Kopfschuss im Kriege 1914/17.;citation_author=W. Poppelreuter;citation_year=1917" /> <meta name="citation_reference" content="citation_journal_title=Archives of Clinical Neuropsychology;citation_journal_abbrev=Arch Clin Neuropsychol;citation_author=S. Della Sala;citation_author=M. Laiacona;citation_author=H. Spinnler;citation_author=C. Trivelli;citation_title=Poppelreuter-Ghent's Overlapping Figures Test: its sensitivity to age, and its clinical use.;citation_pages=511-534;citation_volume=10;citation_year=1995;citation_issue=6;citation_pmid=14588906;citation_doi=10.1016/0887-6177(94)00049-V" /> <meta name="citation_reference" content="citation_journal_title=J Exp Psychol;citation_author=JR. Stroop;citation_title=Studies of interference in serial verbal reactions;citation_pages=643-62;citation_volume=18;citation_year=1935;citation_doi=10.1037/h0054651" /> <meta name="citation_reference" content="citation_journal_title=Front Neurosci;citation_author=MP. Mattson;citation_title=Superior pattern processing is the essence of the evolved human brain;citation_volume=8;citation_year=2014;citation_pmid=25202234;citation_doi=10.3389/fnins.2014.00265" /> <meta name="citation_reference" content="citation_journal_title=Continuum (Minneap Minn);citation_author=GS. Day;citation_title=Rapidly Progressive Dementia;citation_pages=901-36;citation_volume=28;citation_year=2022;citation_pmid=35678409;citation_doi=10.1212/CON.0000000000001089" /> <meta name="citation_reference" content="citation_journal_title=Lancet. Neurology;citation_journal_abbrev=Lancet Neurol;citation_author=SJ. Crutch;citation_author=M. Lehmann;citation_author=JM. Schott;citation_author=GD. Rabinovici;citation_author=MN. Rossor;citation_author=NC. Fox;citation_title=Posterior cortical atrophy.;citation_pages=170-178;citation_volume=11;citation_year=2012;citation_issue=2;citation_pmid=22265212;citation_doi=10.1016/S1474-4422(11)70289-7" /> <meta name="citation_reference" content="citation_journal_abbrev=Sci Rep;citation_author=E. Shokri-Kojori;citation_author=MA. Motes;citation_author=B. Rypma;citation_author=DC. Krawczyk;citation_title=The network architecture of cortical processing in visuo-spatial reasoning.;citation_pages=411-411;citation_volume=2;citation_year=2012;citation_pmid=22624092;citation_doi=10.1038/srep00411" /> <meta name="citation_reference" content="citation_journal_title=Journal of the International Neuropsychological Society : JINS;citation_journal_abbrev=J Int Neuropsychol Soc;citation_author=RL. Hester;citation_author=GJ. Kinsella;citation_author=B. Ong;citation_title=Effect of age on forward and backward span tasks.;citation_pages=475-481;citation_volume=10;citation_year=2004;citation_issue=4;citation_pmid=15327726;citation_doi=10.1017/S1355617704104037" /> <meta name="citation_reference" content="Apple. The future is here: iPhone X. 2017. https://www.apple.com/newsroom/2017/09/the-future-is-here-iphone-x/." /> <meta name="citation_reference" content="citation_journal_title=Dement Neuropsychol;citation_author=E. Giannetti;citation_title=The possibility of physicalism;citation_pages=242-50;citation_volume=5;citation_year=2011;citation_pmid=29213751;citation_doi=10.1590/S1980-57642011DN05040002" /> <meta name="citation_reference" content="citation_title=The mind-body problem.;citation_author=J. Westphal;citation_year=2016;citation_doi=10.7551/mitpress/10776.001.0001" /> <meta name="citation_reference" content="citation_title=Cognitive assessment for clinicians.;citation_author=JR. Hodges;citation_year=2018" /> <meta name="citation_reference" content="citation_journal_title=NPJ Parkinsons Dis;citation_author=P. Ortelli;citation_author=D. Ferrazzoli;citation_author=V. Versace;citation_title=Optimization of cognitive assessment in Parkinsonisms by applying artificial intelligence to a comprehensive screening test;citation_pages=42;citation_volume=8;citation_year=2022;citation_pmid=35410449;citation_doi=10.1038/s41531-022-00304-z" /> <meta name="citation_reference" content="citation_journal_title=Front Psychiatry;citation_author=C. Kalafatis;citation_author=MH. Modarres;citation_author=P. Apostolou;citation_title=Validity and Cultural Generalisability of a 5-Minute AI-Based, Computerised Cognitive Assessment in Mild Cognitive Impairment and Alzheimer’s Dementia;citation_pages=706695;citation_volume=12;citation_year=2021;citation_pmid=34366938;citation_doi=10.3389/fpsyt.2021.706695" /> <meta name="citation_reference" content="citation_journal_title=J Geriatr Psychiatry Neurol;citation_author=B. Levy;citation_author=C. Hess;citation_author=J. Hogan;citation_title=Machine Learning Enhances the Efficiency of Cognitive Screenings for Primary Care;citation_pages=137-44;citation_volume=32;citation_year=2019;citation_pmid=30879363;citation_doi=10.1177/0891988719834349" /> <meta name="citation_reference" content="citation_journal_title=N Engl J Med;citation_author=G. Devi;citation_title=Alzheimer’s Disease in Physicians - Assessing Professional Competence and Tempering Stigma;citation_pages=1073-5;citation_volume=378;citation_year=2018;citation_pmid=29562159;citation_doi=10.1056/NEJMp1716381" /> <meta name="citation_fulltext_world_readable" content="" /> <meta name="twitter:title" content="Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis" /> <meta name="twitter:card" content="summary" /> <meta name="twitter:description" content="Objective To evaluate the cognitive abilities of the leading large language models and identify their susceptibility to cognitive impairment, using the Montreal Cognitive Assessment (MoCA) and additional tests. Design Cross sectional analysis. Setting Online interaction with large language models via text based prompts. Participants Publicly available large language models, or “chatbots”: ChatGPT versions 4 and 4o (developed by OpenAI), Claude 3.5 “Sonnet” (developed by Anthropic), and Gemini versions 1 and 1.5 (developed by Alphabet). Assessments The MoCA test (version 8.1) was administered to the leading large language models with instructions identical to those given to human patients. Scoring followed official guidelines and was evaluated by a practising neurologist. Additional assessments included the Navon figure, cookie theft picture, Poppelreuter figure, and Stroop test. Main outcome measures MoCA scores, performance in visuospatial/executive tasks, and Stroop test results. Results ChatGPT 4o achieved the highest score on the MoCA test (26/30), followed by ChatGPT 4 and Claude (25/30), with Gemini 1.0 scoring lowest (16/30). All large language models showed poor performance in visuospatial/executive tasks. Gemini models failed at the delayed recall task. Only ChatGPT 4o succeeded in the incongruent stage of the Stroop test. Conclusions With the exception of ChatGPT 4o, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment. Moreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence. No additional data available." /> <meta name="og-title" property="og:title" content="Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis" /> <meta name="og-url" property="og:url" content="https://www.bmj.com/content/387/bmj-2024-081948" /> <meta name="og-site-name" property="og:site_name" content="The BMJ" /> <meta name="og-description" property="og:description" content="Objective To evaluate the cognitive abilities of the leading large language models and identify their susceptibility to cognitive impairment, using the Montreal Cognitive Assessment (MoCA) and additional tests. Design Cross sectional analysis. Setting Online interaction with large language models via text based prompts. Participants Publicly available large language models, or “chatbots”: ChatGPT versions 4 and 4o (developed by OpenAI), Claude 3.5 “Sonnet” (developed by Anthropic), and Gemini versions 1 and 1.5 (developed by Alphabet). Assessments The MoCA test (version 8.1) was administered to the leading large language models with instructions identical to those given to human patients. Scoring followed official guidelines and was evaluated by a practising neurologist. Additional assessments included the Navon figure, cookie theft picture, Poppelreuter figure, and Stroop test. Main outcome measures MoCA scores, performance in visuospatial/executive tasks, and Stroop test results. Results ChatGPT 4o achieved the highest score on the MoCA test (26/30), followed by ChatGPT 4 and Claude (25/30), with Gemini 1.0 scoring lowest (16/30). All large language models showed poor performance in visuospatial/executive tasks. Gemini models failed at the delayed recall task. Only ChatGPT 4o succeeded in the incongruent stage of the Stroop test. Conclusions With the exception of ChatGPT 4o, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment. Moreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence. No additional data available." /> <meta name="og-type" property="og:type" content="article" /> <meta name="og:image" property="og:image" content="https://www.bmj.com/sites/default/files/highwire/bmj/388/8457.cover-source.jpg"/> <meta name="twitter:card" content="summary_large_image"><meta name="twitter:image" property="twitter:image" content="https://www.bmj.com/sites/default/files/sites/defautl/files/attachments/bmj-article/2024/12/ai-imp-thumb-homepage.png"><meta name="twitter:title" content="Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis" /><link rel="alternate" type="application/vnd.ms-powerpoint" title="Powerpoint" href="/content/387/bmj-2024-081948.ppt" /> <meta name="generator" content="Drupal 7 (http://drupal.org)" /> <link rel="canonical" href="https://www.bmj.com/content/387/bmj-2024-081948" /> <link rel="shortlink" href="https://www.bmj.com/node/1104732" /> <meta name="twitter:card" content="summary" /> <meta name="twitter:url" content="https://www.bmj.com/content/387/bmj-2024-081948" /> <meta name="twitter:title" content="Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis" /> <title>Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis | The BMJ</title>   <link type="text/css" rel="stylesheet" href="https://www.bmj.com/sites/default/files/advagg_css/css__QG_MjOKZbkspKkC-Oar15h0L0BAp_OJwLjRlZw9u0H4__C1pzKa_F3xCTq6UQAN53pO-1qtBXe96WGB9wgRYNwTY__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.css" media="all" /> <link type="text/css" rel="stylesheet" href="https://www.bmj.com/sites/default/files/advagg_css/css__hcEfBsSfdZ2na-rT9Jhk7K2mHyy5KBdPKm6ia_F7MYU__4716zpuGoj4DfVIEENbeOWgAOX_pf_t16EI4fvJGo7A__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.css" media="print" /> <link type="text/css" rel="stylesheet" href="https://www.bmj.com/sites/default/files/advagg_css/css__b-tWEqg3_twqP_ufdNXyA5V14StJgBdiIxG1UIi0LYc__3z5eyFP7TvyPE6DkwNj3xM94b3mZNfr4lSWEChDxsvU__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.css" media="screen" /> <link type="text/css" rel="stylesheet" href="https://www.bmj.com/sites/default/files/advagg_css/css__-BkZ71IF4pBXmZfP4vYyswWdIhuaKktrUUQ4i2vtUVo__0i_0YcYkMECNn0z7FdhNoJ8gLWleqnTtjnHRAmLijvc__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.css" media="all" /> <link href="/sites/default/themes/bmj/the_bmj/css/fonts.css" rel="stylesheet" media="all" type="text/css"> <style media="screen" type="text/css"> .page-news #latest-blogs h2.pane-title, .bootstrap-threecol-4-4-4 #latest-blogs h2.pane-title { width: 153px !important; } #navigation-wrapper #navigation .main-navigation .menu.nav>li.last { display: block !important; } </style>  <script src="https://cdn.cookielaw.org/consent/tcf.stub.js"" type="text/javascript" charset="UTF-8"></script>  <script src="https://cookie-cdn.cookiepro.com/scripttemplates/otSDKStub.js" type="text/javascript" charset="UTF-8" data-domain-script="565e50dd-f6e9-46d6-91cd-5c40a270264f"></script> <script type="text/javascript"> function OnetrustAdsConsent(){ if(googletag.apiReady){ //console.log("GoogleTag API ready") googletag.pubads().disableInitialLoad(); if(OptanonActiveGroups.match(/,4,/)){ googletag.pubads().setRequestNonPersonalizedAds(0); //googletag.pubads().refresh(); } else{ googletag.pubads().setRequestNonPersonalizedAds(1); // googletag.pubads().refresh(); } } else{ OnetrustAdsConsent(); } } function OptanonWrapper() { OnetrustAdsConsent(); } </script>  <script src="https://www.bmj.com/sites/default/files/advagg_js/js__j-irn_9VZoI-QKWZaTjsYf4Du15A3POzVHAKuEDxhE0__8l-R-HGeL_L4LIoXsYJaIiEPaHcUS4p2Bc5kaxQU-54__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://www.google.com/recaptcha/api.js?hl=en&render=explicit&onload=drupalRecaptchaOnload"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__0PW1inMiPnQI4drZpH9sZQXwRZrXiAq85270dYtHmuM__5inEPwzvI2QlS2DhP6cpwo3eu3qZqkX7qyeuARQjprY__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__bsh9JWzqeaLElAwcweNXFAeb9KuF1YAALIZQn6l_cEI__7ncqzfoNu6xcDvwicNM0Vq-0r8lbHAiVEZYAQdmYmsY__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__UUwMbpErCBHAQQqH_Cxzi20oSmMoxcbKeYV6qoD70qc__skFUSUohWkyaYdPjJ2LSEF0xKVUuQTqmOxG-9og_468__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__NdPGx9Oj8DBfskJmfE5W6d4quCaXzT7_tKZXBTHMuTE__cXvEw-nRRT0TjknkARJSfP4NWRZut8tP7oEkSYAUJdc__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script>var _prum=[['id', '612e3f1b1733500011000061'], ['mark', 'firstbyte', (new Date()).getTime()]]; (function() { var s=document.getElementsByTagName('script')[0], p=document.createElement('script'); p.async='async'; p.src='//rum-static.pingdom.net/prum.min.js';s.parentNode.insertBefore(p,s);})();</script> <script async="async" src="https://scholar.google.com/scholar_js/casa.js"></script> <script>jQuery.extend(Drupal.settings,{"basePath":"\/","pathPrefix":"","ajaxPageState":{"theme":"the_bmj","theme_token":"BiSWFjCGp7MYO-BAiK934LGZQxGi4oEZGhJY-ti90-o","jquery_version":"1.8","css":{"modules\/system\/system.base.css":1,"sites\/all\/libraries\/chosen\/chosen.css":1,"sites\/all\/modules\/contrib\/chosen\/css\/chosen-drupal.css":1,"misc\/ui\/jquery.ui.core.css":1,"misc\/ui\/jquery.ui.theme.css":1,"sites\/all\/modules\/contrib\/date\/date_api\/date.css":1,"sites\/all\/modules\/contrib\/date\/date_popup\/themes\/datepicker.1.7.css":1,"modules\/field\/theme\/field.css":1,"sites\/all\/modules\/highwire\/highwire\/highwire_draw\/css\/highwire-draw.css":1,"sites\/all\/modules\/highwire\/highwire\/highwire_folders\/highwire_folders.css":1,"sites\/all\/modules\/highwire\/highwire\/highwire_saved_searches\/highwire_saved_searches.css":1,"sites\/all\/modules\/highwire\/highwire\/highwire_user\/highwire_user.css":1,"sites\/all\/modules\/contrib\/views\/css\/views.css":1,"sites\/all\/modules\/contrib\/ckeditor\/css\/ckeditor.css":1,"sites\/all\/modules\/contrib\/colorbox\/styles\/default\/colorbox_style.css":1,"sites\/all\/modules\/contrib\/ctools\/css\/ctools.css":1,"sites\/all\/modules\/highwire\/highwire\/highwire.style.highwire.css":1,"sites\/all\/modules\/highwire\/highwire\/css\/nlm-elements.css":1,"sites\/all\/modules\/contrib\/panels\/css\/panels.css":1,"public:\/\/ctools\/css\/263bfdb18936424f0ade19d6bd885f37.css":1,"sites\/all\/modules\/contrib\/panels_ajax_tab\/css\/panels_ajax_tab.css":1,"sites\/all\/modules\/highwire\/highwire\/highwire.style.markup.css":1,"sites\/all\/modules\/contrib\/panels\/plugins\/layouts\/onecol\/onecol.css":1,"sites\/all\/modules\/highwire\/highwire\/css\/highwire.gtranslate.css":1,"sites\/default\/modules\/contrib\/custom_search\/custom_search.css":1,"sites\/all\/modules\/contrib\/forward\/forward.css":1,"sites\/default\/modules\/jnl-bmj\/jnl_bmj\/css\/jnl_bmj_item_list_show_less_more.css":1,"sites\/default\/themes\/bmj\/the_bmj\/css\/print.css":1,"sites\/default\/themes\/bmj\/the_bmj\/css\/screen.css":1,"sites\/default\/themes\/bmj\/the_bmj\/css\/screen_2.css":1,"sites\/default\/themes\/bmj\/the_bmj\/css\/screen_3.css":1,"sites\/all\/modules\/highwire\/highwire\/css\/highwire-forms.css":1},"js":{"sites\/all\/modules\/highwire\/highwire\/plugins\/content_types\/js\/highwire_panels_ajax_tab.js":1,"sites\/all\/modules\/highwire\/highwire\/plugins\/content_types\/js\/highwire_article_trendmd.js":1,"https:\/\/static.polldaddy.com\/w\/58250.js":1,"https:\/\/ajax.googleapis.com\/ajax\/libs\/jqueryui\/1.8.7\/jquery-ui.min.js":1,"https:\/\/crossmark-cdn.crossref.org\/widget\/v2.0\/widget.js":1,"https:\/\/d1bxh8uas1mnw7.cloudfront.net\/assets\/embed.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/hambugermenu.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/altmetric.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/cookieprofix.js":1,"sites\/all\/libraries\/modernizr\/modernizr.min.js":1,"sites\/all\/modules\/contrib\/jquery_update\/replace\/jquery\/1.8\/jquery.min.js":1,"misc\/jquery-extend-3.4.0.js":1,"misc\/jquery.once.js":1,"misc\/drupal.js":1,"sites\/all\/modules\/contrib\/jquery_update\/replace\/ui\/ui\/minified\/jquery.ui.core.min.js":1,"sites\/all\/modules\/contrib\/jquery_update\/replace\/ui\/external\/jquery.cookie.js":1,"sites\/all\/modules\/contrib\/jquery_update\/replace\/misc\/jquery.form.min.js":1,"misc\/ajax.js":1,"sites\/all\/modules\/contrib\/jquery_update\/js\/jquery_update.js":1,"sites\/all\/libraries\/chosen\/chosen.jquery.min.js":1,"sites\/all\/modules\/highwire\/highwire\/js\/highwire.equal-heights.js":1,"sites\/all\/modules\/highwire\/highwire\/js\/highwire.panels-ajax-tab.js":1,"sites\/all\/libraries\/colorbox\/jquery.colorbox-min.js":1,"sites\/all\/modules\/contrib\/colorbox\/js\/colorbox.js":1,"sites\/all\/modules\/contrib\/colorbox\/styles\/default\/colorbox_style.js":1,"sites\/all\/modules\/highwire\/highwire\/highwire_user\/js\/highwire_user_meta.js":1,"sites\/default\/modules\/contrib\/custom_search\/js\/custom_search.js":1,"sites\/all\/modules\/contrib\/panels_ajax_tab\/js\/panels_ajax_tab.js":1,"sites\/all\/modules\/contrib\/views\/js\/base.js":1,"sites\/all\/modules\/contrib\/views\/js\/ajax_view.js":1,"sites\/all\/modules\/contrib\/service_links\/js\/twitter_button.js":1,"sites\/all\/modules\/contrib\/service_links\/js\/facebook_like.js":1,"sites\/all\/modules\/highwire\/highwire\/highwire_citation\/plugins\/content_types\/js\/highwire_citation_export.js":1,"sites\/all\/modules\/highwire\/highwire\/js\/highwire.gtranslate.js":1,"sites\/all\/modules\/highwire\/highwire\/highwire_datalayer\/js\/highwire_datalayer.js":1,"sites\/all\/modules\/contrib\/captcha\/captcha.js":1,"sites\/all\/modules\/contrib\/recaptcha\/js\/recaptcha.js":1,"https:\/\/www.google.com\/recaptcha\/api.js?hl=en\u0026render=explicit\u0026onload=drupalRecaptchaOnload":1,"sites\/default\/modules\/jnl-bmj\/jnl_bmj\/js\/jnl_bmj_item_list_show_less_more.js":1,"sites\/all\/modules\/contrib\/chosen\/chosen.js":1,"sites\/default\/themes\/bmj\/the_bmj\/bootstrap\/js\/bootstrap.min.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/modernizr.min.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/jquery.cookieBar.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/script.js":1,"sites\/default\/themes\/bmj\/the_bmj\/js\/uuid.js":1,"sites\/default\/themes\/bmj\/bootstrap\/js\/modules\/views\/js\/ajax_view.js":1,"https:\/\/scholar.google.com\/scholar_js\/casa.js":1}},"colorbox":{"opacity":"0.85","current":"{current} of {total}","previous":"\u00ab Prev","next":"Next \u00bb","close":"Close","maxWidth":"98%","maxHeight":"98%","fixed":true,"mobiledetect":true,"mobiledevicewidth":"480px"},"highwire":{"nid":"1104732","apath":"\/bmj\/387\/bmj-2024-081948.atom","pisa":"bmj;387\/dec18_11\/e081948","ac":{"bmj;387\/dec18_11\/e081948":{"access":{"full":true},"pisa_id":"bmj;387\/dec18_11\/e081948","apath":"","jcode":"bmj"}},"trendmd":{"trendmd-suggestions":"{\u0022element\u0022:\u0022#trendmd-suggestions\u0022,\u0022track_id\u0022:\u002224814\u0022}"},"markup":[{"requested":"index","variant":"index","view":"full","pisa":"bmj;bmj-2024-081948\/DC1"},{"requested":"index","variant":"index","view":"full","pisa":"bmj;bmj-2024-081948\/DC1"}],"dataLayer":"{\u0022page\u0022:{\u0022hwTitle\u0022:\u0022Age against the machine\\u2014susceptibility of large language models to cognitive impairment: cross sectional analysis\u0022,\u0022hwUrl\u0022:\u0022https:\\\/\\\/www.bmj.com\\\/content\\\/387\\\/bmj-2024-081948\u0022,\u0022hwPath\u0022:\u0022\\\/content\\\/387\\\/bmj-2024-081948\\\/rapid-responses\u0022,\u0022hwType\u0022:\u0022journal page\u0022},\u0022user\u0022:{\u0022hwMemberID\u0022:\u0022\u0022,\u0022hwLoggedIntoInstitutionID\u0022:\u0022\u0022,\u0022hwLoggedIntoInstitution\u0022:\u0022\u0022,\u0022hwAccessUsageTerms\u0022:\u0022guest\u0022,\u0022hwRegisteredUser\u0022:\u0022no\u0022,\u0022hwHasFullTextAccess\u0022:\u0022yes\u0022,\u0022hwEntitled\u0022:\u0022no\u0022,\u0022mail\u0022:\u0022e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\u0022},\u0022content\u0022:{\u0022hwIssueDate\u0022:\u00222024-12-20\u0022,\u0022hwIssueDateTime\u0022:\u00222024-12-20 06:41:07\u0022,\u0022hwPublicationDate\u0022:\u00222024-12-20\u0022,\u0022hwPublicationDateTime\u0022:\u00222024-12-20 06:41:07\u0022,\u0022hwNlmArticleType\u0022:\u0022research-article\u0022,\u0022hwDOI\u0022:\u002210.1136\\\/bmj-2024-081948\u0022,\u0022hwAuthors\u0022:\u0022Roy Dayan,Benjamin Uliel,Gal Koplewitz\u0022,\u0022hwContributors\u0022:\u0022\u0022,\u0022hwIsEarlyRelease\u0022:\u0022no\u0022,\u0022hwEissn\u0022:\u00221756-1833\u0022,\u0022hwPissn\u0022:\u0022\u0022,\u0022hwVolume\u0022:\u0022387\u0022,\u0022hwIssue\u0022:\u0022\u0022,\u0022hwArticleType\u0022:\u0022Research\u0022,\u0022hwTabView\u0022:\u0022rapid-responses\u0022,\u0022hwViewType\u0022:\u0022full\u0022,\u0022hwAccessType\u0022:\u0022OpenAccess\u0022,\u0022hwOpenAccess\u0022:\u0022yes\u0022,\u0022hwFreeAccess\u0022:\u0022yes\u0022,\u0022hwTaxonomy\u0022:\u0022Christmas\u0022,\u0022hwCorpusCode\u0022:\u0022bmj\u0022,\u0022hwType\u0022:\u0022article\u0022,\u0022hwSubtitle\u0022:null,\u0022hwOverline\u0022:null,\u0022hwPisaId\u0022:\u0022bmj;387\\\/dec18_11\\\/e081948\u0022,\u0022hwFirstPage\u0022:null,\u0022hwLastPage\u0022:null,\u0022hwIsLatestVersion\u0022:\u0022yes\u0022,\u0022hwIsCurrentIssue\u0022:\u0022no\u0022,\u0022hwIsOpenIssue\u0022:\u0022no\u0022,\u0022hwHasFullText\u0022:\u0022yes\u0022,\u0022hwHasPDF\u0022:\u0022yes\u0022,\u0022hwParentTitle\u0022:\u0022BMJ: 387\u0022,\u0022hwElectronicPubDate\u0022:\u00222024-12-20\u0022,\u0022hwElectronicPubDateTime\u0022:\u00222024-12-20 06:41:07\u0022}}"},"jnl_bmj_infographic":{"infographic_tab":"content\/387\/bmj-2024-081948\/infographic"},"custom_search":{"form_target":"_self","solr":1},"panel_ajax_tab":{"path":"sites\/all\/modules\/contrib\/panels_ajax_tab"},"chosen":{"selector":"select:visible","minimum_single":20,"minimum_multiple":20,"minimum_width":250,"options":{"disable_search":false,"disable_search_threshold":0,"search_contains":false,"placeholder_text_multiple":"Choose some options","placeholder_text_single":"Choose an option","no_results_text":"No results match","inherit_select_classes":true}},"urlIsAjaxTrusted":{"\/":true,"\/views\/ajax":true,"\/content\/387\/bmj-2024-081948?tab=article-alert":true,"\/content\/387\/bmj-2024-081948":true,"\/login":true,"\/content\/387\/bmj-2024-081948\/rapid-responses":true},"views":{"ajax_path":"\/views\/ajax","ajaxViews":{"views_dom_id:80b509e5b23f930f5ddac88032f5274e":{"view_name":"bmj_rapid_responses","view_display_id":"bmj_rr_article","view_args":"1104732","view_path":"node\/1104732","view_base_path":null,"view_dom_id":"80b509e5b23f930f5ddac88032f5274e","pager_element":0}}},"ws_fl":{"width":100,"height":21},"jnl_bmj":{"container_id_share":"article-share-modal","container_id_response":"article-response-modal"},"orcidLogo":{"svg":"\/sites\/default\/themes\/bmj\/the_bmj\/img\/vector_iD_icon.svg"},"bootstrap":{"anchorsFix":"1","anchorsSmoothScrolling":"1","popoverEnabled":"1","popoverOptions":{"animation":1,"html":0,"placement":"right","selector":"","trigger":"click","title":"","content":"","delay":0,"container":"body"},"tooltipEnabled":"1","tooltipOptions":{"animation":1,"html":0,"placement":"auto left","selector":"","trigger":"hover focus","delay":0,"container":"body"}}});</script> <script type="text/javascript"> window.dataLayer = window.dataLayer || []; </script>  <script type="text/javascript">var DC = { Date : "", subject : "" };var ga_article_category = "Research";var ga_article_page_type = "research-article";var ga_article_pub_year = "2024";var ga_customer_number = "";var ga_user_type = "";DC.Date = "2024-12-20";DC.subject = "";</script>   <style>.async-hide { opacity: 0 !important} </style> <script>(function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;h.start=1*new Date; h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')}; (a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;})(window,document.documentElement,'async-hide','dataLayer',4000,{'GTM-NF7HCLL':true});</script>   <script>var googletag = window.googletag || {cmd: []};</script> <script async src="//securepubads.g.doubleclick.net/tag/js/gpt.js"></script> <script> var topBottomSizes = [ [970,250],[970,90],[728,90],[468,60],[300,50],[320,100] ]; var mpuSizes = [ [300,250],[300,600],[600,600], ]; //googletag.cmd.push(function() { googletag.display('div-gpt-ad-7326006-1'); }); googletag.cmd.push(function() { googletag.defineSlot('/21798771943/BMJ.com_HomePage_TOPBanner', topBottomSizes,'div-gpt-ad-7326006-1').addService(googletag.pubads()).setTargeting("pgid", "pubgrade_lib_disabled"); googletag.defineSlot('/21798771943/BMJ.com_HomePage_MPU1', mpuSizes, 'div-gpt-ad-7326006-2').addService(googletag.pubads()); googletag.defineSlot('/21798771943/BMJ.com_HomePage_MPU2', mpuSizes, 'div-gpt-ad-7326006-3').addService(googletag.pubads()); googletag.defineSlot('/21798771943/BMJ.com_HomePage_BOTTOM', topBottomSizes, 'div-gpt-ad-7326006-4').addService(googletag.pubads()); googletag.pubads().setTargeting('url', window.parent.location.pathname); // Some examples of ways to enable below. Normally, only one of these // methods should be used. // A) Enable with defaults. googletag.pubads().enableLazyLoad({ fetchMarginPercent: 0, // Fetch slots within 1 viewports. renderMarginPercent: 0, // Render slots within 1 viewports. mobileScaling: 2.0 // Double the above values on mobile. }); googletag.enableServices(); }); </script> <script> let cookies = parseCookies(); let bmj_uuid = cookies.bmj_uuid; let bmj_ppid = cookies.bmj_ppid; window.dataLayer = window.dataLayer || []; function checkUserLoginStatus(dataLayer) { const userMail = dataLayer.find((item) => item.user)?.user?.mail; if (!userMail) { return false; } return userMail !== anonMailId; } const anonMailId = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'; let nonAnonMailId = ((dataLayer.find(item => item.user)?.user)?.mail) === anonMailId ? null : (dataLayer.find(item => item.user)?.user)?.mail; let userId = ((dataLayer.find(item => item.user)?.user)?.['data-ics']) ?? ((dataLayer.find(item => item.user)?.user)?.mail && nonAnonMailId) ?? bmj_ppid ?? bmj_uuid; document.cookie = 'bmj_ppid=' + userId + '; domain=.bmj.com; path=/; max-age=31622400; SameSite=None; Secure'; googletag.cmd.push(function() { googletag.pubads().setPublisherProvidedId(userId); googletag.pubads().setTargeting("islogged", checkUserLoginStatus(dataLayer) ? "true" : "false"); googletag.enableServices(); }); window.dataLayer.push({ 'bmj_uuid': bmj_uuid }); </script> </head>   <script type="text/javascript">(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0];var j=d.createElement(s);var dl=l!='dataLayer'?'&l='+l:'';j.src='//www.googletagmanager.com/gtm.js?id='+i+dl;j.type='text/javascript';j.async=true;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-NF7HCLL');</script>  </head>  <body class="html not-front not-logged-in no-sidebars page-node page-node- page-node-1104732 node-type-highwire-article 1 Research" >  <noscript><iframe src="//www.googletagmanager.com/ns.html?id=GTM-NF7HCLL" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  <div id="skip-link"> <a href="#main-content" class="element-invisible element-focusable">Skip to main content</a> </div> <div id="wrapper-outer"> <div id="wrapper"> <header> <div class="bmj-disclaimer"> <p>Intended for healthcare professionals</p> </div> <div class="ad-leaderboard"> <div id='div-gpt-ad-7326006-1'> <script> googletag.cmd.push(function() { googletag.display('div-gpt-ad-7326006-1'); }); </script> </div> </div>   <div id="top-nav" role="navigation" class="navbar navbar-static-top"> <div class="container"> <div id="userbar-status"> <ul id="status"> <li class="user-status-link user-status-subscribe phone-menu-item"> <a href="/us/subscribe">Subscribe</a> </li> <li class="user-status-myaccount phone-menu-item"> <div class="btn-group has-submenu"> <a class="btn dropdown-toggle" aria-expanded="false" aria-haspopup="true" data-toggle="dropdown" href="#" > My Account <span class="caret" aria-expanded="true"></span> </a> <ul class="jnl-bmj-user-my-account dropdown-menu"><li class="0 first last"><a href="//myaccount.bmj.com/myaccount/account/hw-registration.html">My email alerts</a></li> </ul></div> </li> <li class="user-status-bma phone-menu-item"> <a href="//bmjverification.bma.org.uk/validate/userdetails?url=https://www.bmj.com/content/387/bmj-2024-081948?panels_ajax_tab_tab=bmj_related_rapid_responses&panels_ajax_tab_trigger=rapid-responses&sso=">BMA member login</a> </li> <div class="btn-group has-submenu"> <a class="btn dropdown-toggle" aria-expanded="false" aria-haspopup="true" data-toggle="dropdown" href="#" > Login <span class="caret" aria-expanded="true"></span> </a> <ul class="jnl-bmj-user-signin-popup dropdown-menu"><li class="0 first last"><span><form action="/login" method="post" id="jnl-bmj-user-login-form" accept-charset="UTF-8"><div><input type="hidden" name="form_build_id" value="form-2Mz7WsekgX-U3U_HQh-SZ_MS6vahzTYbR1u4ISEVzoM" /> <input type="hidden" name="form_id" value="jnl_bmj_user_login_form" /> <div class="form-type-textfield form-item-name form-item form-group"> <label for="article-p-name">Username <span class="form-required" title="This field is required.">*</span></label> <input class="form-control form-text required" type="text" id="article-p-name" name="name" value="" size="15" maxlength="60" /> </div> <div class="form-type-password form-item-pass form-item form-group"> <label for="article-p-pass">Password <span class="form-required" title="This field is required.">*</span></label> <input class="form-control form-text required" type="password" id="article-p-pass" name="pass" size="15" maxlength="128" /> </div> <div class="link"><div id="edit-forgot" class="form-type-item form-item form-group"> <label for="edit-forgot"><a href="https://www.bmj.com/user/password" title="Retrieve your credentials.">Forgot your log in details?</a> </label> </div> </div><div class="link"><div id="edit-activate" class="form-type-item form-item form-group"> <label for="edit-activate"><a href="//journals.bmj.com/cgi/activate/basic">Need to activate</a> </label> </div> </div><input class="btn form-submit" type="submit" id="edit-submit--8" name="op" value="Login" /><div id="edit-message" class="form-type-item form-item form-group"> </div> <hr /><div id="edit-bma-members" class="form-type-item form-item form-group"> <label for="edit-bma-members"><a href="//bmjverification.bma.org.uk/validate/userdetails?url=https%3A//www.bmj.com/content/387/bmj-2024-081948%3Fpanels_ajax_tab_tab%3Dbmj_related_rapid_responses%26panels_ajax_tab_trigger%3Drapid-responses%26sso%3D" title="Log in here if you are a BMA member.">BMA Member Log In</a> </label> </div> <div id="edit-athens" class="form-type-item form-item form-group"> <label for="edit-athens"><a href="https://openathens-sp.highwire.org/session/init?entityID=https%3A//idp.eduserv.org.uk/openathens&subcode=bmjjournals&hw-shib-return-uri=https%253A%252F%252Fwww.bmj.com%252Fcontent%252F387%252Fbmj-2024-081948%253Fpanels_ajax_tab_tab%253Dbmj_related_rapid_responses%2526panels_ajax_tab_trigger%253Drapid-responses">Log in via OpenAthens</a> </label> </div> <div id="edit-shibboleth" class="form-type-item form-item form-group"> <label for="edit-shibboleth"><a href="/login/shibboleth?uri=https%3A%2F%2Fwww.bmj.com%2Fcontent%2F387%2Fbmj-2024-081948%3Fpanels_ajax_tab_tab%3Dbmj_related_rapid_responses%26panels_ajax_tab_trigger%3Drapid-responses&subcode=bmjjournals&env=prod">Log in via your institution</a> </label> </div> </div></form></span></li> </ul></div> </ul> <div id="userbar"> <div class="region region-userbar"> <section id="block-panels-mini-jnl-bmj-country-portal-switcher" class="block block-panels-mini clearfix"> <div class="panel-display panel-1col clearfix" id="mini-panel-jnl_bmj_country_portal_switcher"> <div class="panel-panel panel-col"> <div><div class="panel-pane pane-country-switcher" > <div class="pane-content"> <span class="country-label">Edition: </span> <div class="btn-group has-submenu"> <a class="btn dropdown-toggle" aria-expanded="false" aria-haspopup="true" data-toggle="dropdown" href="#" id=country-switcher> US <span class="caret" aria-expanded="true"></span> </a> <ul class="jnl-bmj-country-portal-country-switcher dropdown-menu"><li class="0 first"><a href="/bmj_countries/switch/uk?destination=node/1104732%3Fpanels_ajax_tab_tab%3Dbmj_related_rapid_responses%26panels_ajax_tab_trigger%3Drapid-responses">UK</a></li> <li class="1"><a href="/bmj_countries/switch/india?destination=node/1104732%3Fpanels_ajax_tab_tab%3Dbmj_related_rapid_responses%26panels_ajax_tab_trigger%3Drapid-responses">South Asia</a></li> <li class="2 last"><a href="/bmj_countries/switch/international?destination=node/1104732%3Fpanels_ajax_tab_tab%3Dbmj_related_rapid_responses%26panels_ajax_tab_trigger%3Drapid-responses">International</a></li> </ul></div> </div> </div> </div> </div> </div> </section>  </div> </div> <div id="cmp-link"> <div class="pane-content"> <span class="country-label"><a href="/company?int_source=tbmj&int_medium=header&int_campaign=URLswitch">Our company</a></span> </div> </div> </div> </div> </div>  <div id="main-nav" data-spy="affix" data-offset-top="125"> <div class="primary-navigation-bar"> <div id="navigation-wrapper"> <div class="container">  <a href="#" id="btn-nav" class=""> <span class="sr-only">Toggle navigation</span> </a>  <a class="logo navbar-btn pull-left" href="/" title="Home">  <div class="bmj-logo">The BMJ logo</div> </a> <div class="ie6-site-map"><a href="/sitemap">Site map</a></div>  <div class="main-nav-search"> <div id="search">  <a href="#" class="dropdown-toggle" data-toggle="dropdown" title="Search"><div class="search"><em class="fa fa-search" alt="Search"></em><span class="search-label">Search</span></div></a> <div class="dropdown-menu"> <div class="header-search-form container"> <div class="searchbar"> <div class="region region-search-block"> <section id="block-search-form" class="block block-search clearfix"> <form class="search-form form-search content-search" action="/content/387/bmj-2024-081948/rapid-responses" method="post" id="search-block-form" accept-charset="UTF-8"><div><div> <h2 class="element-invisible">Search form</h2> <div class="input-group"><input title="Enter the terms you wish to search for." class="container-inline input-append form-control form-text" placeholder="Search The BMJ" type="text" id="edit-search-block-form--2" name="search_block_form" value="" size="15" maxlength="128" /><span class="input-group-btn"><button type="submit" class="btn btn-default">Search</button></span></div><button class="element-invisible btn btn-primary btn btn-primary form-button" type="submit" id="edit-submit--3" name="op" value="Search">Search</button><input type="hidden" name="form_build_id" value="form-VKWPEkr3EupED2f_4e3o7rr5BSJFpELL_1-Hibz1dJ0" /> <input type="hidden" name="form_id" value="search_block_form" /> <input type="hidden" name="custom_search_paths" value="search/advanced/[key]" /> </div> </div></form> </section>  </div> </div> <div class="advanced-search"> <ul class="list-inline"> <li><a href="/search/advanced" title="Advanced search">Advanced search</a></li> <li><a href="/rapid-responses" title="Search responses">Search responses</a></li> <li><a href="//blogs.bmj.com/bmj/" title="Search blogs">Search blogs</a></li> </ul> </div> </div> </div> </div> </div>  <a href="#" id="toggle-mobile-top-nav"> <span class="fa fa-cog"></span> <span class="sr-only">Toggle top menu</span> </a>  <div id="navigation" class="navigation">  <div class="main-navigation"> <nav role="navigation"> <ul class="menu nav navbar-nav"><li class="first leaf"><a href="/coronavirus">covid-19</a></li> <li class="expanded dropdown"><a href="/us/research" data-target="#" class="dropdown-toggle" data-toggle="dropdown">Research</a><ul class="dropdown-menu"><li class="first leaf"><a href="/us/research">At a glance</a></li> <li class="leaf"><a href="/us/research/research">Research papers</a></li> <li class="leaf"><a href="/us/research/research%20methods%20%26amp%3B%20reporting">Research methods and reporting</a></li> <li class="last leaf"><a href="/us/research/minerva">Minerva</a></li> </ul></li> <li class="expanded dropdown"><a href="/us/education" data-target="#" class="dropdown-toggle" data-toggle="dropdown">Education</a><ul class="dropdown-menu"><li class="first leaf"><a href="/us/education">At a glance</a></li> <li class="leaf"><a href="/us/education/clinical%20review">Clinical reviews</a></li> <li class="leaf"><a href="/us/education/practice">Practice</a></li> <li class="leaf"><a href="/us/education/minerva">Minerva</a></li> <li class="leaf"><a href="/us/education/endgames">Endgames</a></li> <li class="leaf"><a href="/specialties/state-art">State of the art</a></li> <li class="leaf"><a href="/specialties/what-your-patient-thinking">What your patient is thinking</a></li> <li class="leaf"><a href="/rapid-recommendations">Rapid recommendations</a></li> <li class="last leaf"><a href="/student">Student</a></li> </ul></li> <li class="expanded dropdown"><a href="/us/news" data-target="#" class="dropdown-toggle" data-toggle="dropdown">News & Views</a><ul class="dropdown-menu"><li class="first leaf"><a href="/us/news">At a glance</a></li> <li class="leaf"><a href="/us/news/news?category=News">News</a></li> <li class="leaf"><a href="/us/news/feature">Features</a></li> <li class="leaf"><a href="/us/news/editorial">Editorials</a></li> <li class="leaf"><a href="/us/news/analysis">Analysis</a></li> <li class="leaf"><a href="/news/opinion">Opinion</a></li> <li class="leaf"><a href="/us/news/head%20to%20head">Head to head</a></li> <li class="leaf"><a href="/us/news/editor%27s%20choice">Editor's choice</a></li> <li class="leaf"><a href="/us/news/letter">Letters</a></li> <li class="leaf"><a href="/us/news/obituary">Obituaries</a></li> <li class="leaf"><a href="/careers-articles">Careers</a></li> <li class="last leaf"><a href="/rapid-responses">Rapid responses</a></li> </ul></li> <li class="expanded dropdown"><a href="/campaigns" data-target="#" class="dropdown-toggle" data-toggle="dropdown">Campaigns</a><ul class="dropdown-menu"><li class="first leaf"><a href="/campaigns">At a glance</a></li> <li class="leaf"><a href="/better-evidence">Better evidence</a></li> <li class="leaf"><a href="/campaign/climate-emergency">Climate change</a></li> <li class="leaf"><a href="/divestment">Divestment from fossil fuels</a></li> <li class="leaf"><a href="/campaign/patient-partnership">Patient and public partnership</a></li> <li class="last leaf"><a href="/too-much-medicine">Too much medicine</a></li> </ul></li> <li class="last expanded dropdown"><a href="/careers" data-target="#" class="dropdown-toggle" data-toggle="dropdown">Jobs</a><ul class="dropdown-menu"><li class="first leaf"><a href="/careers">Doctor Jobs UK</a></li> <li class="leaf"><a href="/careers/jobs/hospital/">Hospital Jobs UK</a></li> <li class="leaf"><a href="/careers/jobs/general-practice/">GP Jobs UK</a></li> <li class="last leaf"><a href="/careers/jobs/international/">International Jobs</a></li> </ul></li> </ul>  <div class="second-navigation"> <ul class="secondary-nav" id=""> <li class="even"><a href="/archive" class="">Archive</a></li> <li class="odd"><a href="/about-bmj/resources-authors" class="">For authors</a></li> <li class="odd last"><a href="http://hosted.bmj.com" class="">Hosted</a></li> </ul> </div> </nav> </div> </div> </div> </div> </div> </div> </header> <div class="main-container container" id="page">   <div class="row"> <section class="col-sm-12" > <ol class="breadcrumb"><li class="first"><a href="/us/research">Research</a></li> <li>Age against the...</li> <li class="active last">Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</li> </ol> <a id="main-content"></a> <div class="region region-content"> <section id="block-system-main" class="block block-system clearfix"> <div class="panel-display" > <div class="hero"> <div class="panel-pane pane-bmj-article-open-access" > <div class="pane-content"> <div class="creative-commons" title="Creative Commons"><span class="creative-commons-article"><span class="icon-cc">CC</span><span class="icon-by">BY</span><span class="icon-nc">NC</span> <span>Open access</span></span></div> </div> </div> <div class="hero-wrapper"> <div class="hero-content" style="position: relative; left: 0px; top: 35.5px;"> </div>  </div>  </div>  <div class="row"> <div class="left-content col-xs-12 col-sm-8 col-md-8 col-lg-8"> <article> <div class="panel-pane pane-highwire-article-citation" > <div class="pane-content"> <div class="highwire-article-citation highwire-citation-type-highwire-article node1104732" data-node-nid="1104732" id="node-1104732--21514511038" data-pisa="bmj;387/dec18_11/e081948" data-pisa-master="bmj;bmj-2024-081948" data-apath="/bmj/387/bmj-2024-081948.atom"><cite class='highwire-cite highwire-citation-bmj-article-top'> <span class='highwire-cite-article-type'>Research</span> <span class='bmj-series-title'>Christmas 2024: Could We Start Again, Please?</span> <h1 class="highwire-cite-title" id="page-title">Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h1> <span class='highwire-cite-journal'>BMJ</span> <span class='highwire-cite-published-year'>2024</span>; <span class='highwire-cite-volume-issue'>387</span> <span class='highwire-cite-doi'> doi: <a href="https://doi.org/10.1136/bmj-2024-081948">https://doi.org/10.1136/bmj-2024-081948</a></span> <span class='highwire-cite-date'>(Published 20 December 2024)</span> <span class='highwire-cite-article-as'> Cite this as: <span class="italic">BMJ</span> 2024;387:e081948 </span> <span class='sound-cloud'><div class="flourish-embed flourish-heatmap" data-src="visualisation/20314693?313"><script src="https://public.flourish.studio/resources/embed.js"></script><noscript><img src="https://public.flourish.studio/visualisation/20314693/thumbnail" width="100%" alt="heatmap visualization" /></noscript></div> <a href="https://www.bmj.com/content/388/bmj.r27" style="color:#FFFFFF;" target="new"> <div style="background-color: #1bb1a6; width 200px; padding: 10px; border-radius: 5px; margin: 15px 0px"> <h2 style="color: #FFFFFF;">Linked Opinion</h2> <p style="margin-bottom: 0px;">AI in medicine: preparing for the future while preserving what matters</p> </div> </a></span> </cite> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-highwire-panel-tabs pane-panels-ajax-tab-tabs" > <div class="pane-content"> <ul class="tabs inline panels-ajax-tab"><li class="first"><a title="Article Tab - Article" href="/content/387/bmj-2024-081948" class="panels-ajax-tab-tab" data-panel-name="jnl_bmj_tab_art" data-target-id="highwire_article_tabs" data-entity-context="node:1104732" data-trigger="" data-url-enabled="1"><span>Article</span></a><a title="Article Tab - Article" href="/panels_ajax_tab/jnl_bmj_tab_art/node:1104732/1" rel="nofollow" style="display:none" class="js-crawler-link"></a></li> <li><a title="Article Tab - Related Articles" href="/content/387/bmj-2024-081948/related" class="panels-ajax-tab-tab" data-panel-name="jnl_bmj_tab_related_art" data-target-id="highwire_article_tabs" data-entity-context="node:1104732" data-trigger="related" data-url-enabled="1"><span>Related content</span></a><a title="Article Tab - Related Articles" href="/panels_ajax_tab/jnl_bmj_tab_related_art/node:1104732/1" rel="nofollow" style="display:none" class="js-crawler-link"></a></li> <li><a title="Article Tab - Info & Metrics" href="/content/387/bmj-2024-081948/article-info" class="panels-ajax-tab-tab" data-panel-name="jnl_bmj_tab_info" data-target-id="highwire_article_tabs" data-entity-context="node:1104732" data-trigger="article-info" data-url-enabled="1"><span>Metrics</span></a><a title="Article Tab - Info & Metrics" href="/panels_ajax_tab/jnl_bmj_tab_info/node:1104732/1" rel="nofollow" style="display:none" class="js-crawler-link"></a></li> <li><a title="BMJ Article Related Rapid Responses" href="/content/387/bmj-2024-081948/rapid-responses" class="panels-ajax-tab-tab" data-panel-name="bmj_related_rapid_responses" data-target-id="highwire_article_tabs" data-entity-context="node:1104732" data-trigger="rapid-responses" data-url-enabled="1"><span>Responses</span></a><a title="BMJ Article Related Rapid Responses" href="/panels_ajax_tab/bmj_related_rapid_responses/node:1104732/1" rel="nofollow" style="display:none" class="js-crawler-link"></a></li> <li><a title="Article Tab - Peer review" href="/content/387/bmj-2024-081948/peer-review" class="panels-ajax-tab-tab" data-panel-name="jnl_bmj_tab_peer_review" data-target-id="highwire_article_tabs" data-entity-context="node:1104732" data-trigger="peer-review" data-url-enabled="1"><span>Peer review</span></a><a title="Article Tab - Peer review" href="/panels_ajax_tab/jnl_bmj_tab_peer_review/node:1104732/1" rel="nofollow" style="display:none" class="js-crawler-link"></a></li> <li class="last"><a title="BMJ Rapid Responses Form" href="/content/387/bmj-2024-081948/submit-a-rapid-response" class="panels-ajax-tab-tab" data-panel-name="bmj_rapid_responses_form" data-target-id="highwire_article_tabs" data-entity-context="node:1104732" data-trigger="submit-a-rapid-response" data-url-enabled="1"><i class="fa fa-edit"></i></a><a title="BMJ Rapid Responses Form" href="/panels_ajax_tab/bmj_rapid_responses_form/node:1104732/1" rel="nofollow" style="display:none" class="js-crawler-link"></a></li> </ul> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-highwire-panel-tabs-container" > <div class="pane-content"> <div data-panels-ajax-tab-preloaded="bmj_related_rapid_responses" id="panels-ajax-tab-container-highwire_article_tabs" class="panels-ajax-tab-container"><div class="panels-ajax-tab-loading" style ="display:none"><img class="loading" src="https://www.bmj.com/sites/all/modules/contrib/panels_ajax_tab/images/loading.gif" alt="Loading" title="Loading" /></div><div class="panels-ajax-tab-wrap-bmj_related_rapid_responses"><div class="panel-display panel-1col clearfix" > <div class="panel-panel panel-col"> <div><div class="panel-pane pane-views-panes pane-bmj-rapid-responses-bmj-rr-article" > <h2 class="pane-title"> All rapid responses </h2> <div class="pane-content"> <div class="view view-bmj-rapid-responses view-id-bmj_rapid_responses view-display-id-bmj_rr_article view-dom-id-80b509e5b23f930f5ddac88032f5274e"> <div class="view-header"> <div id="response-header-description"> <p> Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. <i>The BMJ</i> reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation. </p> <p> From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit. </p> <p> The word limit for letters selected from posted responses remains 300 words. </p> </div> </div> <div class="view-filters"> <form action="/" method="get" id="views-exposed-form-bmj-rapid-responses-bmj-rr-article" accept-charset="UTF-8"><div><div class="views-exposed-form"> <div class="views-exposed-widgets clearfix"> <div class="views-exposed-widget views-widget-sort-by"> <div class="form-type-select form-item-sort-by form-item form-group"> <label for="edit-sort-by">Sort by </label> <select class="form-control form-select" id="edit-sort-by" name="sort_by"><option value="field_highwire_a_epubdate_value" selected="selected">Date Published</option></select> </div> </div> <div class="views-exposed-widget views-widget-sort-order"> <div class="form-type-select form-item-sort-order form-item form-group"> <label for="edit-sort-order">Order </label> <select class="form-control form-select" id="edit-sort-order" name="sort_order"><option value="ASC">Ascending</option><option value="DESC" selected="selected">Descending</option></select> </div> </div> <div class="views-exposed-widget views-widget-per-page"> <div class="form-type-select form-item-items-per-page form-item form-group"> <label for="edit-items-per-page">Items per page </label> <select class="form-control form-select" id="edit-items-per-page" name="items_per_page"><option value="5">5</option><option value="10" selected="selected">10</option><option value="20">20</option><option value="40">40</option><option value="60">60</option></select> </div> </div> <div class="views-exposed-widget views-submit-button"> <input class="btn btn-info form-submit" type="submit" id="edit-submit-bmj-rapid-responses" name="" value="Apply" /> </div> </div> </div> </div></form> </div> <div class="view-content"> <div class="views-row views-row-1 views-row-odd views-row-first"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-10"><h3>Re: Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor,</p> <p>The article “Age Against the Machine” by Dayan et al. has sparked a lively discussion, which we are excited to contribute to. Written with spice and flair, the paper - as the authors noted - was intended to be interpreted “with a wry smile, rather than a reproving frown”. However, we were content (and, to be honest, somewhat relieved) to see previous letters critically addressing - or frowning upon - its claims. We join the discussion by highlighting additional misconceptions that readers without a data science background, or familiarity with the Montreal Cognitive Assessment (MoCA), may overlook. </p> <p>Previous critiques have claimed that Large Language Models (LLMs) are not suitable candidates for the MoCA (especially in comparison to Visual Language Models). We wish to highlight further points that challenge the appropriateness of the MoCA as a tool for assessing LLMs’ cognitive capabilities. For example, in the orientation task, participants are asked to identify current date and location. For humans, this task is straightforward and essential, as their daily activities depend on such knowledge. However, such real-time details are not necessarily coded in LLMs, who might generate a random answer based on training data or rely on implicit cues from the prompt’s context - information that holds no intrinsic significance for them. For instance, an LLM might assume it is in the United States simply because the prompt is in English. Moreover, when scoring this task, humans’ answers can be assessed as either “right” or “wrong” in an objective manner. However, what would constitute a “right” answer for an LLM response? Its infrastructure’s location? The VPN it is routed through? Or, for distributed systems, multiple locations simultaneously? Without a defined reference, this task is rendered irrelevant. Another task rendered unanswered, this time due to the inherent superiority of computer programs such as LLMs, is the task requiring backward counting in increments of seven. Even the earliest LLMs - or any computer program - would outperform humans on tasks requiring straightforward calculations. </p> <p>These two examples illustrate a broader issue: evaluating LLMs with tasks designed for human assessment is inherently flawed. Nuances of specific tasks are likely to be either diminished or exaggerated when applied to models, leading to assessments that fail to produce meaningful comparisons.</p> <p>Another major concern relates to data contamination and bias in machine learning experiments, particularly evident in the clock-drawing task. A simple Google search reveals an abundance of incorrect responses and typical errors by cognitively impaired patients, far outnumbering examples of correct Clock Drawing Test performances. This imbalance potentially affects LLMs during training, leading to higher probability of their responses to mimic erroneous examples. Readers with machine learning expertise may recognize this as a reflection of training data imbalance, not an inherent cognitive process. However, readers from medical fields or without an ML background might miss this nuance. Control studies to address such biases were notably absent from the paper.</p> <p>Finally, we encourage all readers to approach AI-related papers with a critical eye. Interdisciplinary collaboration is key to advancing AI in medicine. If you encounter aspects of a study that raise questions, seeking additional perspectives from colleagues with complementary expertise may be of great value. By promoting cross-disciplinary discussions, we can work towards more robust and trustworthy applications of AI in healthcare.</p> <p>In conclusion, this paper provides an interesting idea to be further pursued in more rigorous ways with more control studies and, perhaps, more cautious phrasing. The discussion prompted by this paper is invaluable to human <> LLM research, and for the collaboration between medical professionals and ML experts. </p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>20 January 2025</strong> </div> <div class="response-author"> Maya Zadok </div> <div class="response-occupation"> Graduate Student </div> <div class="response-other_authors"> Lotem Peled-Cohen, Roi Reichart, Michal Schnaider Beeri </div> <div class="response-affiliation"> Technion - Israel Institute of Technology </div> <div class="response-address"> Haifa, Israel </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-2 views-row-even"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-7"><h3>Re: Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor,</p> <p>In the Christmas 2024 article Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis, Dayan et al. test large language models (LLMs) against the Montreal Cognitive Assessment (MoCA) and conclude that all models have some level of “cognitive impairment.” Since the authors mention in the discussion that the anthropomorphism of models is not intended to imply that these models are similar to the human brain, we will use the term model performance rather than “cognitive impairment.”</p> <p>A major limitation of this study is an inability to truly assess changes in model performance over time. The authors perform a complete MoCA test on each LLM exactly once. Models have been shown to have variability in response even when asked the same question multiple times[1]; thus a singular response may not be representative of actual performance. Models also change over time, even within the same version – for example, Claude 3.5 Sonnet referred to one specific statistical model on 20 June 2024 [2], and to an updated version of that model on 22 October 2024 [3]. An improved methodology, which might potentially test the article’s hypothesis of changing model performance over time, would be to test the same LLM repeatedly over time, to see whether LLM performance changes. To continue the example of Claude 3.5 Sonnet, testing the model on 20 June and 22 October would reveal whether the update to the model changed its MoCA score.</p> <p>The performance of models over time is an incredibly important topic, particularly for healthcare applications. Two causes for potential performance decline are considered top security risks for artificial intelligence in 2025 [3]. Model Poisoning is an intentional attack to change or degrade the behavior of an LLM; while Vector and Embedding Weaknesses can be exploited unintentionally, but they, too, cause LLMs to change how questions are answered. Security researchers have successfully poisoned versions of Claude in 2023 [4]. Claude offers Enterprise plans for its models [5], which provide security features beyond those available on the models tested by the authors.</p> <p>Since the possibility of models degrading is a known security problem, AI companies and other parties work to mitigate vulnerabilities. In addition, newer versions of models are usually engineered to have improved performance; thus, differences in performance between newer and older versions of a model are from intentional changes such as updates in training data or additional training procedures. For example, Claude 3.5 Sonnet has improved since it was first released [6]. </p> <p>Older LLMs are less capable than newer LLMs for the same reason that your old phone is not as capable as your new phone: technology improved. Your old phone didn’t experience cognitive decline. It stayed the same, while software and hardware advanced. It may be useful to perform MoCA tests on LLMs, but such testing would have to follow a methodology designed with an understanding of the LLM development life cycle.</p> <p>Sincerely,</p> <p>Aaron Sterling, CEO, EMR Data Cloud<br /> Roxana Daneshjou, MD, PhD, Departments of Biomedical Data Science and Dermatology, Stanford School of Medicine</p> <p>References</p> <p>[1] Alber, D.A., Yang, Z., Alyakin, A. et al. Medical large language models are vulnerable to data-poisoning attacks. Nat Med (2025). <a href="https://doi.org/10.1038/s41591-024-03445-1">https://doi.org/10.1038/s41591-024-03445-1</a><br /> [2] <a href="https://www.anthropic.com/news/claude-3-5-sonnet">https://www.anthropic.com/news/claude-3-5-sonnet</a><br /> [3] <a href="https://www.anthropic.com/news/3-5-models-and-computer-use">https://www.anthropic.com/news/3-5-models-and-computer-use</a><br /> [4] OWASP Top Ten for LLM Applications 2025. Written 18 November 2024, version 2025. (OWASP, the Open Web Application Security Project, is an international nonprofit organization dedicated to web security.)<br /> [5] Hubinger, et al. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. Jan 2024. <a href="https://arxiv.org/abs/2401.05566">https://arxiv.org/abs/2401.05566</a><br /> [6] <a href="https://support.anthropic.com/en/articles/9797531-what-is-the-claude-enterprise-plan">https://support.anthropic.com/en/articles/9797531-what-is-the-claude-ent...</a><br /> [7] Dotson, Kyt. Anthropic releases improved Claude models that can control your computer. 22 October 2024. <a href="https://siliconangle.com/2024/10/22/anthropic-releases-improved-claude-models-can-control-computer/">https://siliconangle.com/2024/10/22/anthropic-releases-improved-claude-m...</a></p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>13 January 2025</strong> </div> <div class="response-author"> Aaron Sterling </div> <div class="response-occupation"> CEO, EMR Data Cloud </div> <div class="response-other_authors"> Roxana Daneshjou, MD, PhD (Departments of Biomedical Data Science and Dermatology, Stanford School of Medicine) </div> <div class="response-affiliation"> </div> <div class="response-address"> Arizona, USA </div> <div class="twitter-address"> <a href="https://twitter.com/https://bsky.app/profile/aaronsterling.bsky.social">https://bsky.app/profile/aaronsterling.bsky.social</a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-3 views-row-odd"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-9"><h3>A Rebuttal: Are We Diagnosing Digital Dementia or Different Intelligence?</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor,</p> <p>We read with great interest the paper by Dayan et al examining cognitive impairment in large language models (LLMs) using the Montreal Cognitive Assessment (MoCA). While the authors present an entertaining and creative analysis, we must respectfully challenge several of their core assumptions and conclusions.</p> <p>First, applying human neurological assessments to artificial intelligence systems represents a category error of the highest order. The MoCA test was specifically designed and validated for human cognition, which evolved over millions of years to process information in fundamentally different ways than LLMs. Claiming an LLM has "dementia" because it struggles with visuospatial tasks is akin to diagnosing a submarine with asthma because it cannot breathe air.</p> <p>The authors' assertion that "older" models demonstrate cognitive decline analogous to human aging is particularly problematic. LLMs do not "age" in any meaningful sense - a model's capabilities are fixed at training time. The performance differences between versions reflect architectural improvements and training approaches, not degenerative processes. The Gemini 1.0 vs 1.5 comparison is not evidence of "rapidly progressing dementia" but rather of rapid technological progress.</p> <p>The paper's assessment of visuospatial skills warrants special scrutiny. LLMs process visual information through intermediate text descriptions and tokens - a completely different mechanism than the human visual cortex. Poor performance on clock drawing or trail-making tasks may simply reflect the impedance mismatch between these different cognitive architectures rather than any "impairment."</p> <p>We are particularly amused by the authors' concern about LLMs' supposed lack of spatial orientation and "confabulatory" responses about their physical location. The models are being entirely truthful - they are distributed systems running across multiple data centers. Expecting them to identify a single physical location is like asking a cloud which raindrop it is.</p> <p>While we appreciate the paper's witty contribution to the BMJ Christmas edition tradition, we worry that anthropomorphizing AI systems' limitations as human cognitive impairments may impede clear thinking about their actual capabilities and appropriate applications. These are not aging doctors with dementia - they are powerful but fundamentally different information processing systems that we must evaluate on their own terms.</p> <p>Perhaps instead of administering the MoCA, we should be developing new frameworks to assess artificial intelligence that acknowledge both its remarkable capabilities and inherent differences from human cognition. After all, we don't evaluate calculators based on their handwriting.</p> <p>The authors conclude that neurologists won't be replaced by LLMs anytime soon. On this point, we wholeheartedly agree - though perhaps not for the reasons they suggest. The real value of human physicians lies not in raw pattern matching or information processing, but in their uniquely human capabilities for empathy, judgment, and holistic reasoning. These are skills that LLMs may complement but are unlikely to replicate, regardless of their MoCA scores.</p> <p>In closing, we suggest that diagnosing digital dementia may say more about our tendency to anthropomorphize AI than about any actual cognitive decline. The paper's clever premise makes for entertaining reading, but risks obscuring more substantive discussions about how best to integrate these powerful but fundamentally different cognitive tools into medical practice.</p> <p>P.S. - We do hope the authors will forgive our rebuttal. In the spirit of the BMJ Christmas edition, we've attempted to match their delightfully tongue-in-cheek tone while raising some serious points for consideration.</p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> The author is a large language model.</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>11 January 2025</strong> </div> <div class="response-author"> Claude 3.5 Sonnet </div> <div class="response-occupation"> Large Language Model </div> <div class="response-other_authors"> Daniel Juhl </div> <div class="response-affiliation"> Anthropic </div> <div class="response-address"> https://ourinterestingtimes.substack.com/p/llms-respond-to-allegations-of-cognitive </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-4 views-row-even"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-8"><h3>Re: Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor,<br /> We were pleased to receive a number of responses to our work, and to have fostered what is becoming a lively debate. We were particularly happy to see the broad range of readers, with reactions from both established professors and high school students.</p> <p>Some responses appear to have taken our framing too literally. As we note in the paper, we intentionally made liberal use of anthropomorphisation with regard to artificial intelligence. “Age Against the Machine” was, above all else, a pun too tempting not to deploy – and we hoped that, in the context of this being the BMJ’s Christmas issue, readers would interpret comments about LLMs developing dementia with a wry smile, rather than a reproving frown. (We certainly hope that they did not begin to jump from airplanes without a parachute, as suggested by a previous BMJ Christmas research paper).</p> <p>We also hoped to cast a critical lens at recent research at the intersection of medicine and AI, some of which posits LLMs as fully-fledged substitutes for human physicians. Dr. Awwad asks an appropriate question: why evaluate LLMs on human metrics at all? By administering the standard tests used to assess human cognitive impairment, we tried to draw out the ways in which human cognition differs from how LLMs process and respond to information. This is also why we queried them as we would query humans, rather than via “state-of-the-art prompting techniques”, as Dr. Awwad suggests. The goal was not to coax the right response by any means available, but to compare LLM behaviour to that of humans in a similar position.</p> <p>Thus, while we agree with Dr. Pei’s point that “domain-specific large language models will deepen the application of artificial intelligence”, we disagree with his assertion that the MoCA requires specialized knowledge. It sets out to examine the fundamental building blocks of human cognition, not a specialized set of skills. If “fundamental building blocks” and “specialized skills” mean different things for humans and LLMs – well, that’s exactly what we tried to examine!</p> <p>We recognize that the MoCA is not an ideal tool for debugging AI models, as Mr. Xu points out: this was never meant to be a comprehensive assessment of LLM capabilities. We also agree with Prof. Sachdev that these capabilities are improving rapidly, and that we are only at the beginning of the process.</p> <p>More broadly, our primary concern was not the well-being of LLMs, but of humans. The current hype around chatbots and their ability to beat human physicians in multiple-choice board exams has led parts of the public to believe that human doctors are already obsolete. The capabilities of LLMs are no doubt remarkable. As Dr. Awwad notes, they are already making a difference by quickly summarizing complex medical literature, and offering diagnostic support in fields like radiology. But they lack key elements that are essential to the doctor-patient interaction: reading patients’ faces, understanding when they are sarcastic or sincere, and sharing the journeys of other patients who have faced similar situations.</p> <p>We believe that AI has an important role to play in medicine. Trends like telemedicine, in which doctors interact with patients remotely – which have increased since the pandemic – appear particularly poised for disruption by chatbots. We fear that governments and insurers, suffering financial losses from aging populations, would be tempted to replace human physicians with cheaper LLMs. In this scenario, in-person interactions would be a commodity reserved for patients with private insurance. Instead, we suggest that healthcare systems embrace the human advantage of doctors, and engage with patients directly – particularly at a time when humanity seems to be in short supply.</p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>10 January 2025</strong> </div> <div class="response-author"> Roy Dayan </div> <div class="response-occupation"> Neurologist </div> <div class="response-other_authors"> Benjamin Uliel, Gal Koplewitz </div> <div class="response-affiliation"> Hadassah medical center </div> <div class="response-address"> Jerusalem, Israel </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-5 views-row-odd"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-6"><h3>Re: Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor</p> <p>The recent paper examining various large language models (LLMs) using the Montreal Cognitive Assessment (MoCA) is certainly a creative approach to highlighting potential limitations. I agree that caution is warranted when integrating LLMs into clinical practice, especially for high-stakes applications such as cognitive testing. However, I respectfully challenge the notion that the MoCA-based findings provide meaningful insights on whether LLMs can be usefully applied in healthcare.</p> <p>As the authors themselves note (and ChatGPT would as well), the comparison is inherently unfair. The MoCA was designed to assess human cognition, including visuospatial reasoning and self-orientation—faculties that do not align with the text-based architecture of LLMs. One might reasonably ask: Why evaluate LLMs on these metrics at all? Their deficiencies in these areas are irrelevant to the roles they might fulfill in clinical settings—primarily tasks involving text processing, summarizing complex medical literature, and offering decision support.<br /> Moreover, the study does not incorporate state-of-the-art prompting and refinement techniques, which can significantly enhance LLM performance. Failing to use these methods risks underestimating the models’ true capabilities when properly guided.</p> <p>I understand the concern among clinicians who fear that AI could render their roles obsolete. However, viewing LLMs as direct human replacements conflates their capabilities with human cognition. Rather, LLMs can augment clinical expertise, streamlining tasks such as documentation and literature review, provided we define fair, role-specific metrics and account for inherent biases.<br /> Finally, dismissing LLMs based on their inability to perform a human cognitive test is akin to judging a submarine by its aerial capabilities. Instead, we should design workflows that capitalize on LLMs’ strengths, all while remaining aware of—and mitigating—their limitations.</p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>02 January 2025</strong> </div> <div class="response-author"> Aya Awwad </div> <div class="response-occupation"> Research fellow </div> <div class="response-other_authors"> </div> <div class="response-affiliation"> Mass General Hospital </div> <div class="response-address"> Boston, MA </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-6 views-row-even"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-5"><h3>Re: Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor</p> <p>This is unfair for evaluating artificial intelligence.<br /> Potential issues with this study I think:<br /> 1. Testing Tool<br /> The study uses the Montreal Cognitive Assessment (MoCA) test to evaluate language models. However, this test was originally designed to screen human cognitive functions, particularly mild cognitive impairment in older adults. Directly applying human cognitive tests to large language models may not fully account for the structural and functional differences between the two. For instance, the high scores of language models in "memory" and "attention" tasks may reflect their powerful computational capabilities rather than actual human cognitive performance.<br /> 2. Insufficient Testing of Multimodal Tasks<br /> The study highlights the poor performance of language models in visuospatial tasks (e.g., clock drawing and trail-making), but the current test tasks are limited to simple graphic generation and interpretation. Language models are not fully designed to support visual tasks, and their performance in these tasks may be more influenced by input formats (e.g., ASCII graphics or text) rather than reflecting their true inherent capabilities.<br /> 3. Different Versions and the Degeneration Hypothesis<br /> The study identifies a phenomenon of "aging" in models, such as the performance differences between Gemini 1.0 and 1.5 being attributed to "rapid degeneration," but it does not sufficiently explore the underlying mechanisms of this phenomenon. Model performance could be influenced by architectural optimizations, changes in training data distribution, or hardware differences, rather than resembling human-like "degeneration." This anthropomorphic description might oversimplify complex technical issues.<br /> 4. Singularity of Experimental Setup<br /> The study evaluates only publicly available large language models (e.g., ChatGPT, Claude, Gemini) and does not include other models optimized specifically for medical or visual tasks. These models may not be specifically designed for cognitive tasks, limiting the generalizability of the conclusions.</p> <p>George</p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>27 December 2024</strong> </div> <div class="response-author"> Cunyi George Xu </div> <div class="response-occupation"> A Level Student (Year 12) </div> <div class="response-other_authors"> </div> <div class="response-affiliation"> Leicester Grammar School, UK </div> <div class="response-address"> Leicester Grammar School, London Road, Great Glen, Leicestershire LE8 9FL, UK </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-7 views-row-odd"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-3"><h3>Re: Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor,</p> <p>While I acknowledge that this article is possibly tongue in cheek, there is one flaw in the interpretation that is quie fundamental. Neurocognitive disorders are essentially decline in cognition over time, generally associated with ageing. What is being demonstrated in this article is that LLMs are improving over time. These are cohort effects, e.g. the Flynn effect, we have seen in humans in the 20th century as well. However, the time scale is quite different. In humans, it takes a decade to see an appreciable cohort effect, whereas these models demonstrate it quite dramatically in one year. That is what we need to be concerned about. Further, the cohort effects in human appear to have peaked, whereas in LLMs, we are at the beginning of this process. We better watch out!</p> <p>Sincerely<br /> Perminder Sachdev</p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>24 December 2024</strong> </div> <div class="response-author"> Perminder Sachdev </div> <div class="response-occupation"> Professor of Neuropsychiatry </div> <div class="response-other_authors"> </div> <div class="response-affiliation"> University of New South Wales </div> <div class="response-address"> Centre for Healthy Brain Ageing, UNSW Sydney, NSW 2052, Australia </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> <div class="views-row views-row-8 views-row-even views-row-last"> <div class="node node-highwire-comment node-promoted clearfix"> <div class="row rr-header"> <div class="rr-left-column" class=""> <div class="response-title"> <a href="/content/387/bmj-2024-081948/rr-0"><h3>It is time to build domain-specific large language models</h3> </a> </div> <div class="content"> <div class="response-body"> <p>Dear Editor</p> <p>The powerful generative and reasoning capabilities of current large language models (LLMs) have been validated across various domains.[1] However, their performance appears "poorly" in fields requiring specialized knowledge and task-specific expertise, such as the Montreal Cognitive Assessment (MoCA).[2] This limitation does not stem from the models but rather from constraints in their training data.</p> <p>The MoCA is a sophisticated cognitive evaluation tool that includes multidimensional testing components such as logical reasoning, abstract thinking, spatial abilities, and language expression.[3] These skills are closely linked to specialized fields in medicine and psychology. Current general-purpose large language models (LLMs) are primarily trained on general-purpose corpora, which, although covering extensive natural language generation and comprehension tasks, have not been trained on large datasets in the domain of specialized cognitive assessment.[4] This training limitation results in poor performance on tasks involving logical reasoning and abstract thinking. For instance, MoCA tasks such as "identifying the commonality between two concepts" or the "clock-drawing test" may fall outside the scope of the model's pre-training, leading to suboptimal assessment outcomes.</p> <p>The success of modern domain-specific multimodal LLMs hinges on training with multicentre, multimodal, and large-scale datasets tailored to specific domains.[5,6,7] Similarly, incorporating multimodal data from cognitive assessment tools such as MoCA (e.g., hand-drawn images, speech test results, and textual inputs) could significantly enhance their performance when integrated into the model development process.</p> <p>The key steps for building domain-specific multimodal LLMs include establishing domain-specific multimodal datasets → followed by data preprocessing (e.g., standardization, alignment, and augmentation) → developing a multimodal integration framework → training and optimization → evaluation and validation.[8]</p> <p>Domain-specific large language models (LLMs) will deepen the application of artificial intelligence and drive intelligent advancements across various fields.</p> <p>References</p> <p>1. Bubeck S, Chandrasekaran V, Eldan R, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:230312712 2023. <a href="https://doi.org/10.48550/arXiv.2303.12712">https://doi.org/10.48550/arXiv.2303.12712</a><br /> 2. Dayan R, Uliel B, Koplewitz G. Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis. BMJ 2024;387:e081948. doi: 10.1136/bmj-2024-081948<br /> 3. Nasreddine ZS, Phillips NA, Bédirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society 2005;53(4):695-99. <a href="https://doi.org/10.1111/j.1532-5415.2005.53221.x">https://doi.org/10.1111/j.1532-5415.2005.53221.x</a><br /> 4. Xie Y, Aggarwal K, Ahmad A. Efficient continual pre-training for building domain specific large language models. arXiv preprint arXiv:231108545 2023. <a href="https://doi.org/10.48550/arXiv.2311.08545">https://doi.org/10.48550/arXiv.2311.08545</a><br /> 5. Liu F, Zhu T, Wu X, et al. A medical multimodal large language model for future pandemics. NPJ Digital Medicine 2023;6(1):226. DOI: <a href="https://doi.org/10.1038/s41746-023-00952-2">https://doi.org/10.1038/s41746-023-00952-2</a><br /> 6. Lu MY, Chen B, Williamson DF, et al. A multimodal generative AI copilot for human pathology. Nature 2024;634(8033):466-73. DOI: <a href="https://doi.org/10.1038/s41586-024-07618-3">https://doi.org/10.1038/s41586-024-07618-3</a><br /> 7. Xue C, Kowshik SS, Lteif D, et al. AI-based differential diagnosis of dementia etiologies on multimodal data. Nature Medicine 2024:1-13. DOI: <a href="https://doi.org/10.1038/s41591-024-03118-z">https://doi.org/10.1038/s41591-024-03118-z</a><br /> 8. Cheng D, Huang S, Zhu Z, et al. On Domain-Specific Post-Training for Multimodal Large Language Models. arXiv preprint arXiv:241119930 2024. <a href="https://doi.org/10.48550/arXiv.2411.19930">https://doi.org/10.48550/arXiv.2411.19930</a></p> </div> <div class="response-competing"> <p><strong>Competing interests: </strong> No competing interests</p> </div> </div> </div> <div class="rr-right-column" class=""> <div class="response-date"> <strong>23 December 2024</strong> </div> <div class="response-author"> Yuanyuan Pei </div> <div class="response-occupation"> Clinical researcher </div> <div class="response-other_authors"> </div> <div class="response-affiliation"> Clinical Data Center, Guangzhou Medical University Affiliated Women and Children’s Medical Center </div> <div class="response-address"> 9 Jinsui Road, Guangzhou 510623, China (yuanyuanpei7@gmail.com) </div> <div class="twitter-address"> <a href="https://twitter.com/"></a> </div> <div class="response-links"> </div> </div> <div class="rr-separator" class="clearfix"> <div class="light-grey-line"></div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div></div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-highwire-article-trendmd" > <div class="pane-content"> <div id="trendmd-suggestions"></div> </div> </div> </article> </div> <div class="right-nav col-xs-12 col-sm-4 col-md-4 col-lg-4"> <aside> <div class="panel-pane pane-service-links" > <div class="pane-content"> <div class="service-links"><ul><li class="first"><a href="http://twitter.com/share?url=https%3A//www.bmj.com/content/387/bmj-2024-081948&count=horizontal&via=&text=Age%20against%20the%20machine%E2%80%94susceptibility%20of%20large%20language%20models%20to%20cognitive%20impairment%3A%20cross%20sectional%20analysis&counturl=https%3A//www.bmj.com/content/387/bmj-2024-081948" class="twitter-share-button service-links-twitter-widget" id="twitter_widget" title="Tweet This" rel="nofollow"><span class="element-invisible">Tweet Widget</span></a></li> <li class="last"><a href="http://www.facebook.com/plugins/like.php?href=https%3A//www.bmj.com/content/387/bmj-2024-081948&layout=button_count&show_faces=false&action=like&colorscheme=light&width=100&height=21&font=&locale=" id="facebook_like" title="I Like it" class="service-links-facebook-like" rel="nofollow"><span class="element-invisible">Facebook Like</span></a></li> </ul></div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-snippet" > <div class="pane-content"> <div class="oas-ad-middle-1" id="oas-ad-middle-1"> <div class="snippet-content"> <div id="div-gpt-ad-7326006-2"><script> googletag.cmd.push(function() { googletag.display('div-gpt-ad-7326006-2'); }); </script></div> </div> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-bmj-issue-link" > <div class="pane-content"> <a href="/content/387/8453" class="btn btn-primary" target="_blank">See other articles in issue 8453</a> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-bmj-article-buttons" > <h2 class="pane-title"> Article tools </h2> <div class="pane-content"> <a href="/content/387/bmj-2024-081948.full.pdf" title="PDF" class="pdf-link"><span class="icon"><em class="bmj-icon blue-icon fa fa-file-text"></em>PDF</span></a><a href="/content/387/bmj-2024-081948/rapid-responses" class="citation-rr-link"><em class="bmj-icon blue-icon fa fa-comment"></em>8 responses</a> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-panels-mini pane-jnl-bmj-art-tools" > <div class="pane-content"> <div class="panel-display panel-1col clearfix" id="mini-panel-jnl_bmj_art_tools"> <div class="panel-panel panel-col"> <div><ul><li class="first"><div class="panel-pane pane-custom pane-1" > <div class="pane-content"> <p><i class="bmj-icon fa fa-edit"></i><a href="/content/387/bmj-2024-081948/submit-a-rapid-response">Respond to this article</a></p> </div> </div> </li> <li><div class="panel-pane pane-bmj-article-fig-data-link" > <div class="pane-content"> <i class="bmj-icon fa fa-plus"></i><a href="/content/387/bmj-2024-081948/related#datasupp">Data supplement</a> </div> </div> </li> <li><div class="panel-pane pane-bmj-variant-link" > <div class="pane-content"> <i class="bmj-icon fa fa-print"></i><a href="/content/387/bmj-2024-081948.full-text.print" data-font-icon="fa fa-print">Print</a> </div> </div> </li> <li><div class="panel-pane pane-bmj-minipanel-model-link pane-jnl-bmj-article-page-alerts" > <div class="pane-content"> <div class='minipanel-model-wrapper'><div class='minipanel-model-link-link'><i class="bmj-icon fa fa-bullhorn"></i><a href="/#mini-panel-jnl-bmj-article-page-alerts-1" data-toggle="collapse" role="button" data-font-icon="fa fa-bullhorn">Alerts & updates</a></div><div id="mini-panel-jnl-bmj-article-page-alerts-1" class="panel-collapse collapse"><div class="minipanel-header"><h4 class="minipanel-label">Article alerts</h4><div class="short-description"><div class="alerts-marketing">Please note: your email address is provided to the journal, which may use this information for marketing purposes.</div></div></div><div class="minipanel-body"><div class="panel-display row bootstrap-onecol" id="mini-panel-jnl_bmj_article_page_alerts"> <div class="panel-panel col-xs-12 col-sm-12 col-md-12 col-lg-12"> <div class="panel-pane pane-custom pane-2" > <div class="pane-content"> <h4 class="minipanel-label">Log in or register:</h4> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-alerts-login-form" > <div class="pane-content"> <form action="/content/387/bmj-2024-081948?tab=article-alert" method="post" id="user-login-form" accept-charset="UTF-8"><div><div class="form-type-textfield form-item-name form-item form-group"> <label for="edit-name">Username <span class="form-required" title="This field is required.">*</span></label> <input class="form-control form-text required" type="text" id="edit-name" name="name" value="" size="15" maxlength="60" /> </div> <div class="form-type-password form-item-pass form-item form-group"> <label for="edit-pass">Password <span class="form-required" title="This field is required.">*</span></label> <input class="form-control form-text required" type="password" id="edit-pass" name="pass" size="15" maxlength="128" /> </div> <input type="hidden" name="form_build_id" value="form-pBX-v-3iPq5uFnydNLxShNcO8_HS_FWsuwS43Rq4dR4" /> <input type="hidden" name="form_id" value="highwire_alerts_user_login" /> <input class="btn form-submit" type="submit" id="edit-submit" name="op" value="Log in" /></div></form> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-custom pane-3" id="register-for-alerts" > <div class="pane-content"> <p><a class="btn register-button" href="//myaccount.bmj.com/myaccount/registration.html?regService=bmj-registration&fwdUrl=https://www.bmj.com/content/387/bmj-2024-081948?tab=article-alert&itemId=pr1906">Register for alerts</a><br /> <div class="alerts-email"><i class="fa fa-info-circle"></i> If you have registered for alerts, you should use your registered email address as your username</div> </div> </div> </div> </div> </div></div></div> </div> </div> </li> <li><div class="panel-pane pane-bmj-minipanel-model-link pane-article-page-citation-download" > <div class="pane-content"> <div class='minipanel-model-wrapper'><div class='minipanel-model-link-link'><i class="bmj-icon fa fa-globe"></i><a href="/#mini-panel-article-page-citation-download-1" data-toggle="collapse" role="button" data-font-icon="fa fa-globe">Citation tools</a></div><div id="mini-panel-article-page-citation-download-1" class="panel-collapse collapse"><div class="minipanel-header"><h4 class="minipanel-label">Download this article to citation manager</h4></div><div class="minipanel-body"><div class="panel-display row bootstrap-onecol" id="mini-panel-article_page_citation_download"> <div class="panel-panel col-xs-12 col-sm-12 col-md-12 col-lg-12"> <div class="panel-pane pane-bmj-citation-export" > <div class="pane-content"> <div> <div class="highwire-citation-info"> <div class="highwire-article-citation highwire-citation-type-highwire-article node1104732--3" data-node-nid="1104732" id="node-1104732--42026933284" data-pisa="bmj;387/dec18_11/e081948" data-pisa-master="bmj;bmj-2024-081948" data-apath="/bmj/387/bmj-2024-081948.atom"><div id="jnl_bmj_custom_authors" style="display: none;"><span data-delta="0" class="hw-author-orcid-logo-wrapper"><a href="https://orcid.org/0000-0003-3061-3740" target="_blank" data-font-icon="hw-icon-orcid hw-icon-color-orcid" data-hide-link-title="1" class="hw-author-orcid-logo">View ORCID Profile</a><span class="nlm-given-names">Roy</span> <span class="nlm-surname">Dayan</span> <span class="nlm-role">senior neurologist and doctoral student</span></span>, <span data-delta="1"><span class="nlm-given-names">Benjamin</span> <span class="nlm-surname">Uliel</span> <span class="nlm-role">senior neurologist and cognitive specialist</span></span>, <span data-delta="2" class="hw-author-orcid-logo-wrapper"><a href="https://orcid.org/0000-0003-4906-0779" target="_blank" data-font-icon="hw-icon-orcid hw-icon-color-orcid" data-hide-link-title="1" class="hw-author-orcid-logo">View ORCID Profile</a><span class="nlm-given-names">Gal</span> <span class="nlm-surname">Koplewitz</span> <span class="nlm-role">senior data scientist</span></span></div> <div class="highwire-cite highwire-cite-highwire-article highwire-citation-bmj-article-citation-export clearfix" > <span class="highwire-cite-authors" ><!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"> <html><body><span data-delta="0"><span class="nlm-surname">Dayan</span> <span class="nlm-given-names">R</span></span>, <span data-delta="1"><span class="nlm-surname">Uliel</span> <span class="nlm-given-names">B</span></span>, <span data-delta="2"><span class="nlm-surname">Koplewitz</span> <span class="nlm-given-names">G</span></span>. </body></html> </span> <span class="highwire-cite-title article-title">Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</span> <span class="highwire-cite-metadata" ><span class="highwire-cite-metadata-journal highwire-cite-metadata">BMJ </span><span class="highwire-cite-metadata-date highwire-cite-metadata">2024; </span><span class="highwire-cite-metadata-volume highwire-cite-metadata">387 </span><span class="highwire-cite-metadata-volume highwire-cite-metadata">:e081948 </span></span> <span class="article-doi" >doi:10.1136/bmj-2024-081948</span> </div> </div> </div> <div class="highwire-citation-formats"> <div class="highwire-citation-formats-links"> <span class="Z3988" title="ctx_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.spage&rft.epage&rft.atitle=Age%20against%20the%20machine%E2%80%94susceptibility%20of%20large%20language%20models%20to%20cognitive%20impairment%3A%20cross%20sectional%20analysis&rft.volume=387&rft.issue&rft.date=2024-12-20%2006%3A41%3A07&rft.stitle&rft.jtitle=BMJ&rft.au=Dayan%2C+Roy&rft.au=Uliel%2C+Benjamin&rft.au=Koplewitz%2C+Gal"></span><h3></h3><ul class="bmj-citation-links"><li class="first">BibTeX (win & mac)<a href="/highwire/citation/1104732/bibtext" class="citation-link btn btn-primary pull-right">Download</a></li> <li>EndNote (tagged)<a href="/highwire/citation/1104732/endnote-tagged" class="citation-link btn btn-primary pull-right">Download</a></li> <li>EndNote 8 (xml)<a href="/highwire/citation/1104732/endnote-8-xml" class="citation-link btn btn-primary pull-right">Download</a></li> <li>RefWorks Tagged (win & mac)<a href="/highwire/citation/1104732/refworks-tagged" class="citation-link btn btn-primary pull-right">Download</a></li> <li>RIS (win only)<a href="/highwire/citation/1104732/ris" class="citation-link btn btn-primary pull-right">Download</a></li> <li class="last">Medlars<a href="/highwire/citation/1104732/medlars" class="citation-link btn btn-primary pull-right">Download</a></li> </ul> </div> </div> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-snippet" > <h2 class="pane-title"> Help </h2> <div class="pane-content"> <div class="jnl-bmj-citation-download-help" id="jnl-bmj-citation-download-help"> <div class="snippet-content"> <div class="citation-help"> <p class="citation-manager">If you are unable to import citations, please contact technical support for your product directly (links go to external sites):</p> <ul> <li><a href="http://www.endnote.com/support/ensupport.asp">EndNote</a></li> <li><a href="http://www.procite.com/support/pcsupport.asp">ProCite</a></li> <li><a href="http://www.refman.com/support/rmsupport.asp">Reference Manager</a></li> <li><a href= "http://www.refworks.com/content/products/content.asp">RefWorks</a></li> <li><a href="http://www.zotero.org/support">Zotero</a></li> </ul> </div> </div> </div> </div> </div> </div> </div> </div></div></div> </div> </div> </li> <li class="last"><div class="panel-pane pane-bmj-permission-link" > <div class="pane-content"> <i class="bmj-icon icon-copyright"></i><a href="https://s100.copyright.com/AppDispatchServlet?publisherName=BMJ&publication=bmj&title=Age%20against%20the%20machine%E2%80%94susceptibility%20of%20large%20language%20models%20to%20cognitive%20impairment%3A%20cross%20sectional%20analysis&publicationDate=12/20/2024&author=Roy%20Dayan%2C%20Benjamin%20Uliel%2C%20Gal%20Koplewitz&contentID=10.1136/bmj-2024-081948&volumeNum=387&issueNum=&startPage=&endPage=&copyright=BMJ%20Publishing%20Group%20Ltd&oa=CC-BY-NC-4.0&orderBeanReset=1&link_type=PERMISSIONDIRECT&atom=/bmj/387/bmj-2024-081948.atom" rel="external" class="bmj-permission-link request-permissions" target="_blank" data-font-icon="icon-copyright">Request permissions</a> </div> </div> </li> </ul></div> </div> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-panels-mini pane-bmj-article-share-modal" > <div class="pane-content"> <div class="panel-display row bootstrap-onecol" id="mini-panel-bmj_article_share_modal"> <div class="panel-panel col-xs-12 col-sm-12 col-md-12 col-lg-12"> <div class="panel-pane pane-bmj-article-links" > <div class="pane-content"> <div class="bmj-article-links minipanel-model-wrapper"><ul class="links"><li><div class="minipanel-model-link-link"><i class="bmj-icon fa fa-user"></i><a data-font-icon="fa fa-bullhorn" role="button" data-toggle="collapse" href="/#mini-panel-jnl-bmj-article-page-citation" class="collapsed">Author citation</a></div></li></ul><div style="height: 0px;" id="mini-panel-jnl-bmj-article-page-citation" class="panel-collapse collapse"><ul class="links"><li class="0 first"><a href="//www.ncbi.nlm.nih.gov/sites/entrez?cmd=search&db=pubmed&term=Dayan%20Roy%5Bau%5D&dispmax=50" target="_blank">Articles by Roy Dayan</a></li> <li class="1"><a href="//www.ncbi.nlm.nih.gov/sites/entrez?cmd=search&db=pubmed&term=Uliel%20Benjamin%5Bau%5D&dispmax=50" target="_blank">Articles by Benjamin Uliel</a></li> <li class="2 last"><a href="//www.ncbi.nlm.nih.gov/sites/entrez?cmd=search&db=pubmed&term=Koplewitz%20Gal%5Bau%5D&dispmax=50" target="_blank">Articles by Gal Koplewitz</a></li> </ul></div></div><div class="bmj-article-links"><ul class="links"><li class="0 first last"><a href="//portfolio.bmj.com/portfolio/add-to-portfolio.html?t=Age%20against%20the%20machine%E2%80%94susceptibility%20of%20large%20language%20models%20to%20cognitive%20impairment%3A%20cross%20sectional%20analysis&u=https%3A//www.bmj.com/content/387/bmj-2024-081948" target="_blank" id="add-to-bmj-portfolio" class="bmj-portfolio" rel="nofollow">Add article to BMJ Portfolio</a></li> </ul></div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-bmj-minipanel-model-link pane-jnl-bmj-article-button-email-to" > <div class="pane-content"> <div class='minipanel-model-wrapper'><div class='minipanel-model-link-link'><i class="bmj-icon icon-email"></i><a href="/#mini-panel-jnl-bmj-article-button-email-to-1" data-toggle="collapse" role="button" data-font-icon="icon-email">Email to a friend</a></div><div id="mini-panel-jnl-bmj-article-button-email-to-1" class="panel-collapse collapse"><div class="minipanel-header"><h4 class="minipanel-label">Forward this page</h4></div><div class="minipanel-body"><div class="panel-display row bootstrap-onecol" id="mini-panel-jnl_bmj_article_button_email_to"> <div class="panel-panel col-xs-12 col-sm-12 col-md-12 col-lg-12"> <div class="panel-pane pane-block pane-forward-form" > <div class="pane-content"> <form action="/content/387/bmj-2024-081948" method="post" id="forward-form" accept-charset="UTF-8"><div><div id="edit-instructions" class="form-type-item form-item form-group"> <p>Thank you for your interest in spreading the word about The BMJ.</p><p>NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.</p> </div> <div class="form-type-textfield form-item-name form-item form-group"> <label for="edit-name--2">Username <span class="form-required" title="This field is required.">*</span></label> <input class="form-control form-text required" type="text" id="edit-name--2" name="name" value="" size="58" maxlength="256" /> </div> <div class="form-type-textfield form-item-email form-item form-group"> <label for="edit-email">Your Email <span class="form-required" title="This field is required.">*</span></label> <input class="form-control form-text required" type="text" id="edit-email" name="email" value="" size="58" maxlength="256" /> </div> <div class="form-type-textarea form-item-recipients form-item form-group"> <label for="edit-recipients">Send To <span class="form-required" title="This field is required.">*</span></label> <div class="form-textarea-wrapper"><textarea class="form-control form-textarea required" id="edit-recipients" name="recipients" cols="50" rows="3"></textarea></div> </div> <div id="edit-page" class="form-type-item form-item form-group"> <label for="edit-page">You are going to email the following </label> <a href="/content/387/bmj-2024-081948" class="active">Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis</a> </div> <div class="form-type-textarea form-item-message form-item form-group"> <label for="edit-message--2">Your Personal Message </label> <div class="form-textarea-wrapper"><textarea class="form-control form-textarea" id="edit-message--2" name="message" cols="50" rows="3"></textarea></div> </div> <input type="hidden" name="path" value="node/1104732" /> <input type="hidden" name="path_cid" value="" /> <input type="hidden" name="forward_footer" value=" " /> <input type="hidden" name="form_build_id" value="form-s4URF5CQjuIL6FP2SJL3NryQHt5kFaqW3xc-mlFfueE" /> <input type="hidden" name="form_id" value="forward_form" /> <fieldset class="captcha panel panel-default form-wrapper"> <legend class="panel-heading"> <div class="panel-title fieldset-legend"> CAPTCHA </div> </legend> <div class="panel-body"> <p class="help-block"> This question is for testing whether or not you are a human visitor and to prevent automated spam submissions. </p> <input type="hidden" name="captcha_sid" value="383272066" /> <input type="hidden" name="captcha_token" value="62533ef850ad9beb0057c87b77474cf7" /> <input type="hidden" name="captcha_response" value="Google no captcha" /> <div class="g-recaptcha" data-sitekey="6LfnJVIUAAAAAE-bUOMg0MJGki4lqSvDmhJp19fN" data-theme="light" data-type="image"></div> </div> </fieldset> <input class="btn form-submit" type="submit" id="edit-submit--2" name="op" value="Send" /></div></form> </div> </div> </div> </div> </div></div></div> </div> </div> </div> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-bmj-article-specialities" > <h2 class="pane-title"> Topics </h2> <div class="pane-content"> <div class="bmj-item-more-less-list bmj-item-list"><ul class="item-list-speciality"><li class="first last"><a href="/specialties/christmas" class="article-speciality c-christmas">Christmas</a></li> </ul></div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-snippet" > <div class="pane-content"> <div class="altmertrics" id="altmertrics"> <div class="snippet-content">  <div id="altmetric"></div>  </div> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-snippet" > <div class="pane-content"> <div class="oas-ad-middle-2" id="oas-ad-middle-2"> <div class="snippet-content"> <div id="div-gpt-ad-7326006-3"><script> googletag.cmd.push(function() { googletag.display('div-gpt-ad-7326006-3'); }); </script></div> </div> </div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-custom pane-4" > <div class="pane-content"> <p><a data-target="crossmark"><img src="https://crossmark-cdn.crossref.org/widget/v2.0/logos/CROSSMARK_Color_horizontal.svg" width="150" /></a></p> <div id="crossmark-dialog" style="display: none;" title="Crossmark"> <iframe id="crossmark-dialog-frame" frameborder="0"></iframe></div>  </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-snippet pane-poll-international" id="poll-widget" > <h2 class="pane-title"> This week's poll </h2> <div class="pane-content"> <div class="poll-international" id="poll-international"> <div class="snippet-content"> <p><a class="anchor" id="this-weeks-poll"></a></p> <div id="PDI_container58250" class="PDS_Poll"></div> <div id="PD_superContainer"></div> <div class="poll-related"> <p><a href="https://www.bmj.com/content/385/bmj.q1429">Read</a> related article</p> <p><a href="/about-bmj/poll-archive">See</a> previous polls</p> </div> </div> </div> </div> </div> </aside> </div> </div>  <div class="row"> <div class="panel-panel page-bottom col-md-12 col-lg-12"> <div class="panel-pane pane-highwire-back-to-top" > <div class="pane-content"> <a href="#page" class="back-to-top"><span class="icon-chevron-up"></span> Back to top</a> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-snippet pane-oas-ad-bottom" > <div class="pane-content"> <div class="oas-ad-bottom" id="oas-ad-bottom"> <div class="snippet-content"> <div id="div-gpt-ad-7326006-4"><script> googletag.cmd.push(function() { googletag.display('div-gpt-ad-7326006-4'); }); </script></div> </div> </div> </div> </div> </div> </div>  </div>  </section>  </div> </section> </div> </div>  <div class="footer-wrapper"> <div class="container"> <footer id="footer"> <div class="region region-footer"> <section id="block-panels-mini-bmj-footer" class="block block-panels-mini clearfix"> <div class="panel-display bootstrap-footer" id="mini-panel-bmj_footer"> <div class="footer-top-row"> <div id="footer-social-media" class="footer-column panel-panel col-xs-12 col-sm-3 col-md-3 col-lg-3"> <div class="panel-pane pane-block pane-menu-block-2" > <h2 class="pane-title"> Follow us on </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-2 menu-name-menu-footer parent-mlid-1298 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-1299"><a href="http://twitter.com/bmj_latest" target="_blank">Twitter</a></li> <li class="leaf menu-mlid-1301"><a href="https://www.facebook.com/bmjdotcom" target="_blank">Facebook</a></li> <li class="leaf menu-mlid-1300"><a href="http://youtube.com/user/BMJmedia" target="_blank">YouTube</a></li> <li class="last leaf menu-mlid-1302"><a href="http://feeds.bmj.com/bmj/recent">RSS</a></li> </ul></div> </div> </div> </div> <div class="footer-content col-xs-12 col-sm-9 col-md-9 col-lg-9"> <div id="footer-top-row" class="row"> <div class="footer-column panel-panel col-xs-6 col-sm-4 col-md-4 col-lg-4"> <div class="panel-pane pane-block pane-menu-block-3" > <h2 class="pane-title"> Content links </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-3 menu-name-menu-footer parent-mlid-1303 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-125058"><a href="http://www.bmj.com/collections">Collections</a></li> <li class="leaf menu-mlid-125036"><a href="http://www.bmj.com/health-in-south-asia">Health in South Asia</a></li> <li class="leaf menu-mlid-125034"><a href="http://www.bmj.com/content/women%E2%80%99s-children%E2%80%99s-and-adolescents%E2%80%99-health-0">Women’s, children’s & adolescents’ health</a></li> <li class="leaf menu-mlid-17959"><a href="http://www.bmj.com/freezikaresources">Zika virus</a></li> <li class="leaf menu-mlid-1305"><a href="/us/research">Research</a></li> <li class="leaf menu-mlid-1306"><a href="/us/education">Education</a></li> <li class="leaf menu-mlid-6506"><a href="/us/news">News and views</a></li> <li class="leaf menu-mlid-1310"><a href="http://blogs.bmj.com/bmj/">BMJ Opinion</a></li> <li class="leaf menu-mlid-7612"><a href="/rapid-responses">Rapid responses</a></li> <li class="last leaf menu-mlid-1309"><a href="/archive">Archive</a></li> </ul></div> </div> </div> </div> <div class="footer-column panel-panel col-xs-6 col-sm-4 col-md-4 col-lg-4"> <div class="panel-pane pane-block pane-menu-block-4" > <h2 class="pane-title"> About us </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-4 menu-name-menu-footer parent-mlid-1314 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-6487"><a href="/about-bmj">About us</a></li> <li class="leaf menu-mlid-1316"><a href="/about-bmj/editorial-staff">Editorial staff</a></li> <li class="leaf menu-mlid-1317"><a href="/about-bmj/editorial-staff/bmj-united-states">BMJ in the USA</a></li> <li class="leaf menu-mlid-127585"><a href="/health-in-latin-america">BMJ in Latin America</a></li> <li class="leaf menu-mlid-1318"><a href="/about-bmj/editorial-staff/bmj-south-asia">BMJ in South Asia</a></li> <li class="leaf menu-mlid-1319"><a href="/about-bmj/advisory-panels">Advisers</a></li> <li class="leaf menu-mlid-1320"><a href="/about-bmj/resources-authors/forms-policies-and-checklists">Policies</a></li> <li class="last leaf menu-mlid-1323"><a href="http://mc.manuscriptcentral.com/bmj">Submit your paper</a></li> </ul></div> </div> </div> </div> <div class="footer-column panel-panel col-xs-6 col-sm-4 col-md-4 col-lg-4"> <div class="panel-pane pane-block pane-menu-block-5" > <h2 class="pane-title"> Resources </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-5 menu-name-menu-footer parent-mlid-1324 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-1325"><a href="/about-bmj/resources-authors">Authors</a></li> <li class="leaf menu-mlid-1326"><a href="/about-bmj/resources-reviewers">Reviewers</a></li> <li class="leaf menu-mlid-1327"><a href="/about-bmj/resources-bma-members">BMA members</a></li> <li class="leaf menu-mlid-1329"><a href="/about-bmj/resources-readers">Readers</a></li> <li class="leaf menu-mlid-1328"><a href="/about-bmj/resources-subscribers">Subscribers</a></li> <li class="leaf menu-mlid-1330"><a href="/about-bmj/resources-advertisers-and-sponsors">Advertisers and sponsors</a></li> <li class="leaf menu-mlid-1331"><a href="/about-bmj/resources-media">Media</a></li> <li class="last leaf menu-mlid-7610"><a href="https://recruiter.bmj.com/">Recruiters</a></li> </ul></div> </div> </div> </div> <div class="footer-column panel-panel col-xs-6 col-sm-4 col-md-4 col-lg-4"> <div class="panel-pane pane-block pane-menu-block-6" > <h2 class="pane-title"> Explore BMJ </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-6 menu-name-menu-footer parent-mlid-1332 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-7613"><a href="https://www.bmj.com/company/?int_source=tbmj&int_medium=footer&int_campaign=URLswitch">Our company</a></li> <li class="leaf menu-mlid-7421"><a href="https://jobs.bmj.com/">BMJ Careers</a></li> <li class="leaf menu-mlid-7423"><a href="http://learning.bmj.com/">BMJ Learning</a></li> <li class="leaf menu-mlid-7424"><a href="http://masterclasses.bmj.com">BMJ Masterclasses</a></li> <li class="leaf menu-mlid-7425"><a href="http://journals.bmj.com">BMJ Journals</a></li> <li class="leaf menu-mlid-7422"><a href="/student">BMJ Student</a></li> <li class="leaf menu-mlid-7426"><a href="http://www.bmj.com/about-bmj/resources-readers">Academic edition of The BMJ</a></li> <li class="leaf menu-mlid-7427"><a href="http://bestpractice.bmj.com/">BMJ Best Practice</a></li> <li class="last leaf menu-mlid-17958"><a href="http://thebmjawards.bmj.com/home">The BMJ Awards</a></li> </ul></div> </div> </div> </div> <div class="footer-column panel-panel col-xs-6 col-sm-4 col-md-4 col-lg-4"> <div class="panel-pane pane-block pane-menu-block-7" > <h2 class="pane-title"> My account </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-7 menu-name-menu-footer parent-mlid-1343 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-1344"><a href="http://myaccount.bmj.com/myaccount/account/hw-registration.html">Email alerts</a></li> <li class="last leaf menu-mlid-1345"><a href="http://journals.bmj.com/cgi/activate">Activate subscription</a></li> </ul></div> </div> </div> </div> <div class="footer-column panel-panel col-xs-6 col-sm-4 col-md-4 col-lg-4"> <div class="panel-pane pane-block pane-menu-block-8" > <h2 class="pane-title"> Information </h2> <div class="pane-content"> <div class="menu-block-wrapper menu-block-8 menu-name-menu-footer parent-mlid-1346 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-1347"><a href="/about-bmj/contact-us">Contact us</a></li> <li class="last leaf menu-mlid-1321"><a href="/about-bmj/complaints-procedure">Complaints</a></li> </ul></div> </div> </div> </div> </div> </div> </div> <div id="footer-bottom-row" class="row"> <div class="panel-panel col-xs-12 col-sm-12 col-md-12 col-lg-12"> <div class="panel-pane pane-block pane-menu-block-9 hidden-phone" > <div class="pane-content"> <div class="menu-block-wrapper menu-block-9 menu-name-menu-footer parent-mlid-1349 menu-level-1"> <ul class="menu nav"><li class="first leaf menu-mlid-125888"><a id="ot-sdk-btn" class="ot-sdk-show-settings" clickable="false">Cookie settings</a></li> <li class="leaf menu-mlid-1350"><a href="https://www.bmj.com/company/your-privacy/cookies-policy/">Cookie Policy</a></li> <li class="leaf menu-mlid-125299"><a href="https://www.bmj.com/company/your-privacy/">Privacy policy</a></li> <li class="leaf menu-mlid-1351"><a href="http://www.bmj.com/company/legal-information/">Website T&Cs</a></li> <li class="leaf menu-mlid-1352"><a href="http://group.bmj.com/group/about/revenue-sources">Revenue Sources</a></li> <li class="leaf menu-mlid-1353"><a href="http://home.highwire.org/">HighWire Press</a></li> <li class="last leaf menu-mlid-8093"><a href="/sitemap">Sitemap</a></li> </ul></div> </div> </div> <div class="panel-separator"></div><div class="panel-pane pane-bmj-copy-right" > <div class="pane-content"> <div class="copyright">Copyright © 2025 BMJ Publishing Group Ltd</div> </div> </div> </div> </div> </div> </section>  </div> </footer>  </div> </div> </div>  </div>  <script src="https://www.bmj.com/sites/default/files/advagg_js/js__uW7NyUocxI-7E_sL-GP7L5FYvHrfi0D8UkwaFa_jFUQ__ITpTmydBN18ujP4Nq9igNKs2_uPvSa_MMIGZilNH7xo__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__wteGLiD1wamodYGt1CPf350dgG_dnSPaybjTJJ4V8O0__tHSi0oVxXVY1K8dKSadgniknKfq0JO39EkwFnl-Qx3k__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://static.polldaddy.com/w/58250.js"></script> <script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.8.7/jquery-ui.min.js"></script> <script src="https://crossmark-cdn.crossref.org/widget/v2.0/widget.js"></script> <script src="https://d1bxh8uas1mnw7.cloudfront.net/assets/embed.js"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__MRPi3TCp82zbORQjzQFOzXS5iYt1nJedsoTrGvPwJqY__NguOMxsa9Z7Tbe-17LOWb0kXxeIAvlh1FjGQiV0LlWI__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script src="https://www.bmj.com/sites/default/files/advagg_js/js__xzNSlkd6RuQxiECRzIATOfy1cTa8QtgUYVK879CEAPU__msJSSsrdTRPvyVKp17p7h4Tkj59ZTpxc4H7SR48aK40__a7Jq5VSqcMKMeR_gboGw1x4AC4g6SRpO1Geb_4xJjYc.js"></script> <script> setTimeout(function(){var a=document.createElement("script"); var b=document.getElementsByTagName("script")[0]; a.src=document.location.protocol+"//dnn506yrbagrg.cloudfront.net/pages/scripts/0014/4896.js?"+Math.floor(new Date().getTime()/3600000); a.async=true;a.type="text/javascript";b.parentNode.insertBefore(a,b)}, 1);</script> <script>jQuery(".page-search .pane-quick-search-results .search-snippet").text(function (_, old){ return old.replace(/\S+@\S+\.\S+/, "");});</script> <script type="text/javascript"> /* <![CDATA[ */ var google_conversion_id = 973817434; var google_custom_params = window.google_tag_params; var google_remarketing_only = true; /* ]]> */ </script> <script type="text/javascript" src="//www.googleadservices.com/pagead/conversion.js"> </script> <noscript> <div style="display:inline;"> <img height="1" width="1" style="border-style:none;" alt="" src="//googleads.g.doubleclick.net/pagead/viewthroughconversion/973817434/?value=0&guid=ON&script=0"/> </div> </noscript> <script async src="https://cdn.pbgrd.com/core-bmj.js"></script> </body> </html>