Toggle Main Menu Toggle Search

Open Access padlockePrints

BEHRT: Transformer for Electronic Health Records

Lookup NU author(s): Dr Ali HassaineORCiD, Dr Dexter CanoyORCiD



This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


© 2020, The Author(s).Today, despite decades of developments in medicine and the growing interest in precision healthcare, vast majority of diagnoses happen once patients begin to show noticeable signs of illness. Early indication and detection of diseases, however, can provide patients and carers with the chance of early intervention, better disease management, and efficient allocation of healthcare resources. The latest developments in machine learning (including deep learning) provides a great opportunity to address this unmet need. In this study, we introduce BEHRT: A deep neural sequence transduction model for electronic health records (EHR), capable of simultaneously predicting the likelihood of 301 conditions in one’s future visits. When trained and evaluated on the data from nearly 1.6 million individuals, BEHRT shows a striking improvement of 8.0–13.2% (in terms of average precision scores for different tasks), over the existing state-of-the-art deep EHR models. In addition to its scalability and superior accuracy, BEHRT enables personalised interpretation of its predictions; its flexible architecture enables it to incorporate multiple heterogeneous concepts (e.g., diagnosis, medication, measurements, and more) to further improve the accuracy of its predictions; its (pre-)training results in disease and patient representations can be useful for future studies (i.e., transfer learning).

Publication metadata

Author(s): Li Y, Rao S, Solares JRA, Hassaine A, Ramakrishnan R, Canoy D, Zhu Y, Rahimi K, Salimi-Khorshidi G

Publication type: Article

Publication status: Published

Journal: Scientific Reports

Year: 2020

Volume: 10

Issue: 1

Online publication date: 28/04/2020

Acceptance date: 11/03/2020

Date deposited: 25/11/2022

ISSN (electronic): 2045-2322

Publisher: Nature Research


DOI: 10.1038/s41598-020-62922-y

PubMed id: 32346050


Altmetrics provided by Altmetric


Funder referenceFunder name
British Heart Foundation (BHF),
National Institute for Health Research (NIHR) Oxford Biomedical Research Centre,
Oxford Martin School (OMS)
UKRI’s Global Challenge Research Fund (GCRF).
University of Oxford