Toggle Main Menu Toggle Search

Open Access padlockePrints

Hi-BEHRT: Hierarchical Transformer-Based Model for Accurate Prediction of Clinical Events Using Multimodal Longitudinal Electronic Health Records

Lookup NU author(s): Dr Dexter CanoyORCiD

Downloads


Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


Abstract

© 2022 IEEE. Electronic health records (EHR) represent a holistic overview of patients' trajectories. Their increasing availability has fueled new hopes to leverage them and develop accurate risk prediction models for a wide range of diseases. Given the complex interrelationships of medical records and patient outcomes, deep learning models have shown clear merits in achieving this goal. However, a key limitation of current study remains their capacity in processing long sequences, and long sequence modelling and its application in the context of healthcare and EHR remains unexplored. Capturing the whole history of medical encounters is expected to lead to more accurate predictions, but the inclusion of records collected for decades and from multiple resources can inevitably exceed the receptive field of the most existing deep learning architectures. This can result in missing crucial, long-term dependencies. To address this gap, we present Hi-BEHRT, a hierarchical Transformer-based model that can significantly expand the receptive field of Transformers and extract associations from much longer sequences. Using a multimodal large-scale linked longitudinal EHR, the Hi-BEHRT exceeds the state-of-the-art deep learning models 1% to 5% for area under the receiver operating characteristic (AUROC) curve and 1% to 8% for area under the precision recall (AUPRC) curve on average, and 2% to 8% (AUROC) and 2% to 11% (AUPRC) for patients with long medical history for 5-year heart failure, diabetes, chronic kidney disease, and stroke risk prediction. Additionally, because pretraining for hierarchical Transformer is not well-established, we provide an effective end-to-end contrastive pre-training strategy for Hi-BEHRT using EHR, improving its transferability on predicting clinical events with relatively small training dataset.


Publication metadata

Author(s): Li Y, Mamouei M, Salimi-Khorshidi G, Rao S, Hassaine A, Canoy D, Lukasiewicz T, Rahimi K

Publication type: Article

Publication status: Published

Journal: IEEE Journal of Biomedical and Health Informatics

Year: 2023

Volume: 27

Issue: 2

Pages: 1106-1117

Print publication date: 01/02/2023

Online publication date: 25/11/2022

Acceptance date: 20/11/2022

Date deposited: 01/03/2024

ISSN (print): 2168-2194

ISSN (electronic): 2168-2208

Publisher: IEEE

URL: https://doi.org/10.1109/JBHI.2022.3224727

DOI: 10.1109/JBHI.2022.3224727

PubMed id: 36427286


Altmetrics

Altmetrics provided by Altmetric


Funding

Funder referenceFunder name
British Heart Foundation
ES/P0110551/1
FS/PhD/21/29110
Oxford NIHR Biomedical Research Centre and the Oxford Martin School, University of Oxford
PG/18/65/33872
UKRI Global Challenges Research Fund

Share