Toggle Main Menu Toggle Search

Open Access padlockePrints

Automated data analysis of unstructured grey literature in health research: A mapping review

Lookup NU author(s): Lena Schmidt, Saleh Mohamed, Dr Nick MeaderORCiD, Professor Jaume Bacardit, Professor Dawn CraigORCiD



This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


© 2023 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd. The amount of grey literature and ‘softer’ intelligence from social media or websites is vast. Given the long lead-times of producing high-quality peer-reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health-related grey literature and soft data, with a focus on (semi)automating horizon scans, health technology assessments (HTA), evidence maps, or other literature reviews. We searched six databases to cover both health- and computer-science literature. After deduplication, 10% of the search results were screened by two reviewers, the remainder was single-screened up to an estimated 95% sensitivity; screening was stopped early after screening an additional 1000 results with no new includes. All full texts were retrieved, screened, and extracted by a single reviewer and 10% were checked in duplicate. We included 84 papers covering automation for health-related social media, internet fora, news, patents, government agencies and charities, or trial registers. From each paper, we extracted data about important functionalities for users of the tool or method; information about the level of support and reliability; and about practical challenges and research gaps. Poor availability of code, data, and usable tools leads to low transparency regarding performance and duplication of work. Financial implications, scalability, integration into downstream workflows, and meaningful evaluations should be carefully planned before starting to develop a tool, given the vast amounts of data and opportunities those tools offer to expedite research.

Publication metadata

Author(s): Schmidt L, Mohamed S, Meader N, Bacardit J, Craig D

Publication type: Review

Publication status: Published

Journal: Research Synthesis Methods

Year: 2024

Volume: 15

Issue: 2

Pages: 178-197

Print publication date: 01/03/2024

Online publication date: 19/12/2023

Acceptance date: 22/11/2023

ISSN (print): 1759-2879

ISSN (electronic): 1759-2887

Publisher: John Wiley and Sons Ltd


DOI: 10.1002/jrsm.1692