Browse by author
Lookup NU author(s): Emeritus Professor John Spencer
Full text for this publication is not currently held within this repository. Alternative links are provided below where available.
Aim: Compare how different researchers performed in screening for informative evidence about medical education. Method: Six researchers with three different levels of involvement in a systematic literature review screened articles by title and (where available) abstract, and then by reading articles they had selected in full text. The reference standard was a consensus decision to include or exclude the article in the final analysis, whose results are published elsewhere. Results: The single screener most involved in the literatu research, who was also the most junior member of the topic review group, achieved a sensitivity approaching 100% and a specificity of 98-100% for informative articles. She far outperformed the other researchers, all of whom had as much or more topic knowledge and greater research experience. Conclusion: It was not possible to improve on the performance of the single motivated and capable primary screener and trying to do so increased the number of uninformative articles retrieved. One interpretation is that the primary searcher was more practiced and focused on the task than her more senior colleagues, yet they tended to become worse rather than better with practice. The fact that a well informed but relatively na-ve person consistently outperformed her more "expert" colleagues might suggest an alternative explanation: Given the patchy standard and qualitative nature of the evidence, perhaps experts found it harder than a novice to make reliable choices, in which case their unreliable performance reflects the nature of present day education evidence. This case study illustrates the value of quality assuring the article selection process. Given the amount of disagreement uncovered by the study, we suggest that consensus between reviewers is an important reference standard against which the performance of any single primary screener should be checked.
Author(s): Dornan T, Littlewood S, Margolis SA, Ypinazar V, Scherpbier A, Spencer J
Publication type: Article
Publication status: Published
Journal: Medical Teacher
Year: 2007
Volume: 29
Issue: 4
Pages: e72-e75
Print publication date: 01/05/2007
ISSN (print): 0142-159X
ISSN (electronic): 1466-187X
Publisher: Informa Healthcare
URL: http://dx.doi.org/10.1080/01421590601045015
DOI: 10.1080/01421590601045015
PubMed id: 17786735
Altmetrics provided by Altmetric
