Toggle Main Menu Toggle Search

Open Access padlockePrints

The Rashomon Effect: Which features of a speaker’s talk do listeners notice?

Lookup NU author(s): Professor Paul SeedhouseORCiD, Dr Müge SatarORCiD



This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND).


The same L2 speaking performance may be analysed and evaluated in very different ways by different teachers or raters. We present a new, technology-assisted research design which opens up to investigation the trajectories of convergence and divergence between raters. We tracked and recorded what different raters noticed when, whilst grading a speaker’s videoed talk in the IELTS Speaking Test. We devised graphs of the ‘noticing trajectories’ of raters to compare the relative degrees of convergence and divergence between all raters at any point. The possible reasons for convergence/divergence were elicited using stimulated recall interviews. We found that raters’ noticing and rating tended to converge in relation to easily identifiable features such as idioms. However, for some stretches of speaker talk, there was clear divergence in terms of features noticed and rated by raters; in real IELTS tests, a background score-monitoring system deals with divergence. The research design enables delicate depiction of how the process of rater convergence/divergence unfolds over time. Applications are suggested in terms of examiner training, as well as for classroom discourse research and CA data workshops.

Publication metadata

Author(s): Seedhouse P, Satar M

Publication type: Article

Publication status: Published

Journal: Classroom Discourse

Year: 2023

Volume: 14

Issue: 1

Pages: 1-23

Print publication date: 01/03/2023

Online publication date: 21/06/2021

Acceptance date: 10/05/2021

Date deposited: 05/05/2021

ISSN (print): 1946-3014

ISSN (electronic): 1946-3022

Publisher: Taylor and Francis


DOI: 10.1080/19463014.2021.1923540


Altmetrics provided by Altmetric


Funder referenceFunder name
British Council
IELTS-Funded Research Programme