Toggle Main Menu Toggle Search

Open Access padlockePrints

VEO IELTS Project Report: Which specific features of candidate talk do examiners orient to when taking scoring decisions?

Lookup NU author(s): Dr Müge Satar

Downloads


Licence

This is the final published version of a report that has been published in its final definitive form by British Council, 2021.

For re-use rights please refer to the publisher's terms and conditions.


Abstract

The research investigated which specific features of candidate talk IELTS Speaking Test (IST) examiners orient to when taking scoring decisions. We also researched whether the use of the scoring scheme and customized app potentially adds any value to IST examiner development.This was enabled by the development of an IST scoring scheme for an app (VEO) which creates a recording of when exactly in the test the examiners have noticed specific features of candidate talk and taken specific decisions on scoring as a result. Each of 4 IELTS examiners independently viewed 2 test videos using the app and scored it using the scoring tags. We then conducted individual stimulated recall interviews with the examiners involved.We found that the use of the customized IELTS scoring scheme and VEO app illuminates the IST rating process and potentially adds significant value to IST examiner development, specifically the re- certification process. When examiners assign higher scores, they focus on positive evidence, whereas with lower scores, they focus on negative evidence. Fluency & coherence scores are mostly assigned cumulatively. Grammar scores can be more easily tagged in relation to specific features. Examiners notice idioms and reward their use with a high mark, even if not delivered perfectly. Examiners form hypotheses as they listen to the candidate talk, then look for evidence that will confirm or reject these hypotheses. Examiners make scoring decisions in a cumulative way, rather than orientating towards single instances. Pronunciation issues may influence examiner decisions in relation to other criteria.


Publication metadata

Author(s): Seedhouse S, Satar M

Publication type: Report

Publication status: Published

Series Title: IELTS Research Reports Online Series

Year: 2021

Online publication date: 01/05/2021

Acceptance date: 31/03/2020

Report Number: 5

Institution: British Council

URL: https://www.ielts.org/for-researchers/research-reports/online-series-2021-5

Notes: Embargo on the authors' copy until the report is officially published. The publication will be freely accessible online and we will update the full text with the published copy when this is available.


Share