New from Studies in Language Assessment is an article by Daniel Isbell, Dustin Crowther, Jieun Kim and Yoonseo Kim which looks at the speaking tasks included in British Council’s Aptis Test. Specifically, it compares official Aptis scores to assessments of intelligibility and comprehensibility from laypeople recruited from the “Prolific” online research participant pool. Scored responses from 50 test-takers were selected and those participants rated their comprehensibility on a 1-9 scale. For intelligibility, the participants were asked to transcribe the recordings. Those transcriptions were compared to criterion transcripts created by the research team. Among other things, the authors note that “to the ears of layperson listeners… speakers that earned higher Aptis scores were more intelligible and easier to understand.”
I would love to see more of this kind of research. I suppose a similar study could be done of test-taker responses to just about any English test.