This article by Daniel Lam and Angela Gayton is the best thing I’ve read all year. It explores gaps in knowledge about what test scores mean and how they can and ought to be used.
The authors interviewed personnel at a number of universities and illuminating excerpts are shared. I’d like to paste all of the quotes here, but I’ll limit myself to just a few.
Said one program director:
“If you say 6 and then somebody says 6.5, what is the actual difference, other than [that] 6.5 sounds a bit better, or higher, or more proficient if you like? […] So yeah, to me, it’s a fairly heuristic kind of process. People just come up with a number that’s either traditional that they’ve experienced or heard of before. ‘Oh, if this programme has this number, then maybe our programme should have this number.’”
And one EAP coordinator:
“I just find that sometimes it’s not consistent. So sometimes an IELTS 5 for one student, that student does not have the same standard or level of English as another one with an IELTS 5. So there are discrepancies, and I don’t know where that comes from – whether it comes from the assessor who assessed them, or there’s something in the guidelines that allows this sort of discrepancy.”
And:
“Sometimes the language level is not too bad, but because there is nothing behind, like in the thought processes, […] the student doesn’t perform well. And sometimes I think departments may mistake it for language inadequacy, but I don’t think it always is.”
A program director noted:
“I do think someone with a good IELTS score that comes here is a good user of English. But they may not be a good user of the particular type of discourse on which they will be assessed, which is a completely different matter. And I think this expectation or understanding is not there.”
One interviewee “laments” how an IELTS score can be used to make up for a failed assessment in a pre-sessional EAP program.
I could go on! The authors note that some of the comments “reflect the complexity of disentangling general language proficiency from academic literacy skills, and point to the need for a more nuanced understanding of this relationship, as well as, again, what language test scores are (un)able to indicate.”
In light of the recent “File on Four” report, there has been much discourse about how some students lack the language skills needed to excel in their studies. Much has been said about possible cheating. The folks behind IELTS seem to think that acceptance of a wider range of tests is problematic. But the issue is obviously more nuanced. All test scores are valuable, but they have certain shared limitations. Perhaps in 2025, when test prep is more sophisticated than ever and cool down periods between tests have been largely eliminated, yet more attention should be paid to the limitations.
Scroll down to the “discussion” section for tips that university staff should keep in mind.