A new article in British newspaper i is critical of the Duolingo English Test. It reports that:
“Professor John Heathershaw from the University of Exeter linked the acceptance of ‘things like Duolingo tests’ to lower English language standards and described it as a ‘major issue’ when he appeared before the Foreign Affairs Select Committee last week.”
And:
“Aston University in Birmingham is one of the many UK institutions that adopted Duolingo at the height of the pandemic but has now dropped it. A spokesperson told i that the decision was taken because of concerns about student performance.”
And quoting a spokesman from that university:
“There is no evidence that those students on Duolingo were failing or, indeed, fraudulent, but just not performing at the same level as peers on their course, so we have chosen to remove acceptance for the test.”
–
I’m going to babble a bit now. Pardon me.
Older tests like the TOEFL and IELTS are designed around a particular conception of validity. For instance, the makers of the TOEFL would likely argue that we can look at a TOEFL score and make inferences about the test-taker’s ability to do well in university classes because the tasks on the TOEFL resemble (and, more importantly, involve the same knowledge, skills and processes) as the tasks done by students at post-secondary institutions.
Similarly, IDP would tell you that the IELTS General Training Module is a great test for immigration because it includes tasks that resemble what we do in everyday life. Meanwhile, the TOEIC leans heavily on memos and invoices and emails that people encounter in an office environment.
You can read about this concept of validity in Carol Chapelle’s 2008 doorstopper about the creation of TOEFL iBT.
The Duolingo English Test is a little bit different. It is certainly a test of one’s English abilities. That much is obvious. But is it appropriate for university admissions? While questions of an academic nature have been added to the DET in recent months much of the test score is still determined by “describe this picture” tasks and “fill in the missing letters” tasks that don’t closely resemble things done on a university campus.
Duolingo might argue that such tasks totally suit the purpose of the test and that they really do require the relevant skills and knowledge. They could be right. Who knows?
What interests me is that if we reject the idea that validity requires that test items closely resemble tasks performed in real-world contexts we can go ahead and discard all of the older tests and use the Duolingo Test for all possible purposes. Will the receiving institutions bite? That remains to be seen.
