ELT Journal has published Duolingo’s response to last year’s article by Bruce, et al. about new online English tests and their use in UK admissions. Regular readers will surely recall the article, as I have referred to it here several times. It has also been mentioned in the higher-ed press many times. In short, the study suggested that the use of these tests has resulted in the admission of students with poor English abilities.
The authors of the response suggest that the study contains “several substantial flaws that potentially invalidate [its] conclusions.”
To break it down, their main concerns are:
- The study draws conclusions based solely on the opinions of university staff. But it does not include performance data like student marks and progression rates to contextualize these perceptions.
- EAP professionals make up the largest respondent subgroup, “yet they typically work only with students requiring language support, potentially excluding perspectives on higher-proficiency matriculants.”
- While criticisms were leveled against the Duolingo English Test and other new tests, such tests were used by only a handful of the schools represented in the study. This suggests that the students with poor English skills referred to by respondents were actually admitted with IELTS scores.
- I’ll just leave a quote for this one: “Low-quality research, including studies in which the methods and results do not support the conclusions, is potentially detrimental to both future research efforts and society.”
- Several of the authors of the study are affiliated with the owners of the test (IELTS) which was most praised in the study. It is suggested that this was not adequately disclosed.