Now in open access from “Language Testing” is an excellent article about test score comparison tables by Ute Knoch and Jason Fan. It explores the concordance tables provided by the makers of the TOEFL, IELTS, PTE-A and C1 tests.
This is a wonderful area to explore in 2024 – most readers are probably aware that the topic of concordance tables has come up quite a lot in recent years as the English test market has become more competitive.
The article suggests that when it comes to creating score comparison tables, best practice principles are not always fulfilled. It notes:
“Our findings indicate that the information provided on the test provider websites about concordance tables is often vague or insufficient. Test users are not always provided with the research underpinning these concordance tables. When such research is provided, it tends not to fulfill the good practice principles and is usually presented in formats not easily accessible to non-specialist test users.”
And also:
“…the sample sizes are generally too small to provide robust score comparisons. Basic information is often not provided, such as concordance results for subsection scores (which are crucial for the requirements for Australian migration and other policy-makers), the number of observations at different score levels, and their standard errors. Test users are not usually informed about the potential limitations of using published concordance tables.”
But why? Well…
“At the moment, there is little motivation to invest in more robust concordance studies due to the absence of regulatory oversight and minimal demand for high-quality work from test users. It is also important to note that concordance tables are one site in which competition between test providers manifests, who may have a commercial interest in lowering their test scores to make it easier for applicants to achieve certain test score requirements.”
There is much more in the article. Do check it out.