I watched this long webinar from Times Higher Education and Oxford University Press about “rethinking language assessment for a changing world.” It includes a wonderful exchange at around 23:00 about the use of “proxy tasks” to predict success in academic programs. Around that point, Hannah Jones talks about the need for clear and accessible peer-reviewed research into the predictive validity of these sorts of tasks.
To me, that’s a really fair point, as we sort of take it for granted that certain tasks are more predictive than others. But are they really? I’ve written here about how tasks on certain tests sort of LOOK better for various reasons (perhaps because they are quite long)… but a closer examination suggests that surface-level observations aren’t particularly useful.
Tony Green, meanwhile, talked about how all English tests don’t have very much predictive validity at all and that “looking to predictive validity isn’t a very productive route to go down.” He mentioned that other measures, like A-Levels, have even less predictive power than these tests.
Again, this makes a lot of sense because success in university depends on a whole lot of things which are not measured on an English test. And, of course, these things are often acquired and honed sometime after our admission. I, for one, earned my highest GPA only in my final year of studies. Needless to say, it took me some time to get the hang of university-level work and, for that matter, the academic register.
I also liked this bit from Tony:
“…a 250-word essay is not actually what people have to write when they get to university. And I think we could do a good deal more in helping [them] to understand that it’s not what they need to do when they get to university. And there’s a big gap between those things.”
That’s a welcome perspective from someone working on a test. I say this because in recent years some traditional test makers have, perhaps, been overselling their products in response to competition from newer providers.
So what does this all mean when it comes to the development and use of tests? That’s the “rethinking” part and you should probably watch the webinar to learn more about it.