Dan Isbell has written a guest post about washback in English test preparation for the Duolingo English Test blog. It discusses preparation for the DET in particular, and for English tests in general. Isbell divides test prep into three types: activities that improve your English in general, activities that help you perform better on a particular test (test familiarization), and activities that help you game a particular test (templates and guessing strategies, for instance).
I understand that a full report on this topic is forthcoming. I will add a link when it is available.
It’s an interesting thing to explore. Test preparation always includes at least some good washback. No matter what test they are preparing for, most test takers complete at least a few practice tests. As a result, they will spend time consuming stuff in English and producing stuff in English. This is good. But does it have a really meaningful impact on their fluency in the language? I don’t know.
The TOEFL iBT contains two 800-word articles which are excerpted from actual textbooks. Students here in Korea take their preparation pretty seriously and might complete 20 or 30 practice tests before test day (or between several test days). That means they spend a lot of time reading some pretty dense material in English. Does that improve their fluency? Of course. Does it improve their fluency a lot? I don’t know.
My test prep niche is writing.* It gives me great joy to know that my students walk away from their lessons with a noticeably stronger command of English grammar and language use conventions. But, needless to say, there are faster and more economical ways to learn about sentence fragments and collocations.
Does all of this test prep mean that students spend less time on more useful and effective language acquisition approaches? Maybe.
Is it the job of a test maker to give a darn? Or is their only job to accurately measure language fluency? I don’t know.
A few stray thoughts come to mind:
- I’m interested to know how the age of a test impacts the way that students prepare for it. As a test ages, people working in test prep become more and more familiar with the design of that test and can use that knowledge to develop better and extremely granular type 2 strategies. Elderly readers might recall that in the early years of the TOEFL iBT we had just one official book (badly written) and a handful of books from third party publishers (even worse) to go by. We didn’t know very much about the design specifications of test items, nor about how speaking and writing items were scored. Things are obviously much different now. Now we know almost everything there is to know. We know so much nowadays that it might be malpractice to not spend quite a lot of time on test familiarization strategies. Should tests be meaningfully refreshed on a regular basis to mitigate the impact of this factor?
- I love reading about the early history of the Princeton Review. That firm emerged in the early 1980s when the SAT was long in the tooth and probably at its peak terribleness. Princeton Review taught students how to eliminate answer choices without actually reading questions. They also taught students how to recognize unscored sections so they could enjoy a refreshing nap part way through the test.
- In 2019 Malcolm Gladwell and his assistant both took the LSAT for an episode of his “Revisionist History” podcast. They got coaching from the one and only John Katzman beforehand. The point of the episode is that time management (type 2) is the most important thing when it comes to getting a good score on this test. It made LSAT tutors really cranky.
*See also: “The Whale,” 2022.