To learn more about the impact of word count on automated scoring of “writing for academic discussion” prompts that will be included on the TOEFL on July 26, I spent some time answering the sample questions provided by ETS. This sort of experimentation is mildly important, I think, as many test-takers (and tutors) hold the idea that only really long responses get high scores on the current TOEFL. This sometimes results in the creation of monster-sized TOEFL essays.
What did I learn?
Happily, the automated scoring system (e-rater) gave me a perfect score of 5.0 for the following 102-word response to sample question one (about how to repopulate the countryside):
“While I appreciate the solutions presented by Claire and Kelly, I feel that the only way to truly solve this problem is to construct better schools. Parents want their children to be educated at the best possible facilities, but it is almost impossible to find impressive schools in the countryside. Although the government currently provides funding for amenities like science laboratories and lavish libraries in cities, they neglect rural areas. Rural schools, like the one I attended, lack even basic educational supplies like computers and sports equipment. Consequently, parents who are concerned about their kids head for greener pastures, so to speak.”
The question prompt recommends writing about 100 words, so I’m happy. Students can confidently follow the given instructions, I guess.
I fiddled about with a series of shorter (but similar) answers and was able to get a score of 4.0 for the following 49-word response:
“While I appreciate your ideas, I think we need better schools. Parents want their children to utilize excellent facilities, but it’s impossible to find impressive schools in the countryside. Although the government currently funds amenities like libraries in cities, they neglect rural areas. Consequently, parents head for greener pastures.”
That’s as low as I could go and still get a score of 4.0.
But I could get a score of 3.0 for the following 32-word response:
“While I appreciate your suggestions, we need better schools. Children need excellent facilities, but the countryside lacks them. Although the government funds academics in cities, they neglect rural areas. Consequently, parents leave.”
Anything lower than that resulted in a score of 2.0 or less.
A few things are worth mentioning:
- The automated score will be combined with a human score on the real test.
- Obviously word count correlates with other features like range of vocabulary and number of grammatical features.
- The above scores encompass a whole range of scores once they are scaled up. A score of 4.0 from the e-rater could scale up to anything from 21 to 26. My sample would likely be on the low end of this range. It would be nice to get decimals from the ETS website.
You can find a record of everything I submitted along with the e-rater scores over here.