Students often ask me how important it is to speak quickly in the TOEFL speaking section.  Keen students even ask how many words they should include.

I’ve always said that speaking rate is really important.  I’ve urged students to practice speaking quickly, as long as that doesn’t mess with their pronunciation or intonation.  But that’s always just been my gut feeling, stated without solid evidence. In an effort to gather some real data on the issue, I submitted a few of my own practice answers to SpeechRater, the automated scoring software used (together with human raters) by ETS to grade the TOEFL.  I was able to do this by uploading my answers to My Speaking Score, which has licensed SpeechRater.  I encourage both teachers and students to make use of that site.  It is comprehensive and fairly affordable.  A monthly subscription gives you a bunch of credits to upload answers (or if you prefer you can just record them in your browser).  Teacher and student accounts can be linked to facilitate reviews and personal feedback.

I must mention a few disclaimers before I get into the data:

  • The real test uses both a human rater and the SpeechRater.  That means we cannot use SpeechRater alone to completely predict how a given answer would score.
  • While My Speaking Score is meant to be as close to the real TOEFL as possible, it is a third party implementation, so it cannot be perfect.
  • This is a third party blog, not associated with ETS. My interpretation of the numbers below might be totally wrong.

My Sample Answer

Now that I’ve got that out of the way, you should listen to my sample answer:

 

Here’s a transcript:

I think it is much better to study various subjects while we are at university. This is because it helps us to find our true passion. For example, when I was a freshman, I took courses in chemistry, psychology and history. While I loved all of them, eventually I discovered that I was most interested in chemistry. As a result, I ended up majoring in that. Moreover, I gained interdisciplinary insights along the way.

My speaking rate is very low.  It is just 74 words (per the word counter in Google Docs).  I generally recommend about 120 words if students want a perfect score, and about 100 words if they want an “average” score.

However, everything else is pretty good.  My pronunciation and intonation are at a native level.  I’ve included a few fancy words like “passion” and “interdisciplinary” and “insights”.  I’ve also used transitions like “moreover” and “as a result.”  I even included a few conjunctions like “while” and “when.”  There aren’t any “umm” breaks, self-corrections or stutters. 

When I asked experienced TOEFL tutors how this answer would score, I got responses ranging from 24 points to a perfect 30 points.  As you can see, a few of the tutors really liked it!

Meanwhile, SpeechRater gave this one a score of 2.92/4.  That converts to 23 points out of 30.  Good… but not great.

What did SpeechRater Think about my Fluency?

First up, here’s the score for speaking rate.

Not surprisingly, the answer is all the way down in the 14th percentile.  That means I spoke slowly.  There is a penalty for that.  But speaking rate is just one metric.  It can’t account for all the entire seven-point penalty from the SpeechRater.  

Here’s a look at the rest of the fluency metrics:

They aren’t good either.  As you can see, the SpeechRater gave me a poor score for “sustained speech.”  It identified a bunch of disfluencies and I ended up in the 27th percentile.  In this case the disfluencies are silent pauses, but in other answers they might include “uhh” breaks.  It also gave me a fairly poor score for metrics specifically related to pauses, as you can see.  Slowness and pauses usually go hand in hand, as you might expect. 

What did SpeechRater Think of my Pronunciation?

My pronunciation scores were a mixed bag:

Despite my slowness, my rhythm was pretty good.  However, my pronunciation of vowels was merely average.  But how can that be?  I’m a native speaker.  Well, another source of slowness is the way I sometimes draw out my vowel sounds. Notice my pronunciation of “and history” and “all of them” and “most interested”.  The awkwardness is subtle, but noticeable if you are listening for it.  The penalty for doing this is likely small, but I think it added up since I did it multiple times in every sentence.

What did SpeechRater Think of my Vocabulary?

Vocabulary was another mixed bag:

As I mentioned above, I think my answer has a few good words in it.  However, I’m stuck in the 25th percentile for vocabulary depth.  And even though I didn’t really repeat words, my vocabulary diversity score is merely average.  Why?  Well, my guess is that since my total word count is quite low, it is almost impossible for me to include a lot of “uncommon” words.  I mentioned three words I suppose are “uncommon,” but that’s not really enough.  In a more quickly delivered answer I might have had time for seven or eight words, and earned a better score in that domain.  Likewise, a more quickly delivered would have almost automatically included a more diverse vocabulary… and earned a higher score.

A future experiment might involve jamming as many fancy words into an answer of the same length in an attempt to produce the best possible 75 word answer.

What did SpeechRater think of my Grammar?

Ooof.  My grammar score is not good:

I’m all the way down in the 9th percentile.  Again, this is despite the fact that my grammar is flawless.  Again, I think that the brevity of my answer means that I didn’t have the opportunity to use any advanced grammatical structures.  I have a couple of subordinating conjunctions, but that’s about it.  I don’t have any coordinating conjunctions, I don’t have any adverbs and I’m short on adjectives.  There are no conditionals in the answer, either.  Most of the answer is in the past tense. Some people might be able to fit a lot of grammatical conventions into just 75 words, but it isn’t easy. I think my limited use of grammar is common in answers that are delivered slowly.

What did SpeechRater think of my Coherence?

SpeechRater didn’t like my coherence either:

Again, my impression is that my answer was too short to include enough connective devices to please SpeechRater.  There are three obvious transitional phrases in my answer… but an answer with more words overall would naturally have more than that. Likewise, it would probably have a few compound sentences (my answer has none).

Final Words

The point I’m trying to make here is an obvious one, but it is important.  A TOEFL response delivered slowly may draw a low score from the SpeechRater.  In addition to being short overall, it will likely be missing some of the key features the SpeechRater wants to see.  Be careful on test day. 

As I indicated above, a future experiment will be to create the best possible 74 word answer, to see the best-case result for a slow answer.

Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments