The Duolingo English Test blog has a new feature on “jagged profiles” in language assessment. This refers to a language user who is quite strong at one or more aspects of the language and quite weak at some other aspect(s).
I was quite happy to read the following:
“Are jagged profiles common? In a word, yes! Because language assessments, and especially high stakes tests like the DET, usually evaluate multiple language skills at the same time including reading, listening, speaking, and writing, jagged profiles are often detected as a result of such tests. For example, a test taker might score high in reading comprehension but struggle with writing or speaking tasks, and consequently earn lower scores for those skills. This is a common scenario with test takers who have jagged profiles, because we have long known that production-based skills develop later than perception-based skills in L2 learning.”
Regular readers know that jagged profiles are one of the criteria used to justify the cancellation of scores on the TOEFL iBT Test. ETS might argue that jagged profiles do not result in cancellation all by themselves, but only in combination with other factors. That said, I’ve long called for jagged profiles to be removed from the equation entirely. I’ve voiced that opinion more loudly since the shortening of the TOEFL iBT a few months ago, as the removal of variable (unscored) questions reduced the amount of data available to officials in the Office of Testing Integrity who are making these sorts of decisions.
Before the formal appeals process for such cancellations was removed, affected test-takers often reached out to me for assistance in planning their appeals. Usually, that involved helping those test-takers explain the reason for their jagged profile. I was occasionally successful in having cancellations overturned. But usually not. You may recall the story I related last month about an autistic test-taker having his score cancelled (without a refund) in part because of his low speaking score. I reached out to him this week for an update, and was sad to hear that his scores have not been reinstated.