New this month from John Norris and Larry Davis is a detailed comparison of the old TOEFL Independent Writing Task and the new TOEFL Writing for an Academic Discussion Task.

It notes that among test-takers who completed both tasks in operational settings, 50% received the same score (from 0 to 5) on each from a single human rater. 47% received a score that was +/- one point on the same scale.

Furthermore, the article notes: “We saw no difference in terms of the measures of cohesion that we evaluated, and overall very few differences in terms of specific measures of syntactic complexity, grammaticality and mechanics, or word use.”

Some differences were noted, though. According to the article:

“A few linguistic measures differed across tasks in a manner that may suggest a slightly greater orientation toward academic register in the IND writing task. These measures included slightly greater use of academic vocabulary, as well as somewhat longer noun phrases and clauses, features typical of academic writing. On the other hand, responses to the WAD task showed marginally higher lexical density (relative frequency of content words) and somewhat fewer word usage errors, both of which may be associated with shorter responses. “

Also, the Writing for an Academic Discussion task elicited more writing per unit of time.

It is worth noting that e-rater scores were not available for most of the WAD responses studied in this report as automated writing scoring was only added to the TOEFL Essentials Test in late 2022. I’d like to see more research into how human scores for WAD tasks compare to e-rater scores for the same.Also: given the change to the test, it may be a good time for a follow up the research done by Brent Bridgeman, Catherine Trapani and Yigal Attali in 2012 about the possibility that machine scores can differ (in terms of their closeness to human scores) for certain gender, ethnic and country groups.

My friend Pamela Sharpe, author of Barron’s TOEFL, is once again running the Barron’s TOEFL Scholarship Competition.  Dr. Sharpe has been running this scholarship for quite a long time (decades?) and every year helps a selection of students pay to take the TOEFL test.  Last year she helped 12 students pay their fees.

Note that this year the scholarship is open only to test-takers currently within the United States.  Here are the details:

Purpose: Scholarships are awarded by Dr. Pamela Sharpe, the author of Barron’s TOEFL preparation books, to assist with TOEFL registration fees.

Qualifications: Students from all countries who are studying English in the United States and preparing to take the TOEFL.

Amount: Full Scholarships: $200 U.S. for TOEFL registration fees. Partial Scholarships: $100 U.S.

Application: Submit a one-page essay in English on the following topic: What is the best way to prepare for the TOEFL? Include at least three suggestions. Please do not write multiple pages. The scholarship committee will only read the first page.

Required Information: At the bottom of your essay, include the following information:
Your name, your email, and your address in the United States. We regret that no essays will be considered if this information is not available.

Payment: Successful scholarship recipients must be able to receive their $200 or $100 awards in US dollars by check. Please do not request alternative payments.

Deadline: Submit the essay with your application information to [omitted]. Make the submission before September 30 to be considered for one of the annual scholarships. Awards will be made in October.

Now in Open Access at “Language Testing” is a summary of the enquiry (aka score review) and resit policies of all our favorite language tests (Cambridge English Qualifications, Duolingo, IELTS, LanguageCert, PTE, PSI Skills for English, TOEFL, Trinity).

It was done by William S. Pearson of the University of Exeter.

Really useful information to have.

If you are curious about how happy the language testing industry is about changes to the SDS system in Canada, consider the following:

  1. Canada issued 590,570 study permits in 2022. The number of study permits issued increased year over year by 5% in Q1 of 2023 and a whopping 35% in Q2 of 2023.
  2. Immigration minister Marc Miller estimated today that the total number of study permit holders in Canada will hit “about 900,000” in 2023. Compare that to the 948,000 international students currently studying in the USA, according to the most recent Open Doors Statistics.

Almost everyone applying for study permits has access to the SDS system at this point.

My point here is that it’s a pretty big deal for everyone involved in language testing that SDS applicants can pick from a wide range of language tests, and are no longer forced to use IELTS scores. Other language tests certainly have opportunities to grow their market share as a result of this change.

(data source)

The Cheat Sheet” has covered the Pearson cheating problem. Forgive me, but I can’t resist quoting at length. It says:

…according to reports, apparent cheating on the PTE Academic Online Test, a test of English, has caused the test provider to cancel some scores, in turn causing some U.K. universities to hold or delay acceptance offers to foreign applicants.

According to the coverage, cheating was limited to the online versions of the test, delivered in China, by test provider Pearson. After an inquiry, the scores were canceled and some schools removed the test as proof of English competency.

So, two things.

The cheating was during the online test. The news coverage says:

The organisation’s in-person tests are unaffected.

Surprise, surprise.

And, though this is not exceptionally clear in the reporting, it does appear that the cheating was uncovered by the schools – not the test provider. The schools, the paper says, noticed a high number of prospects, ‘applying with full or very high marks.’

So, again – I just do not understand what test providers are doing if they’re not safeguarding their own exams.

But this is a good example of what happens when you don’t. First, people cheat. Second, it becomes public. Third, people stop using your test. Really pretty simple.

Before we move on, this issue apparently impacted hundreds of students. Hundreds. So much so that the coverage says:

Pearson has now stopped delivering the online test in China

Boy, there’s a show of confidence.”

Back to me:

While it is true that university administrators don’t pay a lot of attention to language testing (witness the number of schools that still list score requirements for, say, the TOEFL CBT), this particular substack is widely read by decision makers. Testing companies need to do better if they want their at-home tests to be widely accepted. At this rate they’ll never reach the holy grail of getting an at-home test accepted for immigration purposes.