It looks like the Educational Testing Service has reconsidered a social media campaign touting the “low score cancellation rate” of the newly enhanced TOEFL Home Edition.  It seems that it disappeared a few hours after I wrote about it on LinkedIn.  That’s probably for the best since the campaign was clunkily implemented.  I won’t reproduce images from the campaign here, but you might be able to see one in this Facebook posting.

That said, score cancellation is an issue that ETS will have to keep in mind as they move forward with enhancements to the TOEFL.  It’s no secret that the TOEFL Home Edition has a bit of a “social media problem.”  When potential test takers raise the possibility of taking the Home Edition, others are quick to chime in and warn them about this very issue.  I’ve written many times about test takers sharing stories about curious score and test cancellations – cancellations for having a jagged score profile,* cancellations for utilizing too much RAM during the test, cancellations for unauthorized software being detected immediately after successfully passing a system scan… and others.

Five years in, this remains a pretty big issue that I believe has a measurable impact on the number of people who opt to take the TOEFL Home Edition.  Until yesterday I wasn’t sure that anyone in a leadership position over at ETS truly realized how big that impact was.

Here’s a link to ETS’s Trustpilot page, where many of the comments there touch on this issue (across the whole range of ETS products).  With a rating of 1.1 out of 5 they’ve somehow managed to be more disliked than the folks at Wells Fargo and even American Airlines.

I’ve written again and again about how legacy test makers seem increasingly disconnected from the people who consume their tests.**  Cleary, this issue could have been tackled head on before now.   This is the fourth major revision to the test in six years*** and I don’t think there will be time and money for a fifth revision if this one flops. Success or failure will come down to communicating with customers, understanding customers and, as we say, “meeting them where they are.”

So kudos to whoever came up with the campaign.  Just… maybe work on the wording a bit.

*which still happens, according to recent social media posts.

**the Curse of the MBA?

***the enhancements of 2019, the introduction of at-home testing in 2020, the enhancements of 2023, the enhancements of 2025/26

I’m going to host another “Office Hours” chat on June 25.  The topic will be, once again, upcoming revisions to the TOEFL iBT.  You can register right here.

Note these chats are mostly intended for people who work in test preparation, but everyone is welcome.  Last time we had a fun mix of teachers, test takers and testing firm reps.

And, of course, if anyone from ETS wants to pop in for a few minutes to talk about the revisions I know you’ll have an apt and appreciative audience.

With that out of the way, a few specific thoughtlines come to mind:

1. We should probably unpack the latest news about the revisions.
2. What are the implications for people working in the test prep space?  This is starting to look like a “page one rewrite” when it comes to the prep materials we use.  Will we be, at least temporarily, a bit more dependent on official products than usual?
3. This is starting to look like a bigger revision than originally thought.  How do you figure test takers and student advisors in YOUR market will respond to it?
4. What about the scoring of the test?  Are we looking at a sort of “black box” like many other major tests? How will that impact our work?
5. What about score equivalencies between the old and new version?

In terms of an ever-elusive bigger picture, I’m interested in what everyone’s thoughts are about the best way to be a test prep professional in a world where tests are changing, market shares are shifting and test takers have more options than ever before.

Since this is a Zoom meeting rather than a webinar I can’t really send reminders, so I recommend that you make a reminder in your calendar app of choice. And don’t ask me for a recording after the fact – I don’t record these events since I want everyone to feel comfortable expressing their honest opinions.

(yeah, I had intended to discuss some other test this time… but…)

According to the Financial Times, there has been a “double-digit drop” in applications for the TOEFL in the wake of the Trump administration’s moves regarding international students. Their source is ETS head Amit Sevak.

Says the FT:

“Amit Sevak, head of ETS, which runs the largest English language test for foreign students applying for universities in the US, told the Financial Times there had been an industry-wide double-digit drop in the number of applications for the different tests used.”

Last week’s “Office Hours” were a big success.  The turnout was pretty good, and I didn’t have to do much talking.  The group included a nice mix of test prep folks, test takers and a few testing firm insiders.

As a group, we chatted about a few stimulating topics, including:

1. What it means for a test to have a “multistage adaptive design” and how that might be implemented in the TOEFL reading and listening sections.  One participant seemed pretty sure that the TOEFL reading section could be enhanced with an IELTS style “table completion” task.  I’m not so certain about that will come to pass, even though I think it would be a pretty great change.
2. How the promise of tasks that “better reflect how students use English in real academic settings like group discussions and project work” might reflect a desire to widen the “target domain” of the test.  The test already includes items centered on campus interactions, but they are mostly simulations of service encounters and meetings with professors.
3. What’s going on in China.  A few test prep pros popped into the meeting and shared an update on the scene in that country.  This is always super valuable because even though China is the number one market for English tests, it is barely discussed on this particular platform.
4. How nice it would be to have more speaking tasks on the test.  The TOEFL currently includes just 3 minutes and 45 seconds of oral production, which must be the least of all the major tests (and the minor ones, too).
5. The sorts of test prep materials people favor nowadays.
6. How tests could be more personalized for specific test takers.

I recommended a couple of articles in “Research for Practice Issues and Solutions in Computerized Multistage Testing” by Yan, von Davier and Weiss for anyone who wants to dive into the theories behind multistage testing before January.

I’ll do my best to host another chat early next month.  A few possible areas of focus come to mind:

1. More TOEFL!
2. IELTS:  What’s up?
3. The Pearson English Express Test:  What is it?
4. DET:  What’s changing?  What’s going on?

I might need some assistance with topics 2 through 4.  Do let me know if you are an expert and would like to take part, and help generate some conversation. I don’t record the meetings, so everyone is able to speak frankly and casually.

The Australian Home Office’s page for the language testing REOI has been revised.  Most everything has been removed and it now provides just a brief summary of the still-ongoing process.

As for the future, it now states:

“We may accept new English language tests that satisfy all REOI requirements as an outcome of the REOI process. These tests may replace those currently accepted. We may also prescribe them in Australian migration legislation by mid-2025.”

May accept… may replace… may prescribe…

I assume that the list of acceptable language tests will still be revised based on the results of the REOI process (and pretty soon), but the whole thing seems a bit more wishy-washy than it was a week ago.

If you are looking for the old documentation, you can probably find it via the Wayback Machine.