I really like this article from Cambridge about “Ethical AI for language learning and assessment.” If I were trying to position the IELTS to thrive in today’s brave new world, this is the sort of message I would use. I would express that the test is keeping up with the times, but that the makers of the test take a more cautious and thoughtful approach than its competitors. Sounds easy, right? Perhaps not. Monday I’ll write about some not-helpful articles out of Cambridge that will get in front of many more sets of eyeballs in the days ahead.

This month I read the March 21, 2024 issue of the London Review of books.  A few articles caught my attention

I Adjure You Egg” is a short history of magical amulets and charms in medieval Europe.  A really fun exploration of a corner of European history we don’t often hear about.  People believed in such charms with a lot of sincerity, for a very long time.   It was interesting to know that charms were often scraps of written words rather than some object with more physicality, which is what we often picture when we think of charms.

Llamas, Pizzas, Mandolins” is a long look at AI and its implications.  Yeah, you are probably sick of this topic, but it is one that many still want to read about.  And I bet it will be asked about on the TOEFL for some time to come.

After that, I read “Assessing Academic English for Higher Education” by John M. Norris and Xiaoming Xi of ETS.  As I think I mentioned in an earlier column I am working my way through all of the books on my shelf (about tests or otherwise) that I have actually paid for with my own money.  This one has been on my shelf for a couple of years at least.  This was the perfect time to read it, as it serves as a sort of coda for the “old ETS” before the TOEFL gets blown up into something unrecognizable and mostly unconnected from what came before.  The book is almost sorrowful, as it describes page after page of good research and good ideas about assessing English for academic purposes that never made it into the TOEFL iBT.  Instead of using these ideas, ETS just sort of left the test in its original form, basically unchanged for two decades.  It is no wonder that so many bright minds left to build the Duolingo English Test.  

Finally, I read Richard Sennett’s “The Corrosion of Character,” a short book (really an essay) about work at the turn of this century.  There is some stuff in here that ETSers might appreciate too, especially about the futility of companies trying to reform into something new by “reengineering” themselves.  I was going to quote a relevant part of the book, but it was too long.  So here’s a dramatic reading:

 

This part may hit home:

“…institutions become dysfunctional during the people-squeezing process: business plans are discarded and revised, expected benefits turn out to be ephemeral; the organization loses direction. Institutional changes, instead of following the path of a guided arrow, head in different and often conflicting directions: a profitable operating unit is suddently sold, for example, yet a few years later the parent company tries to get back into the business in which it knew how to make money before it sought to reinvent itself.”

I really like this article from Cambridge about “Ethical AI for language learning and assessment.” If I were trying to position the IELTS to thrive in today’s brave new world, this is the sort of message I would use. I would express that the test is keeping up with the times, but that the makers of the test take a more cautious and thoughtful approach than its competitors. Sounds easy, right? Perhaps not. In a few days I’ll write about some not-helpful articles out of Cambridge that will get in front of many more sets of eyeballs in the days ahead.

ETS’s 990 form for the year ending September 2024 is now available via Propublica.  Some of this information was available in the audit published earlier this year, but much of it is new.  A few things are worth mentioning:

  1. Program service revenues were 908 million dollars.  That’s down 10  million from 2023.  It is the lowest in 15 years, as far as I can tell.
  2. ETS finished the year in the black with a net income of 22 million dollars.
  3. But that’s partially because they sold 95 million dollars of publicly traded securities. That’s the biggest sell-off of assets since they unloaded Prometric in 2018.

I’ll pause for a moment to point out that, to me, this seems like an organization that probably needs to stop losing money.  Cash on hand dropped from 211 million dollars to 46 million during the year. Investments in publicly traded securities dropped from 109 million dollars to, apparently, zero.  There isn’t much left to sell in order to cover for losses incurred while running ETS programs.  They’ve still got a lot of non-publicly traded securities… but those are actual companies or portions of companies.

Anyways, back to the bullet points:

  1. Salaries paid to key executives roughly doubled to 22 million dollars for the year.
  2. CEO Amit Sevak saw his pay more than double, reaching 1.7 million dollars for the year.  That included an incentive based bonus of about 500 thousand dollars.
  3. Two other current executives crossed the million dollar mark.
  4. Three former executives were paid more than a million for the year.
  5. The biggest of the buyouts are listed this year.  They are pretty big.

I noticed that the top 5 contractors were paid 93 million dollars.  The top contractor was CAPGEMINI AMERICA INC.  That’s new. I don’t know what they do.

Back to it:

  1. PSI Services LLC (Domestic) is on the books as “ETS Digital, LLC.”  It is listed as being worth 553 million dollars.  Apparently it lost money during the year.
  2. PSI (Domestic) was given a loan or loan guarantee by ETS to the tune of 223 million dollars.
  3. Here’s a puzzler: there is a thing on the books called EDUCATIONAL TESTING SERVICE DOMESTIC HOLDINGS INC.  This has been around for some time.  It is a C corp.  Usually it is valued at a few million dollars.  Now it is valued at 502 million dollars.  What does it contain?  I don’t know.  All of the existing subsidiaries (Vericant, Kira, Pipplet, etc) are listed separately, so it isn’t them.  I suppose it consists of investments made by ETS Capital (Applyboard, etc) and other securities.  Maybe they actually do have stuff left to sell.

We’ll get a quick look at the year ending September 2025 when the next audit is published (probably in January).  I pray that I am on vacation when that happens.

The National Association of Boards of Pharmacy (NABP) has convened a task force to determine whether foreign trained pharmacists should still be required to submit TOEFL scores as part of their credentialing process.

The resolution establishing the task force notes that “questions have been raised as to whether the TOEFL iBT effectively measures English proficiency in reading, listening, speaking, and writing for this purpose.”  And that “concerns have been raised regarding whether this requirement may be a potential barrier to licensure.”

The task force will meet in November. 

Regular readers know that credentialing of foreign trained pharmacists is one of the most lucrative on-shore use cases of the TOEFL iBT.  The TOEFL is currently the only test accepted for this purpose and, as I’ve discussed here, it is common for would-be pharmacists to take the test 10, 20, 30 or even 50 times. ETS has enjoyed this lucrative monopoly since at least 2008, according to my research. Its loss would be significant.  And, not for nothing, it would be catastrophic for the online TOEFL prep community.

Some* have suggested that ETS has taken this use case for granted over the years.

The resolution establishing the task force was passed at the end of May, so it isn’t a response to recently announced changes to the test.  But it comes at a bad time for ETS.  The NABP will be forced to examine a wholly different test than what they originally signed up for. And officials will surely be aware that back in 2024 their regulatory counterparts in Canada stopped recommending that provincial pharmacy boards accept TOEFL scores.

Given the importance of this use case, I am quite certain that alarms sounded across Lawrence Township the moment the task force was announced. I’m sure that the TOEFL product owner scrambled a team of B2B pros to ensure that his test remains acceptable to the NABP for many years to come.  But maybe I’m wrong.  In any case, you know how to reach me if you need a consult.  

It is safe to assume that other test makers took notice of the task force.  The folks at Michigan Language Assessment, for instance, have long been focused on English testing for medical professionals and have plenty of experience wrangling state licensing boards. Their flagship product has a vague resemblance to the outgoing TOEFL iBT which the NABP has been devoted to for so many years, but without the little annoyances which make some people think of it as a “barrier to licensure.”  I wouldn’t be surprised to hear that our friends in Ann Arbor responded to this development even more quickly than ETS did.

*Just me, really.

I attended two recent online events hosted by ETS. At an ETS EMEA event, I learned that:

  1. The new Official Guide to the TOEFL will be published in January.
  2. Questions in the reading section of the new TOEFL are not all weighted equally. Some are worth more than others.

At the ETS Japan event about AI and human scoring last week, I asked again about the use of AI scoring on the new TOEFL. I hesitate to mention it (because the TOEFL team seems apprehensive about this issue) but I got the clearest suggestion yet that ETS will not send every test taker response to a specific human rater for scoring. This will be a tough row to hoe as ETS has spent the past five years extolling the virtues of having humans individually score every response. Anyway, I could be wrong; maybe they’ll stick with humans for the long run or on a transitional basis for the first few months of the test.

ETS recently hosted a paid of big “TOEFL iBT Experience Day” events in Beijing and Shanghai. Participants in each received a booklet containing three official practice tests matching the new format of the test.  That seems to have been a print-only thing so you’ll have to ask a friend in China to mail you a copy (speaking of which, anyone want to send along a copy to add to my collection of testing ephemera?) but I’ll let you know if a digital version is distributed.

I know I sound like a broken record, but here we have more evidence of a renewed focus on the key Chinese market after several years of faffing about somewhat inefficiently in other markets.

Speaking of which, I noticed that ETS China has a new President/General Manager (brought on in February).  While the previous manager came to ETS from New Oriental with a background in education, the current manager has an exclusively business-oriented background.  That meshes well with the new look ETS of 2025.

I saw that Korean ed-tech Socra AI (known as Riiid until about two weeks ago) has entered into a licensing deal to create a TOEFL prep platform called Santa TOEFL.  Socra says that the platform will “[integrate] official ETS content, providing learners worldwide with updated practice tests and learning solutions aligned with the January 2026 TOEFL revision.”

It is starting to look like new licensing deals will be a bigger part of the TOEFL’s revenue stream as ETS seeks to reboot and reinvent its flagship product.  I wrote a few days ago about the Official TOEFL AI platform launched recently with a partner – as yet unnamed – in China. If this is indeed part of the strategy, it is probably a good idea. To my eye, test forms for the new TOEFL seem a heckuva lot cheaper to develop than test forms for the old version.

But this isn’t totally new. TOEFL already enjoys longstanding partnerships with Chinese firms like New Oriental who for years have paid big bucks to license retired test forms.  They also license retired tests to Chosun Ilbo (weird choice) in Korea which are sold to consumers for $44 a pop.  A partnership with the Japanese office of the Council on International Education Exchange to license the same test forms ended a few years ago.

Not that it matters, but Socra/Riiid is an interesting case.  During the pandemic, that firm raised an eye-watering $170 million dollars of series-D funding from SoftBank, mostly on the back of a good-but-not-great AI-powered TOEIC prep app.**  I think they used the money to buy up a more traditional English study platform, the management of which ended up running the whole merged company.  Is that what they call a reverse takeover?

**Those were the days, my friends.

My latest substack summarizes all the news you need to know about English testing. This one was sent out before the news from Cambridge, though I did send out a bonus issue about that story. The substack now has 322 well-informed subscribers, which I think is pretty great.

The photo of the week shows off one of the final official prep products for the original (pre-iBT) TOEFL test.  Here it is, in case you don’t wanna check out the substack itself:

I saw in this Youtube video that the Duolingo English Test now requires a room scan.  The required scan is a 360 degree motion (with a phone) akin to British Council’s EnglishScore test, plus a look at the space behind the computer.

I’ve never been a fan of room scans.  I find them cumbersome and I’m unconvinced of their effectiveness.  For what it’s worth, this one seems a bit less obtrusive than most of the scans I’ve carried out.  It seems unlikely to result in a request to drag furniture around my house or to take down all of my Stone Cold Steve Austin posters.

I’ll take the test later this month and report back with details.

One wonders if this nod to more traditional at-home testing is a reflection of Duo’s new status as a co-frontrunner for the HOELT tender.  Note also the reminder in this week’s THE article that every DET test is reviewed by a human proctor (something that not everyone is aware of).