Inside Higher Ed  is reporting this evening that testing firm Educational Testing Service (ETS) has offered voluntary buyouts to every employee in the United States with more than two years of service at the firm. The publication refers to this as “massive downsizing.”

This news comes just a few weeks after layoffs that affected about 69 employees, which I wrote about in this space.

According to a video address by CEO Amit Sevak that was sent to employees and obtained by Inside Higher Ed, anyone “on the fence” about the package is encouraged to take it, and that “the purpose is to reduce our staff in the most gracious way we can.”  Sevak reportedly described the buyout as “an opportunity.”

Unsurprisingly, that’s not how employees see it.  Inside Higher Ed quoted a longtime ETS employee as noting:

“This is affecting people who raised their families alongside their work at ETS, people who have spent lifetimes working on a single product… it’s been an hour since the news broke and folks are earnestly sharing self-harm and suicide-prevention hotlines.”

According to Inside Higher Ed, Sevak noted in the video that involuntary layoffs may occur once the voluntary buyout has been completed.

IHE cited a decline in popularity of the GRE as one possible cause of the layoffs. They did not mention increased competition in the English language testing space, which may also be a factor.  They did, however, mention that ETS’s new contract with the College Board is “less lucrative” than in the past. I suspect this is one of the biggest factors, as the 80+ year-old deal with College Board accounted for about 30% of revenues at ETS according to their most recent tax filings.

I’ll post more as I learn it.

Given the recent goings-on at IDP Education (layoffs and downsizing), I think it’s worth repeating my standard pitch for disruption in the English testing industry.  I posted this on LinkedIn a few days ago, but I realize that some very smart people don’t use that platform.

I’ll preface this by pointing out that while on stage at the Duolingo English Test Convention a few days ago I noted that I don’t think that the DET is more valid than other tests, or that it has better items.  Test items are not what make me a fan.

With that out of the way, here’s the pitch.

At the aforementioned DETcon, Duolingo co-founder Luis von Ahn told the story of his personal experience with language testing and how it informed (and continues to inform) the development of the DET.  He has told this story many times now.  In brief, it goes like this:  while applying for schools in the United States he needed a TOEFL score.  Sadly, there were no available spots at test centers in Guatemala, where he lived, and in order to meet his deadlines he was forced to travel outside of the country to take the test.  The cost was significant.

That was in the 1990s. I wasn’t doing test prep at that time.  But when I started teaching around 2010, it was still clear to many people that the test center model was less than perfect. People were still traveling great distances to take tests.  On top of paying for test fees, bus tickets and hotels, test takers experienced lost wages due to  travel times.  A $150 test could easily become a $1500 test.

But here’s the thing: in 2024, the test center situation is even worse.

I speak to test center operators now and then.  They tell me how their relationships with legacy testing firms have changed over the past decade.  Once upon a time, they tell me, they were paid a flat rate for a scheduled test administration, no matter how many people had registered for the given date.  But now, they complain, they are paid based on the number of registrants.

In effect, fewer registrations = less profit.

Can you see where this is going? The test center operators tell me that if there aren’t enough test takers to make an administration on a particular date worthwhile, they find a way to cancel it. When this happens, test takers are left in the lurch (deadlines be damned).  I hear from test takers who have experienced this.  Sure, they get a new test date at no cost, but they can kiss goodbye to the money they’ve spent on hotels and bus tickets.

See what I mean?  A bad thing has been transformed into something even worse.

But at-home testing is here to save the day, right?

Well, when the legacy players got into the at-home business back in 2020, their product (in my opinion) was pretty clunky.  Tests were often terminated for nebulous reasons.  Instructions were badly communicated. I still remember how one at-home test told every test taker to put on their headset just before the listening section started… but terminated the test whenever someone actually followed those instructions because wearing a headset was against the rules of the at-home test.  I’m not kidding. Apparently it took years for them to figure out a way to remove those instructions from the at-home version.

And in 2024?

Again, the product appears to be even worse. Back in ‘20 when something went wrong a person taking a legacy test could usually get a free re-test. But now? If something goes wrong test takers are often told to go pound sand. Back in ‘20 when a jagged score profile was used to justify a score cancellation the test taker could appeal the decision or simply take the test again for free.  But now?  The appeals process has been eliminated and they can take it again only at their own expense. I hear these things every week from heartbroken test takers. I end up with a broken heart, too.

And don’t even get me started on the legacy testing company that recently made big cash payments to at-home test takers after reaching a settlement with the US Attorney’s Office to resolve accusations that they had violated the Americans with Disabilities Act (which was not the first time that accusation had been leveled at that firm, by the way).

But maybe the move to at-home testing has eliminated barriers to education by reducing fees for people in countries where running test centers is expensive due to logistics and security-related issues?

Not really.

Today a kid taking the TOEFL from his bedroom in Japan will pay $199.  A kid in neighboring South Korea will pay $220.  Meanwhile, a kid taking the test from his bedroom in Afghanistan, who thinks an education might be a path to a better life, will pay $230.

A kid in the Palestinian Territories will pay $270.

All for the same at-home test, delivered in the same way, proctored by the same proctors, and graded by the same raters.

I’ve never really gotten an explanation for this.  My guess is that there are iron-clad deals in place with test center owners that prevent equitable pricing for the at-home TOEFL. But it could be something else.

In any case, things are not any better than they were when I started teaching in 2010. And that, in effect, is why I support disruption in English language testing.

I’ll end by reminding my friends at legacy testing companies that this is why many influential people are excited by the idea of disruption, this is why institutions are accepting new tests, and this is why test-takers are choosing those tests.

This is why, in part, there are so many layoffs. This is why you need to start doing better.

A few points are worth mentioning that I couldn’t fit into the above:

  1. Fees for some tests have increased way beyond the rate of inflation in certain markets, despite the fact that those tests include far fewer items than in the past.
  2. I talked to one test center owner in Germany who said he sometimes runs tests at a loss because he feels guilty about canceling dates.
  3. Charging $25 to send a score to an institution is no longer justifiable.
  4. If key parts of your test have been outsourced to private equity your test is likely doomed.
  5. It costs $450 to take the TOEFL from your bedroom in Switzerland.
  6. The next Luis von Ahn will pay $230 to take the TOEFL from his bedroom in Guatemala.  Or he might just opt for the DET.

Last week I attended DETcon24 (that is, the 2024 edition of the Duolingo English Test Convention) in Pittsburgh, Pennsylvania.  That’s the annual event that Duolingo holds to help stakeholders (from universities, mostly) become familiar with the DET.  Since I’m not affiliated with a university, I was really flattered when the Duolingo folks extended an invitation.

While my scribbled notes can still be comprehended, I thought I’d share a few thoughts from the event.  They are:

  1. The presentations on proctoring and security were my favorites.  I’m NDAed, so I can’t write any details.  But I’m certain that if the message about security can be repeated again and again with relevant decision makers in the room, the needle can be moved.  The Duolingo folks need to find a way to communicate this stuff to more people. Perhaps a traveling road show.

 

  1. If you have an at-home test, I really think it is important to do security and proctoring in-house. This allows the tools to be more bespoke and customer friendly.  And, of course, to provide additional security. Needless to say, Duolingo does it this way.  Many of the issues I’ve written about here (cancellations for jagged profiles, cancellations for NVIDIA drivers, cancellations for memory usage) seem, in my opinion, to be examples of security being a blunt one-size-fits-all instrument. If you don’t do security properly people get annoyed and abandon your test (best case scenario) and innocent people suffer unduly (worst case scenario).

 

  1. The overall test volume seems to be higher than my estimates.  Likely because the volume of vouchers given to institutions is quite high.  I’ll see if I can repeat the exact number given for 2023.

 

  1. For some more info on the new writing tasks, read: “The Impact of Task Duration on the Scoring of Independent Writing Responses” now in preprint.

 

  1. I spoke on a panel.  I was very nervous.  I wore a new shirt.

 

  1. I think many decision makers still have an image of the Duolingo Test based on its 2019 iteration.  Duolingo needs to do a better job exposing more people (and more types of people) to the most recent iterations.

 

  1. There was a comment about how and why Duolingo doesn’t really work with test prep providers.  And about not selling test prep materials.  Food for thought. Maybe testing companies shouldn’t sell prep for their tests. Someone made a comment about students having to unlearn IELTS prep. I laughed. But I love all my IELTS friends.

 

  1. Daniel Isbell has an article due out around the end of this year (but correct me if I’m wrong about the timeline). I won’t give away the topic but it will be an absolute must-read for anyone in the test prep space. If you are interested in this kind of stuff, check out his work. He’s the very best.

 

  1. There will be concordance matching the new individual subscores to subscores on other tests.  The 2022 overall concordance will not change. I think that the sum of the subscores will match the overall score (which is not the case with the integrated subscores).

 

  1. An attendee made a comment about how institutions are loathe to change score requirements when concordance tables are changed, because they don’t want to be the only one. That explained a lot.  I thought it was just inertia.

 

  1. I got a plush Duo for Mrs. Goodine. It is a surprise, so don’t tell her.

According to an email I just received, the cost of taking the TOEFL iBT Test will be reduced in Japan due to the “historic depreciation” (I translated this phrase from the Japanese) of the Yen.

From June 5 to September 30, the cost will be $199. That’s down from $245.

If anyone spots changes in other countries, let me know.

IDP Education will lay off about 6% of its global workforce.  I’ve been told that some affected individuals have already received the news.

According to a “Regulatory and Market Update” released to investors, IDP predicts that the international education market will decline by 25% next year.  We’ll see. Caps will impact Canada and Australia volumes significantly, but interest in studying abroad remains high and other destinations will be beneficiaries of those moves.

The most interesting information in the update is stuff that has already happened.  It notes that in FY24 (which ends in June 2024), IDP expects to report a 15 to 20 percent increase in its student placement volumes.  But it expects to also report a 15 to 20 percent decline in IELTS volumes.

Shares are down 7.5% on the news.

I always try to sound like a broken record: old testing monopolies have been broken up and the market is more competitive and consumer friendly.  Students are voting with their wallets and hearts.  They are reaching for other tests.

Legacy test makers like the IELTS partners (and ETS) are required to innovate.  They seem to be aware of that. As this update notes:

“IDP has a strong focused roadmap of product development across its core student placement and IELTS business lines that it believes will underpin long-term shareholder returns.”  

I’m writing this from my hotel room at the 2024 edition of DETcon where I’ve learned that innovation is possible. It takes time and talent, though.

The business proposition will be tricky for IDP perhaps.  IELTS-of-the-future will not be a $300 test.  It will be innovative as heck, but it will be much more affordable.  Volumes will go up, and customer satisfaction will go up too. But I don’t know how it will impact long-term shareholder returns.

I got this press release the other day about TOEFL’s “vibrant rebrand.”

It notes, in part:

“The new logo includes a stylized asterisk symbol, which it shares with the recently rebranded ETS, signifying both brands’ commitment to advancing the science of measurement through groundbreaking research to power human progress. The updated signature TOEFL periwinkle color highlights the brand’s innovativeness and dependability.”

Says newly-hired TOEFL GM Omar Chihane:

“Our learners are defined by a desire to create a stronger, better world around them. English-language proficiency helps them pave the way for the positive change they wish to see. TOEFL supports them along that journey, and this rebrand reflects that.”

A new color scheme (and accompanying logo) isn’t the sort of innovation that we’ll all been begging for.  And this reband probably won’t attract many new test takers.  But branding is important.  One hopes that this is the beginning of something positive from the folks at ETS.

The 2023 “Report on Test Takers Worldwide” for TOEIC is now available. Interesting reading, if you are into that sort of thing.  It includes score data from around the world, but note that most of the test-takers are in Japan and Korea.  Probably more than 80%.

A total of 3.2 million test takers answered the survey. That’s almost unchanged since last year.

In the final pre-pandemic year, about 4.8 million test takers answered the survey.

The TOEFL “Test and Score Data Summary” for 2023 should be available in the next month or two.

I spotted a couple of updates to the TOEFL TestReady platform today. They are:

  1. More free stuff is available. Students can now access the “section test sampler” which provides free access to one task from each section of the TOEFL (unscored, I think) and the “section practice sample” which provides the same (scored, I think). This is really nice, as one complaint I had when the platform launched is that ETS moved a big bunch of free stuff to the “free activity of the day” widget.
  2. There is something about connecting to an “expert teacher.” I guess it hasn’t been launched yet, but users can sign up for updates. I wish I had taken a screenshot, since the info all disappears once you’ve entered an email address. I suppose this is separate from the on-going partnership with Preply as that has already launched. Let me know if you learn anything about this, as sometimes emails don’t reach me.

Here’s a quick summary of changes to “Official TOEFL iBT Tests Volume 1.” I’ll dig into the second volume later this month.

In addition to the below, note that all of the independent writing questions have been replaced with “Writing for an Academic Discussion” questions.

I didn’t spot any changes whatsoever to the speaking questions, or the integrated writing questions.

So. Here’s the list:

Test One:

Reading:  “Petroleum Resources” removed.

Test Two:

Reading:  “The Cambrian Explosion” removed.

Listening:  Lecture about Astronomy removed.  Lecture about Earth Sciences added.

Test Three:

Reading: “William Smith”  (it had a weird “in mentioning x” question) and “Infantile Amnesia” removed. “Pest Control” added.

Test Four:

Reading: “Galileo and his Telescope” and “Europe in the 12th Century (it had a chart question) removed. “Understanding Ancient Mesoamerican Art” added.

Listening:  Conversation at a computer center removed (it was dated).  Conversation at an art museum added.

Test Five:

Reading:  “Cetacean Intelligence” removed (it had a chart question and a weird whole-passage question)

   

Here’s the second part of my musings (originally posted on LinkedIn) on the new Password Plus Test.  Check out part one here.

  1. The reading section is more relaxed than in many other tests. I tried my very best to pick the correct answers, but used only half of the given time (70 minutes).  Compare this to the TOEFL, where time management is the name of the game and tutors generally advise students to skip the articles and jump right to the questions. 
  2. The pre-test check-in took quite some time.  Maybe about 20-30 minutes.  The proctor did not have remote control of my system, like they do for many other tests.  Instead, I shared my screen and was directed to open up the task manager and shut down processes and services myself.  I also had to flip through all of my Chrome profiles to disable extensions.
  3. Speaking of proctors, I had the same proctor all the way through and they were really quick to respond to me at all times. I was very pleased with the proctoring provided by Examity.
  4. Test takers can use paper and a pen to take notes!  That’s wonderful!  But they are limited to just one sheet of paper. That’s probably not enough.
  5. There was one instance when the question displayed on the screen did not exactly match the question that was delivered through audio.
  6. I really like the way that time is allocated in the speaking section.  As I mentioned before, there are five “sections” (each containing multiple questions) and test takers can divvy up the given time (20 minutes) as they wish.  This makes the speaking section somewhat less nerve-wracking than other tests.  ETS folks sometimes ask me why young people shy away from the TOEFL.  I normally tell them that it is because the TOEFL speaking section gives them nightmares.

I took the new Password Skills Plus Test from Password English Language Testing!  This is a new at-home test based upon an existing product that has been offered to institutions for some time.  The owners of the test reached out to me and offered a free voucher, which I couldn’t turn down.  Today I’ll share my initial thoughts, which I previously posted on LinkedIn.  Tomorrow I’ll share a second post, with more specific details (click here to read part two).

By the way, if anyone reading this is preparing for the test and wants a little help, please reach out to me!

Here goes:

I must note that there is A LOT of listening on this test. Test takers are given an hour to complete the section, and will likely use most of it. If you’re looking for a test of one’s ability to comprehend academic lectures, this one’s for you.

The speaking questions lean towards casual conversation, though I did get a challenging “describe this chart” question at the very end.

I really appreciate that the speaking section isn’t as frantic as in the TOEFL.  Test takers are given twenty minutes to complete the section, and may divvy up the time as they wish. They can, for instance, spend just a few seconds preparing for question one, thirty seconds preparing for question two and a full minute preparing for question three. You get what I mean. I think this reduces test taker anxiety without necessarily compromising the validity of the test.

I like the test taker handbook very much. Across its sixteen pages (with plenty of white space) test takers get a clear idea of what to expect on the test and how to avoid violating any rules. That compares favorably to the 46 pages of small print in the bulletin that TOEFL test takers are expected to read.

I’ve already mentioned here that I like the pricing of this test.  It costs 110 GBP (140 USD) in every country in the world. That’s less than most tests.  Rescheduling is free up to 24 hours before the test (and just $5 after that time).

However. There are a few things I didn’t like, and they are worth mentioning.

  1. Logging in was surprisingly challenging. I didn’t get a “click here on test day” link by email. And the Password Plus website doesn’t have a login button. Proctoring is done through Examity, and their website doesn’t have a clear login button either (maybe because they’ve merged with Meazure Learning). I had to google “how to log in to Examity.” I got a little panicky as my test time drew closer. That’s not great TTX.
  2. The Examity software is clunky and blocked either the timer or the question number at all times. Those UI elements should be moved.
  3. I flagged an issue regarding waveforms (more specifically: every other waveform) not moving during the speaking section. I’m pretty sure that was a test software issue and not an issue with my system. My response to one of the questions contains me saying “uh… proctor!!”

Overall, the test seems solid, but the implementation could use some work.

I’m happy to see new tests sprouting up nearly every month. Test takers and score users alike will benefit.