Month: June 2024

You Should Read More – Part 46

Posted on by TOEFL Resources

I have far too many unread copies of “The New Yorker.”  I have signed up for the $6 promotional deal quite a few times and now I’ve got a big pile of unread issues on my shelf.

My catch-up strategy nowadays is to read copies of the magazine when I’m on long flights.  This month I took a terribly long flight – to Pittsburgh for the Duolingo English Test Convention.  And I read six issues!  A few articles stood out:

  • In the April 4, 2022 issue (yes, 2022) I read How an Ivy League School Turned Against a Student.  It tells the story of a student who was admitted to the University of Pennsylvania… perhaps under dubious circumstances.  It is certainly a tawdry article, but interesting if you are into the world of university admissions.
  • In the same issue I enjoyed Killing Wolves to Own the Libs? About the politics of wolf culls.  They may be useful.  They may not be.  It would make a good integrated writing question, I guess.  I’ll add it to the list of pending topics!
  • In the March 20, 2023 issue I enjoyed The Little-Known World of Caterpillars.  I think the title speaks for itself.  The TOEFL often includes reading and listening content about animal species.
  • In the March 27, 2023 issue I read The Button-Pushing Impresario of Balenciaga. Fashion doesn’t show up on the TOEFL too often, but this is a fascinating deep dive into a world that I don’t know much about.  I’m not yet convinced that couture fashion is meaningful art… but I’m getting there.
  • Next, in the April 3, 2023 issue I read The Wild World of Music.  There is something here about elephants playing music.  That matches up with an integrated essay about elephant intelligence people keep sending me.
  • In the same issue I read a book review called Why the Animal Kingdom is Full of Con-Artists.  This is a perfect article.  It mentions a whole bunch of topics that would make for perfect speaking #3 topics, among them brood parasitism, kleptoparasitism and aggressive mimicry.
  • In the April 10, 2023 issue I read The Christian Liberal Arts School at the Heart of the Culture War.  Not very TOEFLy, but another article that might be of interest to people who are into the university scene in the USA.
  • Lastly, in the April 17, 2023 issue I read Family Values, about the mom who started PFLAG.  A very inspiring  story.

 

I also read a few books! 

First up, I read “Standing and Delivering: What the Movie Didn’t Tell” by Henry Gradillas.  It goes more into detail about the circumstances of the “Escalante Miracle” in East LA in the 1980s, a story that came up back in Part 24 of this column. This book is essential reading for anyone interested in the Escalante miracle of the early 1980s and can be read alongside Jay Matthews’ “Escalante: The Best Teacher in America.” Here’s what I wrote about it over on Goodreads:

Written by Henry Gradillas, the principal of Garfield High at the time Escalante was there, this book describes the challenging situation as it existed when he took over the school, and methods he used to address it. I don’t know if the lessons can be applied in 2024, but if nothing else the book is a nice time capsule from the 1980s.

It’s worth reading if you can find a copy.

Finally, I read David Graeber’s Bullshit Jobs, which is about the preponderance of meaningless jobs in the modern USA.  It’s a very funny book.

Now Begins the Next Gen TOEFL?

Posted on by TOEFL Resources

I read on Pereless that ETS is seeking to hire a “Director of Technology- Next Gen ELL.”

The job listing says:

“The Director of Technology – Next Gen ELL role provides strategic leadership, engineering and development roadmap guidance and subject matter expertise to assigned areas of Technology to drive meaningful business outcomes for the next generation English Language Learning products (Next Gen TOEFL). This role acts as the Global SBU Engineering Leader and as the main point of contact, communicating effectively with the SBU leadership, Product Management and Technology leadership, vendors, and across ETS and partners to act as a strong advocate and steward for the Next Gen TOEFL technology portfolio. “

And:

“This role “lives & breathes” all things that are next gen TOEFL, is a key SME with business commercial, operational and solution knowledge, and the single point of contact for all things technology (platforms, architecture, security, production operation) for next gen TOEFL.”

Aha!  There it is – the “Next Gen TOEFL.”  This is the first time I’ve seen ETS use those words in a public space!  Perhaps we can mark June 2024 as the date that work began on the successor to the TOEFL iBT.

Most readers are likely aware that the last time ETS began work on a “next gen” TOEFL was back in 1993.  That’s when ETS launched the “TOEFL 2000” initiative, which culminated in the launch of the TOEFL iBT twelve years later.

I suspect that the next iteration of the TOEFL will not involve 12 years of development.  Back in 1993 there were no serious competitors to the TOEFL in the USA (the University of Michigan’s MELAB is the only one that comes to mind). The IELTS was picking up steam outside of the USA, but it still enjoyed just a fraction of the total test taker volume of the TOEFL.  The competitive landscape is quite different in 2024, as I’ve written about ad nauseam in this space.

TOEFL Prices Might Increase Soon

Posted on by TOEFL Resources

It is worth mentioning that the cost of taking the TOEFL test generally increases (in some countries) on July 1. Last year, price hikes ranged from $10 to $105. Students who are planning to take the test may want to lock in their registrations before the cost goes up.

As always, I could be wrong. Caveat Emptor.

More GRE Cheating in 2024

Posted on by TOEFL Resources

Poets and Quants reported last month that The Ohio State University will no longer accept at-home GRE scores due to concerns about cheating.

Says the article:

“Business schools have overturned acceptances to dozens of African MBA candidates after they were suspected of cheating on the virtual GRE. The cheating scandal on the at-home GRE test, introduced during the pandemic, has caused some MBA admission officials to lose confidence in the security measures put in place by the Educational Testing Service, the administrator of the GRE.”

Programs mentioned in the article include The Ohio State University’s Fisher College of Business and the University of Minnesota’s Carlson School of Management.

An assistant Dean at Carlson is quoted as saying:

“At least 25 people in Nigeria succeeded in getting their scores through and then it scaled after they told other applicants. It is a testing issue, not a cultural or geo-political issue.”

Officials from The Ohio State University have asked ETS to audit recent results.  That audit was ongoing at the time P & Q published the article.

More on ETS Downsizing, Relationship with College Board

Posted on by TOEFL Resources

Inside Higher Ed has greatly expanded and updated its article on the situation at ETS.  Do check it out again if you haven’t looked since it was posted yesterday.

One thing that grabbed my attention was a note that “ETS will no longer administer the SAT.”

This comes after 77 years of ETS involvement in that test.

A College Board spokesman confirmed that “with the SAT Suite’s full transition to digital on College Board’s Bluebook testing platform, we now develop and administer the SAT and PSAT-related assessments directly.”

This represents the end of an era, I suppose, as running the SAT on behalf of the College Board was the whole reason that ETS came into existence in the first place.  The role of ETS in the running of the SAT has steadily diminished since the 2010s, but at one point College Board work represented more than 80% of ETS’s revenue, according to some sources. Indeed, for decades and decades that was the whole raison d’etre of ETS. And as recently as last year College Board work accounted for about 30% of ETS’s revenues.

As Inside Higher Ed points out, ETS will remain involved with the College Board (contributing to the AP program, I guess), but their new contract is much less lucrative than the previous one.  We’ll know the exact figure next time an ETS audit is released.  That will probably happen in January.

Buyouts at ETS, Layoffs Likely to Come

Posted on by TOEFL Resources

Inside Higher Ed  is reporting this evening that testing firm Educational Testing Service (ETS) has offered voluntary buyouts to every employee in the United States with more than two years of service at the firm. The publication refers to this as “massive downsizing.”

This news comes just a few weeks after layoffs that affected about 69 employees, which I wrote about in this space.

According to a video address by CEO Amit Sevak that was sent to employees and obtained by Inside Higher Ed, anyone “on the fence” about the package is encouraged to take it, and that “the purpose is to reduce our staff in the most gracious way we can.”  Sevak reportedly described the buyout as “an opportunity.”

Unsurprisingly, that’s not how employees see it.  Inside Higher Ed quoted a longtime ETS employee as noting:

“This is affecting people who raised their families alongside their work at ETS, people who have spent lifetimes working on a single product… it’s been an hour since the news broke and folks are earnestly sharing self-harm and suicide-prevention hotlines.”

According to Inside Higher Ed, Sevak noted in the video that involuntary layoffs may occur once the voluntary buyout has been completed.

IHE cited a decline in popularity of the GRE as one possible cause of the layoffs. They did not mention increased competition in the English language testing space, which may also be a factor.  They did, however, mention that ETS’s new contract with the College Board is “less lucrative” than in the past. I suspect this is one of the biggest factors, as the 80+ year-old deal with College Board accounted for about 30% of revenues at ETS according to their most recent tax filings.

I’ll post more as I learn it.

Disruption in English Language Testing

Posted on by TOEFL Resources

Given the recent goings-on at IDP Education (layoffs and downsizing), I think it’s worth repeating my standard pitch for disruption in the English testing industry.  I posted this on LinkedIn a few days ago, but I realize that some very smart people don’t use that platform.

I’ll preface this by pointing out that while on stage at the Duolingo English Test Convention a few days ago I noted that I don’t think that the DET is more valid than other tests, or that it has better items.  Test items are not what make me a fan.

With that out of the way, here’s the pitch.

At the aforementioned DETcon, Duolingo co-founder Luis von Ahn told the story of his personal experience with language testing and how it informed (and continues to inform) the development of the DET.  He has told this story many times now.  In brief, it goes like this:  while applying for schools in the United States he needed a TOEFL score.  Sadly, there were no available spots at test centers in Guatemala, where he lived, and in order to meet his deadlines he was forced to travel outside of the country to take the test.  The cost was significant.

That was in the 1990s. I wasn’t doing test prep at that time.  But when I started teaching around 2010, it was still clear to many people that the test center model was less than perfect. People were still traveling great distances to take tests.  On top of paying for test fees, bus tickets and hotels, test takers experienced lost wages due to  travel times.  A $150 test could easily become a $1500 test.

But here’s the thing: in 2024, the test center situation is even worse.

I speak to test center operators now and then.  They tell me how their relationships with legacy testing firms have changed over the past decade.  Once upon a time, they tell me, they were paid a flat rate for a scheduled test administration, no matter how many people had registered for the given date.  But now, they complain, they are paid based on the number of registrants.

In effect, fewer registrations = less profit.

Can you see where this is going? The test center operators tell me that if there aren’t enough test takers to make an administration on a particular date worthwhile, they find a way to cancel it. When this happens, test takers are left in the lurch (deadlines be damned).  I hear from test takers who have experienced this.  Sure, they get a new test date at no cost, but they can kiss goodbye to the money they’ve spent on hotels and bus tickets.

See what I mean?  A bad thing has been transformed into something even worse.

But at-home testing is here to save the day, right?

Well, when the legacy players got into the at-home business back in 2020, their product (in my opinion) was pretty clunky.  Tests were often terminated for nebulous reasons.  Instructions were badly communicated. I still remember how one at-home test told every test taker to put on their headset just before the listening section started… but terminated the test whenever someone actually followed those instructions because wearing a headset was against the rules of the at-home test.  I’m not kidding. Apparently it took years for them to figure out a way to remove those instructions from the at-home version.

And in 2024?

Again, the product appears to be even worse. Back in ‘20 when something went wrong a person taking a legacy test could usually get a free re-test. But now? If something goes wrong test takers are often told to go pound sand. Back in ‘20 when a jagged score profile was used to justify a score cancellation the test taker could appeal the decision or simply take the test again for free.  But now?  The appeals process has been eliminated and they can take it again only at their own expense. I hear these things every week from heartbroken test takers. I end up with a broken heart, too.

And don’t even get me started on the legacy testing company that recently made big cash payments to at-home test takers after reaching a settlement with the US Attorney’s Office to resolve accusations that they had violated the Americans with Disabilities Act (which was not the first time that accusation had been leveled at that firm, by the way).

But maybe the move to at-home testing has eliminated barriers to education by reducing fees for people in countries where running test centers is expensive due to logistics and security-related issues?

Not really.

Today a kid taking the TOEFL from his bedroom in Japan will pay $199.  A kid in neighboring South Korea will pay $220.  Meanwhile, a kid taking the test from his bedroom in Afghanistan, who thinks an education might be a path to a better life, will pay $230.

A kid in the Palestinian Territories will pay $270.

All for the same at-home test, delivered in the same way, proctored by the same proctors, and graded by the same raters.

I’ve never really gotten an explanation for this.  My guess is that there are iron-clad deals in place with test center owners that prevent equitable pricing for the at-home TOEFL. But it could be something else.

In any case, things are not any better than they were when I started teaching in 2010. And that, in effect, is why I support disruption in English language testing.

I’ll end by reminding my friends at legacy testing companies that this is why many influential people are excited by the idea of disruption, this is why institutions are accepting new tests, and this is why test-takers are choosing those tests.

This is why, in part, there are so many layoffs. This is why you need to start doing better.

–

A few points are worth mentioning that I couldn’t fit into the above:

  1. Fees for some tests have increased way beyond the rate of inflation in certain markets, despite the fact that those tests include far fewer items than in the past.
  2. I talked to one test center owner in Germany who said he sometimes runs tests at a loss because he feels guilty about canceling dates.
  3. Charging $25 to send a score to an institution is no longer justifiable.
  4. If key parts of your test have been outsourced to private equity your test is likely doomed.
  5. It costs $450 to take the TOEFL from your bedroom in Switzerland.
  6. The next Luis von Ahn will pay $230 to take the TOEFL from his bedroom in Guatemala.  Or he might just opt for the DET.

Notes from DETcon24

Posted on by TOEFL Resources

Last week I attended DETcon24 (that is, the 2024 edition of the Duolingo English Test Convention) in Pittsburgh, Pennsylvania.  That’s the annual event that Duolingo holds to help stakeholders (from universities, mostly) become familiar with the DET.  Since I’m not affiliated with a university, I was really flattered when the Duolingo folks extended an invitation.

While my scribbled notes can still be comprehended, I thought I’d share a few thoughts from the event.  They are:

  1. The presentations on proctoring and security were my favorites.  I’m NDAed, so I can’t write any details.  But I’m certain that if the message about security can be repeated again and again with relevant decision makers in the room, the needle can be moved.  The Duolingo folks need to find a way to communicate this stuff to more people. Perhaps a traveling road show.

 

  1. If you have an at-home test, I really think it is important to do security and proctoring in-house. This allows the tools to be more bespoke and customer friendly.  And, of course, to provide additional security. Needless to say, Duolingo does it this way.  Many of the issues I’ve written about here (cancellations for jagged profiles, cancellations for NVIDIA drivers, cancellations for memory usage) seem, in my opinion, to be examples of security being a blunt one-size-fits-all instrument. If you don’t do security properly people get annoyed and abandon your test (best case scenario) and innocent people suffer unduly (worst case scenario).

 

  1. The overall test volume seems to be higher than my estimates.  Likely because the volume of vouchers given to institutions is quite high.  I’ll see if I can repeat the exact number given for 2023.

 

  1. For some more info on the new writing tasks, read: “The Impact of Task Duration on the Scoring of Independent Writing Responses” now in preprint.

 

  1. I spoke on a panel.  I was very nervous.  I wore a new shirt.

 

  1. I think many decision makers still have an image of the Duolingo Test based on its 2019 iteration.  Duolingo needs to do a better job exposing more people (and more types of people) to the most recent iterations.

 

  1. There was a comment about how and why Duolingo doesn’t really work with test prep providers.  And about not selling test prep materials.  Food for thought. Maybe testing companies shouldn’t sell prep for their tests. Someone made a comment about students having to unlearn IELTS prep. I laughed. But I love all my IELTS friends.

 

  1. Daniel Isbell has an article due out around the end of this year (but correct me if I’m wrong about the timeline). I won’t give away the topic but it will be an absolute must-read for anyone in the test prep space. If you are interested in this kind of stuff, check out his work. He’s the very best.

 

  1. There will be concordance matching the new individual subscores to subscores on other tests.  The 2022 overall concordance will not change. I think that the sum of the subscores will match the overall score (which is not the case with the integrated subscores).

 

  1. An attendee made a comment about how institutions are loathe to change score requirements when concordance tables are changed, because they don’t want to be the only one. That explained a lot.  I thought it was just inertia.

 

  1. I got a plush Duo for Mrs. Goodine. It is a surprise, so don’t tell her.

TOEFL Price Drop in Japan

Posted on by TOEFL Resources

According to an email I just received, the cost of taking the TOEFL iBT Test will be reduced in Japan due to the “historic depreciation” (I translated this phrase from the Japanese) of the Yen.

From June 5 to September 30, the cost will be $199. That’s down from $245.

If anyone spots changes in other countries, let me know.

IDP Education Layoffs

Posted on by TOEFL Resources

IDP Education will lay off about 6% of its global workforce.  I’ve been told that some affected individuals have already received the news.

According to a “Regulatory and Market Update” released to investors, IDP predicts that the international education market will decline by 25% next year.  We’ll see. Caps will impact Canada and Australia volumes significantly, but interest in studying abroad remains high and other destinations will be beneficiaries of those moves.

The most interesting information in the update is stuff that has already happened.  It notes that in FY24 (which ends in June 2024), IDP expects to report a 15 to 20 percent increase in its student placement volumes.  But it expects to also report a 15 to 20 percent decline in IELTS volumes.

Shares are down 7.5% on the news.

I always try to sound like a broken record: old testing monopolies have been broken up and the market is more competitive and consumer friendly.  Students are voting with their wallets and hearts.  They are reaching for other tests.

Legacy test makers like the IELTS partners (and ETS) are required to innovate.  They seem to be aware of that. As this update notes:

“IDP has a strong focused roadmap of product development across its core student placement and IELTS business lines that it believes will underpin long-term shareholder returns.”  

I’m writing this from my hotel room at the 2024 edition of DETcon where I’ve learned that innovation is possible. It takes time and talent, though.

The business proposition will be tricky for IDP perhaps.  IELTS-of-the-future will not be a $300 test.  It will be innovative as heck, but it will be much more affordable.  Volumes will go up, and customer satisfaction will go up too. But I don’t know how it will impact long-term shareholder returns.

TOEFL’s Vibrant Periwinkle Rebrand

Posted on by TOEFL Resources

I got this press release the other day about TOEFL’s “vibrant rebrand.”

It notes, in part:

“The new logo includes a stylized asterisk symbol, which it shares with the recently rebranded ETS, signifying both brands’ commitment to advancing the science of measurement through groundbreaking research to power human progress. The updated signature TOEFL periwinkle color highlights the brand’s innovativeness and dependability.”

Says newly-hired TOEFL GM Omar Chihane:

“Our learners are defined by a desire to create a stronger, better world around them. English-language proficiency helps them pave the way for the positive change they wish to see. TOEFL supports them along that journey, and this rebrand reflects that.”

A new color scheme (and accompanying logo) isn’t the sort of innovation that we’ll all been begging for.  And this reband probably won’t attract many new test takers.  But branding is important.  One hopes that this is the beginning of something positive from the folks at ETS.