I’m very happy that I can mark the 50th monthly installment in this column with a few words about Sanaz Toossi’s Pulitzer Prize winning play “English.”  As I’ve mentioned a few times in this space, this play is set entirely in a TOEFL classroom in Iran.  It is a very sparse play, but is a touching exploration of what it means to learn a language and to be able to (or not able to) express ourselves when we desire to do so.  It is also quite funny at times.  Highly recommended!

This month, while flying home from India, I plowed through the final four “New Yorker” magazines in my collection.  A few articles stood out.

  • First up, in the April 24 & May 1 2023 issue I read “How Much Can Duolingo Teach Us?” Of course this one is worth mentioning here!  The article briefly mentions the Duolingo English Test and the TOEFL Test!
  • Next, in the May 15, 2023 issue I read “Buzzfeed, Gawker and the Casualties of the Traffic Wars” about what it dubs “publishing in the age of web traffic.”
  • Moving along, in the May 22, 2023 issue I liked “Earth League International Hunts the Hunters,” about an anti-poaching organization.  Very cloak-and-dagger stuff.
  • Finally, in the May 29, 203 issue I read “Two Weeks at the Front in Ukraine.”  I don’t usually include reporting from the war in this column.  I liked this article, though, as this sort of embedded reporting that tells personal stories seems the best we can do in an era where accurate “big picture” reporting seems impossible to find.

More links in a month!

 

Everyone taking the Duolingo English Test must now use a phone as a “secondary camera” during the test. This involves propping a phone against a heavy option and angling it in such a way that it can record the test taker’s screen and keyboard during the test.

This requirement seems designed to target remote-access cheaters. I guess Duolingo’s security team seem to believe they can detect when the action of a test taker’s fingers doesn’t match what is recorded from the screen.

I’ll take the DET before the end of the year to see how straightforward the setup is.

This video explains what to do (or you can read about it on the official DET blog):

I just found some photos I took while I was in Pittsburgh this past June. While in the city, I visited the world famous Carnegie Library to read up on Chomsky, in hopes of better understanding the TOEFL writing and speaking rubrics.

Sadly, I left the library as perplexed as when I arrived.  It turns out that Chomsky is very difficult.  Who knew?

I kid.  But really:  if a test maker creates a scoring rubric and refers test takers to it when they seek guidance on the test, the rubric should contain language that is easily grasped.  Telling test takers that their responses should include “a variety of syntactic structures” doesn’t really help them.

Note the presence of Dippy the Dinosaur in the background of the first photo.  Study alcoves in the library stacks offer a nice view of the natural history wing of the Carnegie Museum.  Local children write letters to the dinosaurs and tape them to the windows.

This PIE News article by Polly Nash has a ton of relevant details re: IDP’s move to begin direct testing in China.  It notes:

  1. IELTS testing in China is valued at about $380 million, or about 40% of the global IELTS value.
  2. Morgan Stanley estimates that about 850,000 IELTS tests are taken in China each year.
  3. Morgan Stanley further estimates that IDP could grab 40% of the Chinese IELTS market by 2028.
  4. IDP will outsource test center operation to an established third party.

If the 850,000 number is accurate, China represents about 50% of all IELTS testing done by the British Council, as their most recent annual report noted the delivery of 1.8 million tests.  This means that competition in this key market could have a significant impact on the British Council.

Outsourcing the test centers could be costly in the short term, but I’m sure IDP could supplement a third party network with “self-operated” centers.  That’s something ETS has pursued in China in recent years.

IDP Education shares are trading at $13.45 as I write this.  Shares are down 32% since the beginning of the year, and are down quite a lot more from their pandemic high of $38.88.

It isn’t necessary to describe once again the challenges that IDP faces, as they are already well known. That said, it is worth taking a moment to write about a potential bright spot that hasn’t been sufficiently covered elsewhere.

According to IDP’s most recent annual report, the company plans to begin “direct testing” of IELTS in China in the near future. That is, they plan to take registrations and operate IELTS test centers around the country.

It may come as a surprise to some readers to learn that IDP doesn’t do those things at the present time. Instead, the British Council has a monopoly on IELTS testing in China. Per an agreement between them and IDP, IDP is paid a flat-rate royalty for every administration of the IELTS that is carried out in the country.  I don’t know what that fee is.

In most other countries, IDP and British Council compete with each other when it comes to delivering IELTS (except for India, where IDP has enjoyed a monopoly since buying out the British Council’s operations in 2022).

What makes the move to direct testing particularly exciting is the fact that during peak testing periods, some students find it hard to book spots at IELTS test centers (and TOEFL test centers, for that matter).  According to anecdotal reports that have been shared with me, test takers sometimes resort to paying third parties to quickly to book spots at test centers as soon as they become available.  Other times they travel outside China to take tests.  You’ll recall reports in the Korean press a few months ago about “exam tourism” to that country.  Said the Chosun Daily at that time:

“The British Council, which administers IELTS, noted that they have observed a significant number of Chinese expeditionary candidates coming to Korea recently. Although they do not have statistics sorted by nationality, it is estimated that as many as half of the 30,000 people who take IELTS in Korea annually could be Chinese.”

I’ve been told that Malaysia is a popular destination for Chinese test takers as well.  Before the war, people went as far as Moscow to take tests.

This suggests to me that if IDP enters the market they won’t have any trouble filling seats at test centers.

Beyond that, a few other points are worth mentioning:

  1. Official numbers are not available to me, but I think it’s safe to say that the IELTS is the most popular test of its type in China, having surpassed the TOEFL in total test volume some years ago.  This is mostly because of the rise of Canada and Australia as study destinations, but also because the IELTS is a very attractive test for individuals in China.  The option of taking the IELTS on paper remains quite popular in China, and that’s something that competing testmakers don’t offer.  Test takers are also drawn to the test because of the 19 prep books published by Cambridge Press.  Again, that’s something that competing tests can’t match. People like the IELTS. When IDP enters the market, they will have a customer base that already appreciates their product.
  2. As discussed recently on the ICEF Monitor Podcast, the slowing of economic growth in China, coupled with high youth unemployment, may create opportunities for firms involved in the study abroad business.  Already, we’ve seen that young people in China are more likely than before to stay in school instead of entering the job market right after their undergraduate studies. This may translate into an uptick in individuals considering the option of studying abroad.  Of course, this could create more demand for the IELTS in China, even as demand shrinks in the key market of India. Because of the specific schools Canada-bound students often target, they may be less impacted by regulatory changes than students in other markets.  Again, this could keep demand for the IELTS high.
  3. If done quickly, the establishment of a test center network across China would give IDP a truly global presence, and could cement their place as a frontrunner to win the second lot (delivery) for the new HOELT test.
  4. The cost of taking the IELTS in China from the British Council is about $460 AUD.  According to IDP’s most recent annual report their worldwide average fee is currently $306. Direct testing will certainly increase their average fee.

Anyhow.  All of the above assumes that IDP Education has found a test center partner and have taken care of whatever obligations they might have to the NEEA.  Those two things could, of course, present challenges in the short term.

I urge everyone interested in testing to keep an eye on China.  It’s easy to miss key details since this particular social network isn’t allowed to operate in the country… but it remains critically important to the plans of all the testing firms.  More important than some people might realize!

I really liked this podcast interview with Melissa Mack about her long tenure at Kaplan and that firm’s growth from a somewhat unprofitable test prep firm to a multi billion dollar education company.

Obviously Kaplan does a lot more than test prep now, but they do come up now and then in some of the test prep communities I participate in (mostly GRE-related) and they did recently purchase the Barron’s brand name (along with their author contracts).

Regular readers might remember my fixation on 1950s and 60s era Kaplan. Using the Internet archive and Google Maps I sought out the exact locations of the original Kaplan Centers. I’ve updated my blog post on that topic a few times since originally publishing it. 

Duolingo just published a report on the “Demographic and Score Properties of Test Takers,” which contains a ton of useful and interesting data about who takes the Duolingo English Test.  It covers the year ending June 30, 2024. 

The report indicates that the overall mean score is 110.5.  Comprehension is the subscore with the highest mean score (118.06), while production is the lowest (88.97).  Percentiles of overall scores and subscores are also provided.  Numbers are also provided depending on test taker intent.

I was happy to see which percentage of test takers were first timers (68%) and repeaters (32%).  I was also happy to see that Koreans students are most likely to be repeaters (42%).  Norwegians, not surprisingly, are least likely (7%).  There is some fun data about repeaters that you can dive into on your own.

There is also demographic information about age and gender and first language.

Most interesting of all is the list of test sessions by test taker ID.  I will reproduce it here:

  • India – 18.91%
  • China – 15.88%
  • Canada – 4.35%
  • South Korea – 3.51%
  • Brazil – 3.42%
  • United States – 3.34%
  • Mexico – 3.12%
  • Indonesia – 2.78%
  • France – 2.09%
  • Pakistan – 1.94%

It isn’t surprising that India and China come out on top.  Those are two very big sending countries.  I was surprised to see South Korea in the number four spot… despite being physically in Korea.  That’s an indication that test watchers must keep their eyes open and always be talking to people in all segments of the industry.

Missing is data on which countries test takers plan to study on, or the countries of receiving institutions.

Below are photos of the endcap displays of TOEFL and IELTS books at the big Kyobo Books location near Seoul City Hall.  Even more books of both types are found on the store’s shelves.  Eagle-eyed readers will spot a mixture of both official and third party publications.

When trying to account for why certain tests are popular in the East Asian market, it is really important to pay attention to the amount of prep materials on the market, especially (but not only) collections of practice tests.

To some extent, here in Korea TOEFL and IELTS have avoided the headwinds they face in other markets because there is such a healthy ecosystem of prep materials available.  The PTE is doing okay here, but it isn’t challenging IELTS to the same extent that it is in India.  Likewise, while the DET has a lot of fans here, it hasn’t supplanted the TOEFL as a test for America-bound students like it has in Europe.

People here want to take tests that they can prepare for on their own.  Young people in Korea spend a lot of time studying for tests, and are pretty good at breaking them down to their component parts and mastering them bit by bit.  But that can only be done when they have access to a ton of super accurate study materials. Reputable third party publishers here are trusted to produce practice tests that are 99% similar to the real thing. That’s one reason why some people here are sticking with the legacy tests.

In markets with less of this sort of stuff, there is less reason to stick with the legacy tests.

In part this explains the high volumes (in Korea) of somewhat older English tests like the G-TELP, TOEIC and OPIc, all of which remain extremely popular in the region.  Surprisingly so.

Test makers should keep this in mind.  They should also remember that the legacy tests can count on third party publishers to crank out a steady stream of materials on their own, but upstarts might have to shoulder some of the burden themselves.

(For the record, I didn’t spot any PTE books and I’m not sure any are available from Korea publishers at all.  I didn’t spot any DET books either, though the influential “Siwon School” company does have a line on the market.)

The Chinese market deserves a post all of its own.  So… I think I’ll save my thoughts on that one for a later date.

In case you are curious, this is what a PTE-Core score report looks like. I think the design is fairly clean and modern.

I appreciate that the score report doesn’t contain any personal information, like my home address. That sort of information probably isn’t useful in a downloadable report, and the report is easier to share on social media when it is omitted.

My account on Pearson’s website also contains a “skills profile” based on my performance on the test. The information it contains could be useful for re-testers who want to narrow the focus of future test prep.

I took the PTE Core from Pearson! As I mentioned in an earlier post, this is a new test from Pearson used for permanent residency and citizenship applications.  As usual, I must type a few notes while the test is still fresh in my mind:

  1. I took this test at the “Pearson Professional Center” near Seoul City Hall.  Like most test centers in Korea, this is a very nice facility.  It is clean and modern and there is no noise from outside.  The staff speak Korean and English. My registration process was orderly.  Earplugs were provided.  The headset mic was not the noise canceling type, but a separate pair of noise canceling earmuffs was provided for use during non-speaking parts of the test.  Tall dividers between terminals helped a bit too (but see below).  I wish everyone could take their tests at such a nice test center.  How much do test center conditions affect scores on this and other tests?  I couldn’t take pictures inside the center, but I will post a few snapshots from outside an in the hallway in a gallery below.
  2. I was given a locker to store my belongings.  I noticed that locker number 4 was labeled 3b.  Locker number 13 was labeled 12b. Cute
  3. I counted about nine people in the room with me, all of whom were taking the PTE-A. I chatted with the test center staff afterward, and learned that demand for the PTE-A is way, way up in Korea.  I was told, though, that the PTE-Core is taken only once in a while.
  4. A Chinese national was at the test center to take the GMAT.  Remember those stories of Chinese citizens flying to Korea to take tests?  Apparently this test center gets PTE test-takers from all over the world.
  5. Security is tight:  my palms were scanned seven times before I began the test, and once afterward.  I was photographed, my signature was confirmed and I was quizzed on the spelling of my name.
  6. I was handed a test taker rules sheet for the PTE-A as I waited.  When I brought this to the attention of the staff, I was handed a PTE-Home rules sheet.  Indeed, it seems the PTE-Core is not frequently taken.
  7. This was my first time taking a test in a full test center since before the pandemic.  I must admit that the noise from other test takers sometimes made it hard to focus.  My recommendation to test takers (both PTE-A and Core) might be to dawdle while reading the untimed instructions at the beginning of the test, which could create an opportunity to answer the final speaking questions without any distractions.  That’s what I did.
  8. The test content is mostly general English, but with a few academic English items. You can go through the Pearson website to get an idea of what the test contains. As I mentioned in an earlier post, like the PTE this test contains a numerous and varied items. This differentiates it from some older tests that focus on fewer and longer items.
  9. The test seems to closely resemble the prep materials provided on the Pearson website. I didn’t notice any glaring differences.  Test takers should go through everything that the Pearson website contains.  That said, I wish there was more to review.
  10. A few things are worth remembering if you are helping someone prepare for this test.  First up: all of the “describe the image” questions I got were about charts and graphs.  I was expecting to mostly describe pictures, since that is what Pearson’s practice test includes.  I wasn’t totally thrown off my game, though, since the “learn about” pages for the test on Pearson’s website include graph-based questions.
  11. Second: There is a penalty for exceeding the word count range stated in the writing questions.  Sometimes that penalty means you get a score of zero.  I knew about this penalty because I had read Pearson’s prep materials.  But the penalty is not mentioned during the test.  Had I not read those materials I would have significantly exceeded the stated word count range every single time.  Staying below the limit is hard, and must be a part of any practice regimen.
  12. Third:  the listening section contains some audio snippets that seem to be taken from news broadcasts and other real-world sources.  They have a ton of background noise and sub-par audio, as you might expect.  The practice materials from Pearson don’t seem to represent this; the ones I listened to all had perfectly clear audio.
  13. For a test used solely for immigration to Canada, there is a heckuva lot of British accented speech here.  And I also heard one particular verb that I am certain 9/10 Canadians have never used in the given context.
  14. I spotted two (possibly three) grammar/language use errors in test items.  I suppose that is a consequence of generating items with AI.
  15. My score was reported in about one hour.  No kidding.  I’ll post a copy of my score report a bit later.  I received a score of 90, which equates to a CLB of 10.

I saw that ETS Strategic Capital has been renamed “ETS Capital” and has new leadership. It is now headed up by Emal Dusst, who comes to ETS from equity fund Sterling Partners.  His LinkedIn work history is about a mile long.

ETS Capital was founded in 2020 following ETS’s billion dollar sale of test delivery firm Prometric to Baring Private Equity. Notable moves include an investment in ApplyBoard and the acquisition of PSI Services LLC.  It was previously led by Ralph E. Taylor-Smith.

In recent days, many people have reported seeing “not available” in the academic discussion response section of their TOEFL score report.  It looks a bit like this:

This means that AI-generated information about your response is not available.  Why does this happen?  It happens because your response could not be scored by ETS’s e-rater AI and was scored only by human raters.

Why could the AI not score your response?  That is unclear, but it sometimes happens if a response is extremely long.

Some students have reported that their writing scores are unusually low when this happens.  I don’t know what to say about that.

Leave a comment below if you have seen this in your score report.