My copy of the Official Guide to the TOEFL (7th edition) finally arrived.  As with previous editions, I will spend the next week taking a quick look at every chapter so that I can note all of the changes.  I’ll start today with Chapter One, which introduces the test in a general way.  Note that I won’t spend too much time talking about big-picture stuff (that is, the changes to the test from last year) since those are obvious.  I’m going to focus on smaller details that most readers might not notice.

You can read the whole blog series on changes at the following links: chapter one, chapter two, chapter three and four, chapter five, the tests.

Page 4:  Deleted the following study advice:  “select all the pronouns (he, him, they, them, and others) and identify which nouns each one refers to in the passage

Page 5: Added a reference to  New Zealand accents appearing on the test.

Page 5: Deleted this description of the headphones:  “test takers wear noise canceling headphones”

Page 5:  The test format chart is updated, of course.  Frustratingly, it uses the same “estimated timing” that ETS uses in its marketing materials so that it can claim that the test takes less than two hours to complete.  For instance, it notes that the “estimated timing” of the reading section is 35 minutes, when in reality it is 36 minutes long.

Page 6: No more references to confirming listening answers in the UI of the test.

Page 8: Description of possible reading passages changed from “historical” to “historical and biographical narrative” 

Page 8:  Shows that the reading article is now on the left side of the screen, and the questions on the right

Page 9/10:  The “category table” reading question is no longer mentioned (important)

Page 13:  “the pictures that accompany the lecture help you to know whether one or several people will be speaking”  changed to “pictures on the computer screen are intended to help you identify the roles of the speakers”

Page 15:  Listening questions worth more than one point are mentioned:  “Most questions are worth one point.  Some questions, however, are worth two points.  Special directions will indicate which, if any, questions are worth two points.  No more than one such question will appear on any test.” (important)

Page 18: campus situation reading passage changed from 80-110 words to 90-115 words.  (back in the 4th edition this was 75-100 words).  Likewise, General/specific reading passage length changed from 80-110 words to 90-115 words.

Page 20:  The following description of the integrated writing task has been deleted: “Test takers write a summary in connected English prose of important points made in the listening passage, and explain how these relate to the key points of the reading passage.  Suggested response length is 150-225 words; however, there is no penalty for writing more as long as it is in response to the task presented.” It has been replaced with:  “Test takers express information in an organized, logical and coherent manner.”

Page 21:  Updated score report description

Page 34:  Updated score report timeline.

 

I saw this interesting article in The Koala yesterday. It mirrors what I’ve observed happening in Australia – since the beginning of 2024 I’ve noticed Duolingo’s Australian office bringing some really talented people into the fold. They are working hard to increase acceptance of DET scores.

Says the article:

“Reflecting on Duolingo’s entry into the Australian market, Blacker noted that Duolingo is working on gaining Australian government acceptance for visa purposes, however, he expressed optimism and cited the overwhelmingly positive response from institutions across the sector and a willingness to accept the DET where possible.”

Earlier, I predicted government acceptance of the DET by governments in 5 to 10 years. I’ve mentioned Australia as being the trickiest of all cases, and acceptance there probably coming closer to the end of that time period. But maybe we’ll see some movement before 2034.

When I mention government acceptance of DET to insider-ish people the response is generally “Nah, never going to happen.” Or “Never going to happen unless X, Y and Z happen first.”

This sort of head-in-sand approach to score use can be risky. Just ask all the ETS folk who were darn certain that American schools would stop accepting the DET once the pandemic ended.

An interesting tidbit from the Australian Financial Review:

“A survey of 11,500 prospective, applied and current students by recruitment firm IDP Education found that there has been a sharp increase in the popularity of the US as news of more restrictive policies in Australia, as well as in Canada and in the UK, have started to influence decisions.

Jane Li, IDP Education’s area director for Australasia and Japan, said the US used to rank last on a preference list of four major student destinations but has now leapt to first. Canada has gone from first to last.”

What does this mean for the business of language testing? Well, you can expect DET’s volume to increase, as you don’t really need a TOEFL, IELTS or PTE score to study in the United States. As I’ve indicated here, I work closely with a decent number of students at very good universities in the USA and most of them applied using DET scores.

Perhaps it means a slight uptick in the use of TOEFL, as that test still has strong associations with schools in the USA.

\In any case, it certainly portends a decline in the use of the IELTS test. I think the importance of Canada to IDP’s business model cannot be overstated.

Someone recently posted a complaint on Reddit.

You can click the image below to read it (and the boilerplate message from ETS) but basically their TOEFL scores were canceled because unauthorized software was detected during the test administration. The test taker is frustrated because the security specialist (proctor) failed to detect or shut down the software during the pre-test check. Their scores were canceled, and no refund was offered.  I’ve written about this many times here on the blog.

In my estimation, this is one of the biggest pain points test takers have experienced since the advent of at-home English testing in 2020. It has also been challenging for test makers because, as you know, bad stories spread very quickly over social media.

A variant of this pain point that was prevalent for the first three years of the at-home TOEFL was what some people dubbed the “insta-cancellation.” In such cases the proctor carried out all of the required pre-test checks. This included remotely accessing the test-taker’s system and manually shutting down unauthorized software. The proctor would then begin the test… which would immediately be terminated due to the detection of unauthorized software. No refund.

Fortunately, the insta-cancellation was entirely eliminated due to changes to the proctoring process introduced at the end of July 2023.

But by that time, reports of this (and similar) problems had already spread quite widely on social media. One could find them quite easily on the typical social media sites… but also on less traditional places like Trustpilot. Heck, once overseas test-takers learned about the BBB, they piled in there to make complaints as well. A bunch of people even left negative reviews on the Google Maps listing for the ETS headquarters in New Jersey. I believe that all of this had a very deleterious impact on the reputation of the at-home TOEFL.

It’s no great surprise that Duolingo’s marketing materials specifically mention their much more generous approach to unauthorized software. I would say that is evidence of some big brains in Pittsburgh, but even I could have figured it out. And, folks, I went to a test-optional university.

I don’t mention this to rag on ETS or to advocate for test takers. I mention it because there are still a lot of people scratching their heads and wondering why the market shifted so quickly, and why both students and institutions were so quick to embrace alternative English tests.  This is one of the reasons.

Some readers might have noticed that GregMat’s YouTube channel is offline.  According to Greg, that’s due to a copyright strike filed by ETS.

If anyone at ETS is reading this, listen up.  You shouldn’t do this sort of stuff.  In order for your tests to thrive you need a vibrant community of Youtubers, prep providers and book authors.  This is especially true when you’ve got a test like the GRE that is on the ropes.  I know, I know, you’re thinking “we don’t need that sort of community because we sent branded bluetooth speakers to some Instagram influencers!”  But seriously, you do need it.  Greg was your very best GRE ambassador, and was quickly becoming your best TOEFL ambassador.  Especially among people in your #1 target demo (young people from India).

Update:  The copyright-striked Vince Kotchian too. So dumb.  The dude was just on “Tests and the Rest” saying decent things about the GRE.

Pearson’s 2024 Q1 trading update is now available. Of the PTE, it says:

“Pearson Test of English declined slightly due to a strong comparator, and we expect performance will ramp through the year.”

We’ll see. I’m bullish on PTE’s potential, but we are now in the brave new world of reduced admissions in Canada, the UK and Australia. One imagines that fewer people will take English tests in general.

Q1 results from Duolingo will be published in a few days. Perhaps they will provide more clues as to what lies ahead.

Below find a complaint from an IELTS test-taker that was shared on social media a few days ago.  Click to embiggen and get a clear view of the image.

This individual experienced a technical problem during his online (at home) IELTS. Their test was not validated and no refund was given. They then spent a large amount of money to take the test at a test center in a different country.  They report a total cost of 1400 Euro to get an IELTS score.

They also shared the complaint they sent to IDP CEO Tennealle O’Shannessy which touches on some of the recurring complaints of test-takers from the Global South.

A few things stand out:

  1. This whole ordeal could have been avoided had the test taker been given the benefit of the doubt and offered a free re-test. Remember that this sort of complaint serves as a big blinking billboard advertisement for tests that do provide the benefit of the doubt when something goes wrong. I’m not sure that the legacy test-makers realize that yet. 
  2. I should probably stop writing here because point number one is the biggest problem that legacy test-makers have when it comes to online testing. Unauthorized software running in the background during the test?  Canceled. No refund. Finished the test really quickly and still got high scores?  Canceled. No refund. Your internet stopped working? Canceled. No refund.  Mom accidentally opened the door to your room? Canceled. No refund.  Something weird happened and we can’t tell you exactly what because of privacy concerns?  Canceled. No refund. I understand that it’s harder for legacy test-makers to provide re-tests because their proctoring systems are more costly.  But it has been four years and clearly the loss of market share is costing them more than it would to pay people to proctor some re-tests.
  3. Don’t get me wrong.  The tests must be canceled when the above things happen.  But free re-tests would go a long way towards rehabilitating the image of these tests among the test-taking population. 
  4. The complaint made to IDP’s CEO is a bit trickier to parse.  But, again, I think it speaks volumes to why certain tests are growing in popularity and others are declining.  It’s increasingly obvious that certain legacy test-makers have seen profits from their home turf in Europe and North America evaporate over the past few years (decades?).  In response, they’ve pivoted to making as much money as they can from test-takers in the Global South. That’s valid, but they’ve shown a somewhat callous disregard for the circumstances from which people there are testing from.  I guess in the long run it doesn’t really matter because better managed firms will come and replace the legacy test-makers… but maybe the legacy players could do better and avoid that fate. 
  5. Don’t take my word for it.  Just look at the share price of IDP Education.  This is not a firm that people are optimistic about. I know, I know, they are also facing headwinds when it comes to placement fees… but IELTS is still, by far, their biggest revenue generator.

Happy April, folks!

First up – new versions of the Official Guide to the TOEFL and the two Official Tests books were published this month, but my copies haven’t arrived so no news about those in this month’s column.  Maybe next month.  Meanwhile, I did read a few things.  They are…

  • Nancy’s Isenberg’s White Trash: The 400-year Untold History of Class in America.  This scholarly look at the class divide in America had been on my to-read list for some time.  It was worth the effort it took to find a copy in Korea.  Check this one out to learn about the sometimes deplorable conditions of the poor in North America, beginning with the early days of colonization.  The story of America, I think, is the story of the poor.
  • In the March 2024 issue of “History Today,” I liked Was the Trojan Horse Real ? , a short article about the fake horse of Greek Mythology.  I’m sure you’ve heard of it – Greek soldiers apparently hid inside of it to better facilitate the capture of the city of Troy.  But was it real or is it just part of a made-up story that has endured for centuries? 
  • I also liked The Golden Age of Medieval Nostalgia.  You’ll have to pay for this one so I will keep it brief, but it’s a fun look at life in Europe in the 14th century when “the world turned upside down” due to significant social changes.  Any number of the trends and events described here could be turned into TOEFL reading questions.  Real TOEFL nerds might recognize “the Little Ice Age,” which is referred to early in the article.
  • Measuring the Shape of the Earth is about the exact sort of “why this?” thing that might show up in a TOEFL reading passage.  Is the earth flatter at the poles or around the equator?  Who cares?  Well, geographers, I guess.  As I’ve written here before, physical geography is a common topic in the TOEFL reading section.

By the way, you can get three issues of History today for Five GBP.  That’s like the best deal in magazines out there.  Just make sure to unsubscribe before the auto-renewal kicks in.

  • Finally, I read the June 2023 issue of the Literary Review of Canada.  It included a short article about the work of John Tuzo Wilson, the so-called “Charles Darwin of Geology.”  He contributed greatly to the theory of plate tectonics.  Geology is another common topic on the TOEFL (really, check out the link above).  And I am 100% sure that plate tectonics have come up more than once on the test.

That’s all for now.  Catch you again in May.

Amazon is now shipping copies of the new Official Guide to the TOEFL. As noted a few days ago, the guide no longer contains certain long running inaccuracies, so it’s probably a good time to record the Saga of the Altruism Question.

In late 2005 the first edition of “The Official Guide to the New TOEFL iBT” was published. It contained numerous inaccuracies. One can’t really blame the writers, as they compiled the book before the test launched. It brings to mind those early Star Trek: The Next Generation paperbacks where Troi calls Riker “Bill” and Tasha Yar has long hair.

The most notable errors were two depictions of the integrated writing task. One about group work (contained in the chapter about the writing section), and one about altruism (found in the practice set). I can go into details in the comments if you like, but basically this question has a very specific form and neither of the samples followed it.

Sadly, these two questions also appeared in the second edition, published in 2006.

By this time, third party publishers were releasing their own TOEFL prep books. And here’s the thing: they naturally based their books on the contents of the Official Guide. As a result, every single one of them contained terrible integrated writing questions. I’ll try to create a slideshow below that highlights some examples.  Sorry… it will probably look like trash on mobile.

For the most part, major publishers are adverse to spending money, so these errors remained in the books for ages. Kaplan included terrible integrated writing questions in their famous purple books right to the day they discontinued them. Princeton Review added a new integrated writing question to the 2024 edition of their TOEFL book which is horrific. If you squint at it long enough you’ll notice that it was inspired by the Official Guide.

Had the original book contained proper questions, this problem could have been avoided.

Anyway, the bad questions remained in the third edition, which was published in 2009.

By this time I was teaching TOEFL. At least twice a week someone would send me a practice essay based on the famous altruism question and ask me to grade it. Every time I’d politely explain that even though the question came from the Official Guide, it wasn’t accurate and it would be a waste of their time and money to have me check it. Fifteen years later, I still have to explain that a few times a month.

The questions remained in the fourth edition, published in 2012. By this time ETS had licensed dozens of retired tests to New Oriental, so the proper format was widely known.

The questions remained in the fifth edition, published in 2017.

Teachers were hopeful that the sixth edition, published in 2021, would not contain these faulty questions given that the book required radical revisions to match the changes to the test of 2019. Sadly…  it appeared once more.

But hey.  It’s 2024 now.  Nineteen years have passed.  The bad questions have finally been removed from the book.

The Chosun published a story yesterday about exam tourism.  It reports that Chinese students are flocking to Korea to take the TOEFL and IELTS tests due to a shortage of testing centers in China.  Perhaps half of IELTS test-takers in Korea are Chinese nationals, according to the article.

It is mentioned that there are only 95 IELTS test centers across China, despite a testing volume of about 500,000 per year.  Yikes! I am not sure how many TOEFL test centers there are in China, but I believe that is the test’s number one market.

Apparently, test-takers previously traveled to destinations like Thailand and Vietnam to take the tests, but have switched to Korea now that the ban on Korean culture in China has been lifted.

Regular readers will recall my interview with a TOEFL and IELTS teacher in China who mentioned that due to a shortage of seats in testing centers, students hire agents to sit in front of a computer and nab appointments as soon as they become available.

The problem is likely due to the fact that ETS and BC/IDP don’t really run the (whole) show in China like they do in other countries.  Test registration is instead handled by the NEEA, which is an organ of the Chinese state.  That can, sometimes, create inefficiencies.  Note that at-home testing is somewhat limited in China and that TOEFL test-takers in that country must use a workaround involving an ETS office in Hong Kong.  I believe the at-home IELTS is simply not available, but correct me if I’m wrong, please.

Not mentioned in the article is the fact that both tests are about $70 cheaper in Korea.  Good deal.

My copy of the new TOEFL iBT Premium from Barron’s just arrived.

I was really lucky to have the opportunity to contribute a little bit to this edition (the 18th). My job was mainly to revise the integrated writing questions and make them a bit more accurate. I also fact checked the boring stuff (payment methods, acceptable IDs, whiteboard rules, supported operating systems, score arrival times, etc) that most people don’t read (but really should read).

I think regular users of this book will really appreciate revisions to the practice reading tests, which have been tightened up quite a lot. The design of the questions more closely matches the look of the real test.

Pamela Sharpe has written every edition of this book, starting in 1978. If you want to know something about the history of the TOEFL program, she’s the one to ask.

 

I just learned that the TOEFL iBT Paper Edition was quietly discontinued in January.

The Paper Edition launched in December of 2021 in select cities across four countries. Test-takers took the Reading, Listening and Writing sections on paper at a test center, and took the speaking section from home on a computer. It included unlimited score reports, which was a nice touch.

There was some demand in India, I think, for a paper option.

This test should not be confused with the TOEFL Paper Based Test, which was a totally different test (based on the 1995 revision of the original TOEFL) that was discontinued in 2017. Nor should it be be confused with the TOEFL Revised Paper Delivered Test, which was just like the TOEFL iBT test but without a speaking section. That was discontinued in 2021.