Some really great research this month from Alina Reid in support of the new ISE Digital Test from Trinity College London.  The research examines how the test’s new integrated reading/writing task connects to real-life academic writing.

This is important in a world where testing companies are quick to claim that their tests are “fit for purpose” without really supporting that claim.  Indeed, the author references studies which seem to question the validity of both the IELTS and TOEFL independent writing tasks.

In the case of Trinity’s test, 64% of surveyed EAP instructors agreed that the task in question resembled the assignments they give in their classes.  Which is good, but not great.  A greater share – 86% of respondents – agreed that the task engages similar skills and strategies that are used in their classes.  That’s a bit better.

Said one respondent:

“…if students are able to write reasonably well [at] this level, they could be equipped with many of the skills and strategies required to write lengthier assignments. You will have concrete evidence as a teacher of the capability of the student in terms of understanding texts, choosing relevant information from sources, synthesizing, and paraphrasing by answering a question like this.”

But another:

“This task does not reflect the real needs of university students. University students need to engage with extensive reading and evaluate the suitability of texts. They would never have to write something so short in forty minutes. Instead, they would have weeks to work on a more extensive research task which would result in a longer piece of academic writing. Shorter texts are more simple to organize and may not emphasize the importance of coherence.”

Personally, my perspective is similar to the latter comment. Tests can be great measures of one’s ability to use the English language. But I’m wary of claims that any two or three hour test can indicate the ability of a student to do meaningful academic work. Especially when a test maker just asks us to take their word for it.

I was happy to discover that Pearson has published updated print guides to the PTE Academic test (one for students, one for teachers). They cost $86 AUD a piece and only seem to be sold from Pearson’s Australian web store. But they do exist. Apparently.

I am still a big believer in print books because they can be stocked by libraries around the world. Libraries remain a major source of free test prep – the long reservation queues for prep books at many libraries speak to that.

I really like this article from Cambridge about “Ethical AI for language learning and assessment.” If I were trying to position the IELTS to thrive in today’s brave new world, this is the sort of message I would use. I would express that the test is keeping up with the times, but that the makers of the test take a more cautious and thoughtful approach than its competitors. Sounds easy, right? Perhaps not. Monday I’ll write about some not-helpful articles out of Cambridge that will get in front of many more sets of eyeballs in the days ahead.

This month I read the March 21, 2024 issue of the London Review of books.  A few articles caught my attention

I Adjure You Egg” is a short history of magical amulets and charms in medieval Europe.  A really fun exploration of a corner of European history we don’t often hear about.  People believed in such charms with a lot of sincerity, for a very long time.   It was interesting to know that charms were often scraps of written words rather than some object with more physicality, which is what we often picture when we think of charms.

Llamas, Pizzas, Mandolins” is a long look at AI and its implications.  Yeah, you are probably sick of this topic, but it is one that many still want to read about.  And I bet it will be asked about on the TOEFL for some time to come.

After that, I read “Assessing Academic English for Higher Education” by John M. Norris and Xiaoming Xi of ETS.  As I think I mentioned in an earlier column I am working my way through all of the books on my shelf (about tests or otherwise) that I have actually paid for with my own money.  This one has been on my shelf for a couple of years at least.  This was the perfect time to read it, as it serves as a sort of coda for the “old ETS” before the TOEFL gets blown up into something unrecognizable and mostly unconnected from what came before.  The book is almost sorrowful, as it describes page after page of good research and good ideas about assessing English for academic purposes that never made it into the TOEFL iBT.  Instead of using these ideas, ETS just sort of left the test in its original form, basically unchanged for two decades.  It is no wonder that so many bright minds left to build the Duolingo English Test.  

Finally, I read Richard Sennett’s “The Corrosion of Character,” a short book (really an essay) about work at the turn of this century.  There is some stuff in here that ETSers might appreciate too, especially about the futility of companies trying to reform into something new by “reengineering” themselves.  I was going to quote a relevant part of the book, but it was too long.  So here’s a dramatic reading:

 

This part may hit home:

“…institutions become dysfunctional during the people-squeezing process: business plans are discarded and revised, expected benefits turn out to be ephemeral; the organization loses direction. Institutional changes, instead of following the path of a guided arrow, head in different and often conflicting directions: a profitable operating unit is suddently sold, for example, yet a few years later the parent company tries to get back into the business in which it knew how to make money before it sought to reinvent itself.”

The cost of taking the TOEFL was adjusted in some countries this week. As was announced earlier, test takers in Japan now pay $195 to take the test (a $50 decrease). Meanwhile, their neighbors in nearby Korea must now pay $228 (an $8 increase). The price hike in Korea is the only increase I’ve been able to spot so far.

Prices have been adjusted downward in some (but not all) EU countries so that changes related to VAT collection no longer make the TOEFL wildly more expensive than other tests. I complained quite a lot about that when ETS first started collecting VAT in Europe some months ago, so I am happy to see this course correction.

While I was traveling around at the beginning of the month, I read another issue of the London Review of Books.  I can’t remember which issue, ’cause I threw it away when I finished with it.  But a couple of fun articles stood out:

  • Even the Eyelashes” is about the role of mummies in Inca society.  It’s a shorter than usual LRB article, which probably makes it better as a study resource.  This topic could make a fun TOEFL reading passage.
  • When Paris Sneezed” explores the reasons why people were so pissed off in Paris between 1748 and 1789.

Meanwhile, continuing my journey through the Norton Library Podcast, I read Charles Chesnutt’s “The Marrow of Tradition.”  Chesnutt’s book is a work of fiction set before and during the Wilmington Massacre of 1898.  I strongly recommend this book; for a 127 year-old novel, it is surprisingly accessible.  For some listening practice, you can check out the two-part podcast episode starting here.

For a little more listening practice, you might enjoy this Paul Wells podcast from a few days ago about the topic of book burning – mostly in Canada, but also elsewhere.  It also touches on the value of reading “the classics,” which matches one of the ongoing themes of this column.

Surprisingly, Pearson Languages is now selling an updated 4th edition of the “Longman Preparation Course for the TOEFL iBT Test.” This is the first update in 11 years.  I did not expect to see an update to this old book out of the blue.

As far as I can tell, the new edition is only available as a 12-month ebook subscription direct from the Pearson site. There is an Amazon listing for a physical paperback, but I don’t believe such a publication actually exists right now.

While browsing the “recent acquisitions” shelf at the Doksan Public Library I spotted this newish DET prep book (just beside a handsome new Korean edition of Ben Hogan’s “Five Lessons”). Given Doksan’s demographic makeup, we don’t get a lot of new test prep books, so finding it was a pleasant surprise.

A few months ago I wrote about how people working in test prep need to keep their eyes open in order to stay current.  Sometimes, I guess, that means visiting their local library.  Too many times have I heard people in test prep say things like “everyone stopped accepting the DET after the pandemic” or even “the DET isn’t very popular nowadays.”

Sometimes these statements are a sort of defense mechanism for individuals who have a livelihood tied up in the old testing duopoly.   But most of the time they are just rooted in ignorance of what is going on in the world of English testing.

(the book is decent enough, by the way)

 

 

It’s the last day of 2024!  How about that?  We made it!

So I read the September/October 2024 issue of Analog Science Fiction & Fact .  I know that I should probably stop bringing up this magazine here because the stories aren’t available for free, but I am what I am.  I must mention Adam-Troy Castro‘s “Minnie and Earl Have a Kitten.”  This is the fourth (and final?) “Minnie and Earl” story to appear in the magazine.  The first appeared in, I guess, the late 1990s.  If you want to experience a warm hug of a science fiction series, you can find an ebook of the first three stories on Amazon or on Hoopla.  They concern a friendly old couple who are discovered, by early Earth astronauts, to be living on the moon.  How they got there isn’t immediately clear.

I also read the November 30, 2023 issue of the London Review of Books.  A few stories captured my attention:

  • Searching for the Bee, about a very old book called The Wonders of Creatures and the Marvels of Creation, which was apparently one of the most widely read books of the 13th century.  I won’t try to describe the peculiar contents of the book here.  Check out the article if you want to learn more.
  • A National Evil, which is about how everyone in Switzerland used to have big goiters.  The history this article tells is so strange.  Apparently for many decades Switzerland was known as a place of goiters, deafness and developmentally disabled children.  Apparently this was all due to a lack of iodine in the diets of people there.  Tourists would travel to the country just to see the goiters.  I’m pretty sure this is the second article I’ve linked to on this topic since I began this column.

Finally, I read Nicholas Lemann’s new book “Higher Admissions: The Rise, Decline and Return of Standardized Testing.”  This slim volume is something of a followup to Lemann’s brilliant “The Big Test,” published about 25 years ago.  It tells the story of (mostly) the SAT – how it came to become dominant, the impact it has had on American education, and why many schools have recently abandoned it.  I actually penned a few words about the book a few days ago.  Do check out that post.

That’s all for now! I will spend most of next month traveling, but I should find time for at least a short list of recommendations.

Continuing along with the Norton Library Podcast, this month I read Bram Stoker’s “Dracula.”  You can check out the podcast episodes starting here.

One unique feature of this classic horror novel is that it is an epistolary novel.  That refers to how the book is presented as a collection of letters, diary entries, phonograph transcripts, newspaper articles and telegrams written by characters in the novel.  While the topic of vampires has likely never appeared on the TOEFL, I am quite certain that at some point there has been an article or lecture about this kind of writing.  So instead of reaching for Dracula, perhaps take a moment to sharpen your reading skills by exploring the wikipedia article on this topic.

Next, I read the 16 November 2023 issue of the London Review of Books.  Yeah… I have another pile of unread magazines.  Fortunately, this is one of my favorite publications and I look forward to working through all of the old issues on my shelf.  A few stories stood out in this issue:

  • Kettle of Vultures” is a quick look at the history of interest.  The charging of it, the collecting of it, the religious implications of it, and more.
  • Red Flag, Green Light” is the story of famous fraudster John Ackah Blay-Miezah.  He may have invented the famous “Nigerian Prince” scam.
  • I Thought You Were Incredible” is a fun read for film fans.  It’s a quick overview of the life of Elizabeth Taylor, with special consideration paid to her relationship with Montgomery Clift.

More of this sort of thing in 30 days.

I’m very happy that I can mark the 50th monthly installment in this column with a few words about Sanaz Toossi’s Pulitzer Prize winning play “English.”  As I’ve mentioned a few times in this space, this play is set entirely in a TOEFL classroom in Iran.  It is a very sparse play, but is a touching exploration of what it means to learn a language and to be able to (or not able to) express ourselves when we desire to do so.  It is also quite funny at times.  Highly recommended!

This month, while flying home from India, I plowed through the final four “New Yorker” magazines in my collection.  A few articles stood out.

  • First up, in the April 24 & May 1 2023 issue I read “How Much Can Duolingo Teach Us?” Of course this one is worth mentioning here!  The article briefly mentions the Duolingo English Test and the TOEFL Test!
  • Next, in the May 15, 2023 issue I read “Buzzfeed, Gawker and the Casualties of the Traffic Wars” about what it dubs “publishing in the age of web traffic.”
  • Moving along, in the May 22, 2023 issue I liked “Earth League International Hunts the Hunters,” about an anti-poaching organization.  Very cloak-and-dagger stuff.
  • Finally, in the May 29, 203 issue I read “Two Weeks at the Front in Ukraine.”  I don’t usually include reporting from the war in this column.  I liked this article, though, as this sort of embedded reporting that tells personal stories seems the best we can do in an era where accurate “big picture” reporting seems impossible to find.

More links in a month!

 

I just found some photos I took while I was in Pittsburgh this past June. While in the city, I visited the world famous Carnegie Library to read up on Chomsky, in hopes of better understanding the TOEFL writing and speaking rubrics.

Sadly, I left the library as perplexed as when I arrived.  It turns out that Chomsky is very difficult.  Who knew?

I kid.  But really:  if a test maker creates a scoring rubric and refers test takers to it when they seek guidance on the test, the rubric should contain language that is easily grasped.  Telling test takers that their responses should include “a variety of syntactic structures” doesn’t really help them.

Note the presence of Dippy the Dinosaur in the background of the first photo.  Study alcoves in the library stacks offer a nice view of the natural history wing of the Carnegie Museum.  Local children write letters to the dinosaurs and tape them to the windows.

Below are photos of the endcap displays of TOEFL and IELTS books at the big Kyobo Books location near Seoul City Hall.  Even more books of both types are found on the store’s shelves.  Eagle-eyed readers will spot a mixture of both official and third party publications.

When trying to account for why certain tests are popular in the East Asian market, it is really important to pay attention to the amount of prep materials on the market, especially (but not only) collections of practice tests.

To some extent, here in Korea TOEFL and IELTS have avoided the headwinds they face in other markets because there is such a healthy ecosystem of prep materials available.  The PTE is doing okay here, but it isn’t challenging IELTS to the same extent that it is in India.  Likewise, while the DET has a lot of fans here, it hasn’t supplanted the TOEFL as a test for America-bound students like it has in Europe.

People here want to take tests that they can prepare for on their own.  Young people in Korea spend a lot of time studying for tests, and are pretty good at breaking them down to their component parts and mastering them bit by bit.  But that can only be done when they have access to a ton of super accurate study materials. Reputable third party publishers here are trusted to produce practice tests that are 99% similar to the real thing. That’s one reason why some people here are sticking with the legacy tests.

In markets with less of this sort of stuff, there is less reason to stick with the legacy tests.

In part this explains the high volumes (in Korea) of somewhat older English tests like the G-TELP, TOEIC and OPIc, all of which remain extremely popular in the region.  Surprisingly so.

Test makers should keep this in mind.  They should also remember that the legacy tests can count on third party publishers to crank out a steady stream of materials on their own, but upstarts might have to shoulder some of the burden themselves.

(For the record, I didn’t spot any PTE books and I’m not sure any are available from Korea publishers at all.  I didn’t spot any DET books either, though the influential “Siwon School” company does have a line on the market.)

The Chinese market deserves a post all of its own.  So… I think I’ll save my thoughts on that one for a later date.