Month: October 2023

You Should Read More – Part 38 (of 100)

Posted on by TOEFL Resources

As usual, I read a bunch of stuff this week.  I’ll get right to it.

First up, I checked out the January 2023 issue of “History Today.”  A few articles seem relevant to TOEFL test-takers.

  • Hawk this Way describes the street sellers that hawked their wares on the streets of London around 1900.  Apparently there were more than 12,000 hawkers at that time in London alone.  Some great vocabulary in here with bits like: “though they traded without formal sanction and frequently fell foul of the law…”.  The article paints a really vibrant picture of an aspect of the city that disappeared around the time of the first world war.  Plenty of historical background is presented.  This article is somewhat similar in length and reading level as a real TOEFL reading passage.
  • The Madman of the North is a fun article about Charles XII of Sweden and his thirst for war.  Today one doesn’t often think of Sweden when thinking of European military history, but apparently people in the early 1700s sure did.
  • The Cold, Cold War is about rival nations trying to be the first to reach the Arctic.  It touches on the life of explorer Robert L. Peary who appears in a TOEFL integrated writing question I’ve checked hundreds of times.  I can’t remember if it is from an ETS source of a third party source, but it questions whether or not he actually reached the pole.  The best part of this article is its depiction of the schemes of Arctic explorer Henry W. Howgate.
  • Decline and Fall is about concerns throughout history regarding decadence.   I’ve already added “the decadent movement” to my list of TOEFL speaking questions in the works.

Next, I checked out the February 2023 issue of the same magazine.  Here’s what I liked:

  • Vile Verse and Desperate Doggerel is about poet William McGonagall.  Was he the worst poet in history?  Was he a visionary?  You decide.  The article brings to mind an old TOEFl speaking question from ETS about “Outsider Art.”
  • The Land Between Rivers is about efforts to establish a steamship service down the Euphrates River in the 19th century.  It’s a long article.
  • The ‘Lost’ Emperor is about a mystery!  A pair of old coins were found that might depict a previously unknown Roman Emperor.   But maybe they don’t.  These coins have been studied.  People have opinions.  There are disagreements.  This would make a perfect Integrated Writing question!

I think I’ve got one more copy of “History Today” on my shelf.  I’ll probably write about it next month.

Meanwhile, I read the July/August 2023 issue of Analog Science Fiction and Fact.  I read every issue of this magazine, but I rarely mention it here because the stories and articles aren’t really available online.  I know it is a bit cliche, but I think the world would be a better place if more people read science fiction now and then.  There are bigger things to think about than the allegiances that divide us.  This month I really enjoyed David Ebenbach‘s “Everybody Needs a Conditions Box” which features the establishment of a colony floating above the surface of Venus.  That’s a topic that has appeared in TOEFL integrated writing questions (and I think I’ve mentioned other stories from the magazine that explore the concept).  This particular story also explores AI in a fun way.  Read it if you can find it.

I also read the October 2023 issue of Apollo.  I suppose it is important to read about art and architecture now and then, as those topics do show up on the TOEFL (and they are often ignored when people seek out “academic reading” material).  A few articles stood out this month:

Finally, I recently discovered a wonderful podcast called The Academic Minute.  This series features very short lectures on various topics by leading academics.  Each episode includes a short introduction and a transcript.  This is perfect practice for the TOEFL speaking section.  I feel like I am the last person to learn about this wonderful resource.  I think I will mine the podcast for topics I can use when writing practice questions.

New Article on Audio-Visual Presentations of Test-Taker Responses

Posted on by TOEFL Resources

“Language Testing” published a really wonderful article last week about a study of the impact that audio-visual modes of presentation have on rater scores of speaking samples.  In this study, raters were given speaking samples with and without video. According to the article, “raters were significantly more likely to score comprehensibility higher when the presentation mode was audio-visual.” This was true in cases when the raters were familiar with the speaker’s accent and also when they were not.

There is, by the way, quite a bit of discussion in the article about accent familiarity, which is worth reading if you aren’t familiar with arguments about how that impacts raters.

The authors conclude that “the study provides evidence that Accent Familiarity and Presentation Mode interact to potentially impact Comprehensibility ratings. Presentation Mode, as a separate variable, independent of Familiarity, is a unique potential source of variation. The results indicate that the audio-visual Presentation Mode has a stronger effect than Accent Familiarity on Comprehensibility ratings. The results suggest that semi-direct tests with audio-only presentation and audio-visual presentation should be evaluated to ascertain how Accent Familiarity and Presentation Mode affect Comprehensibility ratings.”

You’ll need some kind of institutional access (or a full wallet) to read this one.

ApplyBoard and British Council to Team Up

Posted on by TOEFL Resources

I read in a press release that ApplyBoard will partner with the British Council “to help bridge the gap between the testing and application phases of the international education experience.” It says that people taking the IELTS through the British Council “will receive personalised study-abroad matching through ApplyBoard.”

I don’t know exactly what that means, but it sounds fun. And is probably a great opportunity for ApplyBoard to further expand beyond its traditional niche of putting people in Canadian schools.

I like that the British Council is partnering with ApplyBoard, even though that firm is a direct competitor to their close pals at IDP Education.

I also appreciate that ApplyBoard is linking up with the IELTS, despite being owned, in part, by the folks behind the TOEFL.

Once, a very long time ago, I wrote a bunch of articles about the TOEFL for a partnership between Applyboard and that test. But the project was canceled and most of my articles are (presumably) still sitting in a virtual filing cabinet somewhere in southern Ontario. They were pretty good articles.

On Proctoring and the Duolingo English Test

Posted on by TOEFL Resources

Here is a nice new video about how the Duolingo English Test is proctored.

The video refers to the common misconception that the Duolingo test isn’t proctored by humans. It highlights how the AI is used during the test administration, and how three levels of human proctors are used following the test administration.

There are a couple of quick references to the possibility of “human error” in live proctoring (which is what I’ve been going on about in this space forever) but for the most part it talks about the test security benefits of the model.

As the testing industry (hopefully) moves towards more instances of this type of asynchronous proctoring, firms will have to do a really good job convincing stakeholders that the tests can be trusted. I was happy to see that Pearson did a good job describing their approach to test security in the materials accompanying the launch of the new Versant by Pearson test.

It won’t come as a surprise to most of you that there are people who ought to know better that have no idea that these sorts of tests include humans in the proctoring process.

https://youtu.be/JFMcCLt7Vr8

New Pearson English Test!

Posted on by TOEFL Resources

Our friends at Pearson have a new English test called the “Versant by Pearson English Certificate.”

At first glance, it seems like a test for people seeking to demonstrate their English proficiency for workplace purposes. I imagine that it is very similar to the “Versant by Pearson English Test” line of products which is used internally by businesses to assess the language skills of their employees.

There is a lot to like here.

The UI is very pretty. The test is cheap. Proctoring is done by AI during the test and by humans only after the test is finished. No appointment is needed. Test-takers are clearly told they should have 8 gigs of RAM. Complete results come in a couple of hours and are certified within two days. If the test-taker’s internet cuts out during the test they can finish the test as normal and have 24 hours to re-establish a connection and upload their work. There is a cool badge from Credly.

Perhaps Pearson views this test as a modern alternative to the TOEIC test. Will the test challenge the TOEIC in the same way that the DET has challenged the TOEFL? Probably. But that market will be a fairly tough nut to crack since it is so regional and entrenched (Japan and Korea alone probably make up 80% of total test volume). I guess that even grabbing a small share of the market could be considered a success since the TOEIC is really really really popular.

Berkeley Law accepts TOEFL MyBest Scores

Posted on by TOEFL Resources

I learned today that the LL.M program at Berkeley Law accepts TOEFL MyBest Scores. That’s a pretty good advertisement for the whole MyBest Scores thing, as a good number of people taking the TOEFL nowadays intend to apply to an elite law school.

Most of the Japanese students I work with nowadays hope to land at a good law school in the USA. Since their firm pays the fee, they sit for the TOEFL quite a few times. Eventually, this results in a pretty decent “MyBest Scores” line on the score report.

Live Proctoring Can be Frustrating

Posted on by TOEFL Resources

I’ve spent much of this month bellyaching on social media about the poor experiences people have with live proctoring of at-home tests.  And I’m not just talking about TOEFL.  I’ve read complaints from people taking various tests.  Indeed, the LSAT seems to be ground zero for test-taker complaints.  On LinkedIn I wrote many things, but concluded with:

Perhaps proctor switching is a pain point for some at-home tests. It seems to come up in test-taker reports with some regularity. Maybe this is an area which proctoring companies could focus on when training proctors. Or maybe they could ensure that individual instances are supervised by the same person all the way through.

Here’s one of my recent LinkedIn rants, which refers to the wall-of-text in the middle screenshot (below):

Staying on the topic of how students experience at-home testing and shoddy proctoring, here is a recent post from social media. The original post will be viewed by thousands of potential test-takers in the weeks ahead.

The post is a wall of text, so here’s a quick summary of what the poster alleges:

  1. The test-taker has a pretty good experience for most of the test. Their proctor is great.
  2. Part way through the test a NEW proctor takes over and begins speaking to the test-taker. Sadly, this happens during a part of the test which requires the test-taker to listen to a short lecture. The test-taker isn’t able to hear the lecture over the voice of the proctor.
  3. The next part of the test is another question which, again, requires the test-taker to listen to a short lecture. During the lecture, the proctor requests a scan of the test-taker’s room. This prevents the test-taker from hearing the lecture.
  4. Worst of all, the test-taker has a deadline and doesn’t have time to take the test again.

This hits on the two things I’ve been repeating again and again and again in this space. Badly done at-home testing can have a profound impact on the lives of test-takers. But it can also have a profound impact on the reputations of testing companies and on their bottom lines.

I’ve heard similar stories routinely over the past three years.

Business names are censored so people don’t think I’m picking on anyone; this is a problem across the board.

Meanwhile, a petition has been posted regarding live proctoring of the LSAT.  It says, in part:

There have been widespread issues for remote testers since Prometric took over proctoring a few months ago. Proctors have been interrupting tests to enforce rules that do not apply to the LSAT. Test Takers have complained about proctors frequently interrupting exams to tell them they cannot touch their face or that they must remove foam earplugs, even though test takers are allowed to engage in both behaviors. When we signed up for our remote tests, we even got instructions from Prometric that we were not allowed to have paper, writing utensils or a drink in our surrounding area when taking the test. These instructions are not correct as scrap paper and writing utensils are a crucial part of taking the LSAT. We are also allowed to drink water during the exam. So why is LSAC okay with Prometric sending us instructions that are clearly incorrect?

Below are some recent complaints, which I gathered in the span of about a week. One can find similar complaints on places like Reddit and Twitter every day of the week.

TOEFL Home Edition Oddities

Posted on by TOEFL Resources

There was some weirdness coming out of ETS earlier this month. ETS Japan (a subsidiary of ETS) put out an “alert” about the TOEFL Home Edition on Twitter and its website.  I mentioned that on LinkedIn and how I found it puzzling.  The alert was quickly removed. 

If you are curious, here is what puzzled me about the alert:

  1. It recommended system requirements quite higher than those on the ETS website and suggested that interruptions could occur if they are not met.
  2. It recommended things for test-takers to avoid during the test that are not mentioned on the ETS website (though some are buried in the TOEFL Bulletin).
  3. It mentioned that since 2022 testing has become more stringent due to cheating in some countries, and therefore some test-takers may not even realize they have broken a rule.
  4. It recommended that people taking the TOEFL iBT for the first time opt to take it at a test center.

Now, that’s all very good stuff. Indeed, it is similar  to what I tell test-takers who ask for my opinion.  That said, the last part isn’t the most ringing endorsement of the product.  Nor is the third part, I suppose.

There was also stuff in there about test-takers only getting help from ETS Japan if unable to resolve a problem via the US office of ETS. One senses that the announcement came as a result of a certain number of complaints from local test-takers.  Perhaps it came from a feeling of frustration.

I’ve written many times about how at-home test takers have been experiencing high levels of frustration.  I’ve also expressed my feeling that testing companies are not doing all they can do (and should do) for their customers.  If you are new to a testing company and have been tasked with figuring out why your test’s market share is way below 2019… well… this is a big part of it. This isn’t a dig at ETS, as I think it is a problem with all testing companies, to some extent.

But we are on the topic of ETS, for the moment.  One of the traditional criticisms of that organization (see: David Lewis, “None of the Above”) is that entrenched as they are on 400 acres of bucolic New Jersey countryside, they don’t always have a clear picture of how test-takers in the outside world are experiencing their products. And sometimes I get the impression that there is insufficient communication between various departments and offices of the organization (hence a quickly deleted “alert” about their most popular test).

Anyway. The point of all this is to suggest that perhaps some of the people in the TOEFL program who don’t usually deal with complaints from test-takers could call up the folks in Japan and ask what prompted the alert to be posted.  And, moreover, discuss what the head office can do better moving forward.

Most TOEFL Test-Takers in India Planning Graduate Studies

Posted on by TOEFL Resources

PTI published one more interesting bit of data about the TOEFL test in India. According to a report published in the Deccan Herald:

“The data showed that the percentage of Indians taking TOEFL to attend graduate or postgraduate programmes has increased from 70.84 per cent in 2021 to 71.87 per cent in 2022.”

While it is true that about 100% of the Indian test-takers I communicate with are planning to take the test for graduate and postgraduate programs, I really didn’t expect the official figure to be that high. Where are all the college-bound students? And the undergrads? Over at the nearest IELTS test center, I guess.

As I’ve expressed a few times here, the breakup of the IELTS monopoly on language testing for the Canadian SDS is really good news for the folks at ETS. About 350,000 Indian students applied for Canadian study permits last year. I don’t have the statistics, but one imagines that most of them applied through the SDS. A majority of those students are bound for colleges. They represent a market that ETS has not, up to now, tapped into.

With the right marketing and the right attention to customer experience the TOEFL test could expand quite a lot beyond their current niche. And, as a bonus, ETS doesn’t need to compete with the folks at Duolingo, who are very good at marketing and customer experience.

Research into DET/TOEFL/IELTS Score Concordance (Conversion) Tables

Posted on by TOEFL Resources

Now in open access at “Language Testing” is an article by Ramsey Cardwell, Steven Nydick, J.R. Lockwood and Alina von Davier on building concordances between English tests. The article describes the unique method they used to create a concordance table for the DET, TOEFL and IELTS tests. Worth reading, if you are into that sort of thing.

Concordance tables can be amusing. Certain institutions are somewhat lazy about updating their score requirements. Though the folks at Duolingo updated their concordance tables back in August of 2022, many instructions haven’t bothered to adjust their requirements to match them. Pearson updated the concordance tables for the PTE back in 2020 and, again, many institutions haven’t updated their requirements. The tables had to be changed since the tests themselves had changed… but some institutions now have score requirements that are probably lower than they think they are.

Meanwhile, as I’ve noted here, the score requirements for Canada’s Student Direct Stream don’t seem to match any score concordance table I’ve ever seen. Perhaps the folks at IRCC are depending on a score conversion that hasn’t been made available to the public.

I’ve got my fingers crossed that the REOI from the Australian Department of Home Affairs will nudge ETS into producing a new concordance table for the TOEFL and IELTS test. The current one seems to be based on data gathered in 2008 and 2009 and the content and grading of the TOEFL test has changed significantly since that time.

ETS Buys Wheebox!

Posted on by TOEFL Resources

I read on the Internet that ETS has acquired Wheebox, a remote proctoring firm. I’ve written here before about how testing firms really ought to do proctoring in-house, so this makes me happy. Obviously it is too soon to tell if Wheebox will eventually replace Meazure Learning/ProctorU as the proctor of ETS’s major tests, but one imagines that it is a possibility.

Longtime readers will remember that ETS purchased Prometric shortly after the launch of the TOEFL iBT. Though ETS took a very hands-off approach to Prometric, that purchase may have played a big part in ETS’s remarkably smooth transition to the new version of the test. It also turned out to be a smart business move, as Prometric was sold for more than twice its purchase price about a decade later.