Individuals on social media have recently reported having their at-home TOEFL tests proctored by Wheebox rather than ProctorU. These instances seem to be limited to tests scheduled after August 5.

Wheebox is a subsidiary of ETS (acquired in 2023) based in India.

As regular readers know, most of ETS’s at-home tests are currently proctored by ProctorU, a part of Meazure Learning, as part of a contract that runs through 2028. According to ETS’s 990 filings, for the year ending September 2023, ETS paid that firm $25.1 million dollars to provide proctoring services (for various tests).

Despite these reports, I understand that Wheebox is not replacing ProctorU as the primary proctor of the at-home TOEFL at this time.

I returned from vacation to find – what else – a few more complaints about the TOEFL Test.  I’ve attached some screenshots.  I know I sound like a broken record at this point, but I think this is worth paying attention to.

These reports, though shared on different platforms, seem to be about the same case.  You can see that the test was canceled due to what’s called a “jagged score profile.”  This means that the test taker performed quite well in one or more sections of the test, and quite poorly in some other sections of the test.  They also completed at least one section of the test faster than some other section(s).

In cases like this, the result is deemed “invalid” and the test result is canceled.  No refund is offered.  No opportunity to take the test again for free is offered.  There is no appeal.

Keep in mind that no evidence of cheating or malpractice on the part of the test taker is mentioned.  All that is mentioned is validity.

One of the screenshots notes:

“You don’t even have to be a linguist to see why this is utter nonsense. L2 learners *can* have wildly different levels of skills across areas, not to mention other factors like neurodivergence. This is at best discriminatory, and at worst a scam.”

TOEFL test takers have been complaining about this for some years.  I’ve highlighted it in this space many times.

Obviously, if a test score is invalid, it must be canceled.  It must not be sent to recipients.  My objection is to the idea that the test taker is forced to pay a second test fee in order to prove their ability.  Again, keep in mind that in these cases no concerns about impropriety are expressed.  The expressed concerns are merely in regards to validity.

In 2022 and earlier when a test score was canceled due to validity concerns, the test taker was given three options: a refund, a free re-test or an appeal.  In the case of an appeal, they could provide documentation explaining why their score profile was “jagged.”  A panel convened by ETS would review the evidence.  I sometimes assisted test takers in the preparation of this document.  One time I even helped a test taker win the appeal.

This was how ETS handled such cases for decades. Even as far back as the case depicted in the Academy Award nominated film “Stand and Deliver.”

In late 2022, all three options were eliminated.  No one has ever explained why that happened.  I suspect no one remaining at ETS today has any idea. It sounds like “the curse of the MBA” to me, but I could be wrong.

ETS is spending a large amount of money on marketing and rebranding.  I think those efforts will be unlikely to bear fruit due to the frequency with which concerns like these are shared on social media.

Stuff like this is part of the reason why Duolingo and Pearson have been able to grow so rapidly in recent years.  As I noted in a private conversation a few days ago:  Duolingo is loved.  PTE is liked.  IELTS is tolerated.  TOEFL? Well, that’s another matter.

Duolingo has just reported its financials for the second quarter of 2024. Revenue for the Duolingo English Test was 10.7 million dollars, up from 9.8 million in Q2 of 2023.  Note that the price of the test increased from $59 to $65 in this time frame.

If we divide the revenue by the per-test cost, we find that the DET was taken about 164,500 times in the quarter, about a 1% drop from last year.  Note that this is a crude estimate as some test takers pay more than $65 (to get faster results) while some test takers pay less (by purchasing a bundle).  Others take the test via Duolingo’s Access Program, and get vouchers from receiving institutions.   A more precise number of test takers (for 2023) was given at DETcon 2024, but I suppose that is covered by my NDA.

This flat quarter comes after a very strong Q1.

Using the same math as above, we can estimate that DET’s test volume is up about 5% in the first two quarters of 2024 (compared to the same period in 2023).  Perhaps I’ve buried the lede here as Pearson reported a 10% decline in test volume during the same period, and IDP Education is expecting a 15-20% decline in the volume of IELTS test takers for the year ending June of 2024.

It seems that growth is still happening at Duolingo, but we must pay careful attention to the final six months of 2024.

A fellow in Korea was sentenced to three years in jail for helping people cheat on the TOEIC. His scheme involved taking pictures of his answers during the break following the listening section and sending them out to his clients from a bathroom stall. It is more involved than that, but it involves a very real likelihood of many people cramming themselves into a single bathroom stall and I don’t want to write about it in detail.

Regular readers know that the TOEIC is big business in Korea and Japan (and a few European markets). Indeed, the TOEIC may be the most widely taken standardized English test in the world. I once had a job in Korea where I sat at a desk in Gangnam and wrote TOEIC practice questions for eight hours a day. Every day. For almost a year.

It is worth mentioning that test centers are not quite as secure as they ought to be. When Duolingo or Pearson or someone else convinces governments to accept their at-home tests for immigration purposes, the pitch will be (in part) that their approach is actually MORE secure that traditional test center testing. They will probably be correct in that assertion.

The IELTS partnership is picking up the pace! According to a new press release, people who take that test on a computer can now “usually” get their results in 1-2 days. That’s impressive for a test that is graded wholly by humans.

For the record, Pearson says that PTE scores “typically” arrive within two days. Duolingo says that DET scores arrive in two days (and no adverbs are used).

ETS still promises TOEFL scores in 4-8 days. They might have to speed things up in order to remain competitive.

Pearson just posted interim results for the six months ending June 2024. I’m traveling right now and can’t provide a full write-up, but a few things ought to be quickly noted:

  1. PTE’s volume dropped to 546,000 for the 6 months, down 10% compared to the same period in 2023. Pearson predicts “flat to down” growth for the remainder of this year. Note that IDP has suggested IELTS volumes will be down 15 to 20 percent for the year ending June 2024. It’s a tough market for English tests right now.
  2. They note that the PTE “has continued to see market share gains, particularly in India and China.”

A commenter who prefers to remain anonymous wrote the following when I posted about the results to LinkedIn last week:

…if 100 students who [took] an English Language Test in May 2023, 80 of them would be for the IELTS test and 20 for the PTE test.

However in May 2024 the pattern has actually reversed with more people preferring the PTE exam than the IELTS.

This is the situation that I’ve seen in Kerala, India.

Here’s a gallery of recent social media complaints from people who are getting stuck on the “search service” screen when trying to register for the TOEFL test. I wrote about this weeks ago. I first read about it months ago.

I’ve always been struck by how bugs like this are able to linger for weeks or months… or even years. To my eye, ETS seems much slower to fix things than other firms.

This particular bug should be shunted right to the front of the queue as it impacts what has emerged as one of ETS’s most lucrative niche markets – individuals who must take both the GRE and an English test for graduate studies.

ETS has made great strides in recent months when it comes to creating a fun social media profile. Regardless, the company’s rep on socials remains a periwinkle-hued nightmare. One doesn’t have to look too far to find complaints about registration bugs, score cancellations, unresponsive customer service and unhelpful proctors.

As a fun bonus, I’ve included a complaint from a fellow complaining that his proctor forgot to turn off his microphone and unknowingly interrupted the speaking section of the test.

In case you are curious what a “Skills for English: SELT” Test Report Form looks like, here’s mine. A couple of points are worth mentioning:

  1. My results arrived in about 48 hours. Quick!
  2. While “Skills for English” is a pass/fail test for specific CEFR levels, the report indicates that I achieved B2 “with merit.” I suppose that the “with merit” indication is a way for high-performing test takers to show off a bit for score users. That’s a nice touch. I haven’t seen this documented anywhere.
  3. The mug shot is huge. I was photographed by the test center administrator himself, which I think is a feature of the SELT requirements.
  4. The report features the signature of Isabelle Gonthier, PhD, ICE-CCP, who hosts one of my favorite podcasts about this sort of thing. How nice!

I’ve also attached a screenshot of my blissfully simple and straightforward account page for this test.

You can also read my full report on this test.

Continuing along with the Norton Library Podcast, this month I read Jane Austen’s “Emma.”  You can find the podcast episodes on Youtube.  I haven’t read a huge amount of Austen’s work, but since I enjoy the whole “comedy of manners” genre, she has always been on my to-read list.  Interestingly, this story of the “idle rich” ties in nicely with a book I’ll write about next month (spoiler:  it’s “The Meritocracy Trap”).  Let me know if the comments if you’d rather be rich and busy or rich and carefree.

Meanwhile, I spent my time digging into TOEFL books.  Most importantly, I finished my examination of changes in the latest editions of the three official TOEFL books from ETS.  You can find my blog posts below:

As I’ve indicated on the blog a few times, I’m really happy with these new editions.  They have been updated to match the new version of the TOEFL, and great number of small errors and annoyances have been removed.  Buy them!

I also continued my exploration of all the random TOEFL stuff that pops up in places like Hoopla.  This month I read Jackie Bolan’s “TOEFL iBT Reading Practice.”  I posted the following review on Goodreads:

Not a fan of this one. The cover says “TOEFL iBT Reading Practice” but the content is just general academic reading practice. The articles are superficially like what one would find in the TOEFL reading section, but the questions are not even close.

The cover says that this book might help one to “master the reading section of the TOEFL Exam.” While it is true that it might improve one’s academic reading skills in a general sense, mastery requires more accurate practice materials.

Following that, I read Daniel Smith’s “TOEFL Words Vocabulary Enhancer.”  My review went like this:

This book contains about 4000 words, each with a very short definition and sample sentence. There isn’t anything specific to the TOEFL here. There aren’t any practice TOEFL questions. This probably isn’t the best way to study vocabulary for the TOEFL, but I guess it could be used as a supplemental test prep source.

I also read Smiths “TOEFL Grammar with Answer Key” and wrote:

Not recommended. The book is just 1300 multiple choice grammar and vocabulary questions. There is no categorization, no answer explanations and no lessons. Just a bunch of random questions.

There is nothing that connects this content to the TOEFL, except for the fact that improving your grammar will increase your writing score. But if your goal is to improve your grammar in a general way, it is better to reach for something like “English Grammar in Use” from Cambridge.

That’s all for this month.  I’ll be back in August with proper non-fiction and academic stuff.  Stay tuned.

 

 

Ten years ago today, Duolingo launched a project called “Test Center.” Shortly thereafter, “Test Center” morphed into the Duolingo English Test.  That test is now taken more than 800,000 times per year.  That’s quite an accomplishment.

Many people think of the DET as a fairly new product, but ten years is a long time. When I write about the legacy testing firms being technologically far behind their competition (Duolingo and Pearson, mostly) I mean that they are more than a decade behind.  It will not be easy to close that gap. It may be impossible.

Here’s a TechCrunch article from 2014 where the project was announced.  The message of 2014 is similar to the message of today – testing monopolies are bad for test takers and are prone to fraud.

The success of the DET is mostly due to the work of some very bright researchers, engineers and psychometricians (former ETS employees, in many cases).  But the single-mindedness of Duolingo co-founder Luis von Ahn seems to play a pretty big part.  That guy does not like the testing monopolies at all.

The DET is now a major player in the admissions process for American universities. The monopoly once enjoyed by ETS in that market has come to an end.  As recent reports have indicated, times are tough at ETS.

Currently, the folks at Duolingo are heavily promoting their test to students headed to the UK, Australia and Canada.  These efforts are partly why IDP Education recently announced that IELTS test volumes will decline by 15-20% in fiscal year 2024.

Earlier this year I helped a student prepare for the ALP Essay Exam. I couldn’t find much information about the test online, so I thought I would write a few notes here.  I might revise this post in the future, so stop by in the future for updates. If you need tutoring for the ALP Essay exam, you can contact me.

What is the ALP Essay Exam?

The ALP Essay Exam is used by Columbia University to assess the writing skills of students.  It is often used to determine if students have the language skills necessary to take classes at the university. It can also be used to determine if students should take supplementary writing classes (in addition to their regular schedule of classes). Test-takers have 105 minutes to write a standard (four or five paragraph) argumentative essay about a specific topic.  The essay must be based on the contents of two short academic articles.

You can read about it over here.

What Does the ALP Essay Exam Look Like?

You’ll get a question about a serious topic.  Don’t expect something basic and simple like the IELTS.  Instead, expect something that might actually be studied in a first-year university class.  You might get something about gentrification, affirmative action, the use of standardized testing… that sort of thing.  The question might look like this:

“Please read the two passages below.  The authors have differing opinions about the topic of gentrification in the United States. Which author do you agree with, and to what extent?  In your essay you should support your opinion, and challenge the opinions of the author you disagree with.  You have 105 minutes to complete your essay.”

The passages should be fairly short.  Maybe just a paragraph or two, excerpted from a longer article.  They will have opposing opinions on the same topic. The author of each one will be credited

If the topic is gentrification, they might look like this:

“One of the most significant benefits of gentrification is the improvement of housing. Ordinarily, housing presents enormous challenges in the management of urban centers. Therefore, gentrification seems to solve this challenge because it favors the improvement of housing within the gentrified community. In addition, it is believed to stabilize declining areas. In most cities, suburban areas are known to experience degradation leading to the emergence of slums. This phenomenon is caused by the increased strain on urban infrastructure and services. Therefore, gentrification addresses an array of urban management challenges by reducing suburban sprawl and strain on the existing infrastructure.

Another positive effect of gentrification is the increase in property values. As a result, property owners reap high income from real estate investment, and this serves as a means of attraction for potential businesses. It is also suggested that gentrification leads to a significant increase of local fiscal revenues. Moreover, gentrification has led to the rehabilitation of property with little state sponsorship. Therefore, an increase in property values and local fiscal revenues promote economic development of gentrified areas. Economic development is also enhanced by an increase in purchasing power in the centralized economy, although it is uneven.

It is also believed that gentrification leads to increased social mix and reduction in crime rates. This phenomenon has been evidenced in gentrified cities such as London, Atlanta and Washington, DC.

-Caroline Mutuku

and:

Gentrification usually leads to negative impacts such as forced displacement, a fostering of discriminatory behavior by people in power, and a focus on spaces that exclude low-income individuals and people of color.

During gentrification, poorer communities are commonly converted to high-end neighborhoods with expensive housing options such as high-rises and condominiums. As property prices increase, the original residents of the neighborhood are forced out in a variety of ways. First, with an increase in the prices of buildings, the gap between the price of the building and the income that the landlord gets from renting the building grows bigger; landlords thus increase rent prices, which forces out the low-income residents. As building prices continue to increase, the problem exacerbates because it becomes even more profitable to convert these apartment buildings into non-residential areas. Additionally, since investors can earn more money from selling buildings, real-estate dealers have less incentive to improve the buildings. The real estate dealers instead sell the buildings at higher prices. This cycle of rising building prices continues until only large and well-financed investors are able to continue.

Displacement… is disproportionately borne by low-income individuals of color, many of whom are elderly individuals.  Physical frailty makes it more challenging for elderly individuals to resist the actions that landlords take to remove tenants. Researchers have also found that elderly people are more intensively affected by social changes around them; for example, many older adults cited loss of friendships or community networks as a reason to move. 

-Emily Chong

How to Structure the Essay

The structure is fairly easy.  Write an introduction that provides some background on the topic and a clear thesis statement that states your opinion on the topic.  Then write two or three body paragraphs.  Each one should focus on a specific argument in support of your argument or the rebuttal of a specific point in the article you don’t agree with.  Finally, write a conclusion that sums of what you’ve just created.  Aim for 400 to 600 words in total.  Easy, right?

How to Get a Good Score

Getting a good score isn’t so easy.  To award you a high score, the rater needs to see an argument, but they also need to see the use of fairly sophisticated writing techniques.  The list below is drawn from the official ALP website, and a few other sources used in ALP classes at Columbia.

Remember that your essay must also quote from the sources when appropriate.

Remember, also, that in addition to this advanced stuff, your essay needs to show mastery of basic stuff.  That means basic transitions (therefore, however, in addition) and a mix of all three sentence types (simple, compound, complex).  You also need nearly perfect grammar to get a high score.

Sample Paragraphs

I can’t teach you the basic stuff here, but I can show you examples of the advanced concepts mentioned above.

Here’s a sample paragraph from an essay I wrote about mental health.  I’ve underlined parts that use the above techniques.  In order, they are: parallel structure, using the article, appositive, noun clause in subject position, inversion. 

Young people are able to discuss their mental health challenges with others, and are willing to reach out for help when necessary. As the article by Smith indicates, 62% of millennials are comfortable with this. Proof is easy to find. Many organizations have taken up the suggestion of the Center for Workplace Mental Health and created departments which help workers cope with issues as they arise. In addition, employee benefits now include financial support for outside counseling and psychological care.  Even more indicative of this trend  is the recent emergence of businesses which profit from the desire that young people have to discuss their mental health. Several new smartphone apps, services jokingly referred to as “Uber for Counseling,” have made a lot of money connecting people with therapists. With just a few clicks, we can be connected with a therapist and receive their assistance via voice or text. The benefits are clear; when people are willing to talk about issues that challenge them, and there are people willing to listen to them, they can be given strategies that mitigate the negative effects or perhaps eliminate the issues altogether. Rarely do people today find themselves in an environment where they have absolutely no one to turn to.  This is quite a shift from even just a few decades ago, when sufferers of mental illness often felt lost at sea.

Next is part of a paragraph about reparations.  I’ve underlined an example of fronting, and an example of an appositive.  Note the extensive quotes from the article, which are integrated into my own sentences.

While long-term solutions to today’s problems must certainly involve political and economic changes, the political and economic systems are slow to change. With great enthusiasm, conservative journalist Frank Williamson says that “the political interests of African Americans… are best served by equality under the law.” Williamson, an experienced political writer, knows that politicians have been working towards “equality under the law” for decades, and are still far from achieving it.

Here is an introductory paragraph from an essay about inclusive language.  Note how I’ve underlined a parallel structure, fronting, and another parallel structure.  Note that I ended with a clear thesis statement.

They say that people change over time, and that language changes along with them. Nowadays, thanks to the spread of the Internet, language seems to be changing at a more rapid pace than ever before. Rarely do we go a week without reading an article or seeing a social media post that uses a term or phrase that is totally new to us. Many of us want to be supportive of marginalized groups, and we want to express our opinions clearly without being lost in a sea of jargon. Personally, I feel that our choice of words is very important, but we must be careful to avoid being overly judgmental of people who can’t keep up with the newest words.

Wrapping Up

Okay, so that’s a broad look at what the ALP test looks like and what you need to do.  For more help, or tutoring, feel free to contact me. To keep up with the latest changes to this test, contact Columbia University.

 

As part of an ongoing mission to expand my knowledge of testing options in 2024, I took the B2 version of the Skills for English: SELT Test yesterday. While everything is still fresh in my mind, I’ll share a few thoughts. 

Before I begin, I should mention that PSI gave me a voucher to take this test at no cost.

Want me to look at your test? Just let me know! I’ll take any test if it is cheap or I can get a freebie.  And if you are preparing for this test and need assistance, feel free to reach out.

As part of an ongoing mission to expand my knowledge of testing options in 2024, I took the B2 version of the “Skills for English” test yesterday.  While everything is still fresh in my mind, I’ll share a few thoughts.  Before I begin, I should mention that PSI gave me a voucher to take this test at no cost.

Want me to look at your test? Just let me know! I’ll take any test if it is cheap or I can get a freebie.

Note that I took the B2 version since it is likely the most widely-taken version of the “Skills for English” suite of tests. This is the level one needs to demonstrate in order to get a UK student visa for studies at the degree level. Note, also, that like the other versions, this is a pass/fail test. You either prove that you are at the B2 level, or you don’t.

Okay. Here are my initial thoughts:

  1. The registration process is fairly straightforward and painless.  This is an area where the newer tests seem to outshine their “legacy” competitors. Younger test takers really don’t like a test registration process that feels like buying an airplane ticket. They want to get it done quickly!  I’d love to see the stats (for all of the tests) on how many would-be test takers abandon their carts before making a purchase.

 

  1. An adequate amount of free test prep is provided by PSI.  Accessing that material can be tricky, however. It involves passing through an email funnel that results in a link to the “PSI Online Store,” which doesn’t seem to contain the desired materials. This certainly frustrated me.  If anyone from PSI wants to know more about this issue, just reach out.

 

  1. Here in Korea, “Skills for English” costs a bit less than the IELTS (UKVI) and a bit more than the PTE-Academic (UKVI).  It is offered at two test centers in the country.  I was alone when I took the test, so perhaps there isn’t much demand for it in Korea at this time.

 

  1. There is quite a lot of speaking in this test.  Test takers have the opportunity to speak for just over ten minutes.  Spoken responses to test items range from 30 seconds to 5 minutes. This approach is common among the newer tests and I think it is is appealing to test takers who desire an opportunity to “bounce back” from a single bad answer.  Compare this test to the TOEFL iBT, which involves just three minutes and forty-five seconds of speaking.  Many individuals taking that test fear that a single flubbed response might heavily impact their final score.

 

  1. This is advertised as a 190-minute test.  That’s longer than most tests in 2024.  The length could certainly turn off some prospective test takers and I’m surprised that the folks at PSI have maintained it.  That said, students at the C1 or C2 levels will probably not use all of the time given in the reading and writing sections and perhaps lower-level students will appreciate the sometimes leisurely pace of the test.

 

  1. The test includes a hard cap on the length of written responses.  That is, once the test taker hits the maximum word count, they aren’t allowed to write any more.  I like that.

 

  1. My test form contained a whole lot of Britishisms.  Knowledge of these won’t impact one’s score, but it seems like the folks at PSI went out of their way to put in as many as possible in the reading and listening items.  One of them made me laugh out loud. Another made me wonder “do they really say that across the pond?!”

 

  1. Preparing students for this test might involve teaching them how to paraphrase or reformulate ideas.  On my test form, picking correct listening and reading answers often depended on knowing that the words used in the answer choice (or gap-fill) expressed the same thing as different words used in the reading or listening content.

 

  1. The test content is mostly general English, with a small amount of academic English.  This is a quirk of the SELT program, I think.

 

  1. I chatted with the test center operator about the tests people pop in for.  He mentioned that he does a ton of PSI tests, including ones for accountants and pilates instructors.  This reinforced my firm belief that PSI of 2024 is just like ETS in the golden era.  No wonder they made the purchase!