I read a couple of books about tests this month.  They might not be particularly interesting to TOEFL students, but teachers who read this blog might enjoy them.

First up, I read Escalante: The Best Teacher in America by Jay Mathews.  It describes how Jaime Escalante prepared underprivileged students in East Los Angeles  to take the AP Calculus Exam in the 1970s and 80s.  Escalante’s unique approach to this task yielded amazing (and unprecedented) results.  I don’t know if his techniques would work forty years later, but this is a great book for anyone interested in teaching and in the value of testing.  It is also a great advertisement for the AP program, which continues to this day.  Readers might also benefit from its sketch of how public schools operated in LA during those decades.  They faced challenges then, and they face challenges now.

One incident in the book stood out to me.  In 1982, ETS (yes, ETS) determined that several of Escalante’s students may have cheated on the test.  Their suspicious were due in part to a controversial mathematical analysis called a “K Index.”  They were told that they could do one of three things:  cancel the test and get a refund, take the test again, or submit additional information.  The students were told that if they provided additional information, it would be reviewed by a panel of three ETS officials.  They would only have to convince one of the members of the panel to have their scores restored. Or they could turn it all over to the American Arbitration Association.

Skip ahead to 2022, and that’s almost exactly what some students are told when ETS challenges their TOEFL scores.  The mathematical analysis is different of course, but everything else remains the same.  Funny, that.

Next, I read Michael Young’s “The Rise of the Meritocracy: 1870-2033”   Young was a sociologist who coined the term “meritocracy.”  But this book is not in praise of meritocracy, as most are.  The book is actually a dystopia that uses the rise of intelligence testing in the 1940s and 50s as its launching point.  Young tracks a fictional history of the United Kingdom as it slips into a more and more segregated and caste-like society due to its emphasis on “merit” above all else.  It is an interesting thought-experiment.  Especially in 2022 when the general consensus seems to be that meritocracy is always a good thing. I think there is something in here that explains part of our current political chaos, but I’m going to keep the blog politics-free for now. But for more on this topic, check out this debate on IQ2.  Or my review of “The Big Test” a few months ago.

 

 

A federal judge has ruled that it is unconstitutional for  to require test-takers to provide a scan of their room (via camera) before beginning a remotely proctored test.  According to a report by NPR:

Aaron Ogletree, a chemistry student, sat for a test during his spring semester last year. Before starting the exam, he was asked to show the virtual proctor his bedroom. He complied, and the recording data was stored by one of the school’s third-party proctoring tools, Honorlock, according to the ruling documents.

Ogletree then sued his university, alleging that the room scan violated his Fourth Amendment rights protecting U.S. citizens against “unreasonable searches and seizures.” In its defense, Cleveland State argued that room scans are not “searches,” because they are limited in scope, conducted to ensure academic fairness and exam integrity, and not coerced.

U.S. district court Judge J. Philip Calabrese on Monday decided in Ogletree’s favor: Room scans are unconstitutional.

I do not know if this will affect how the TOEFL Home Edition is proctored.  I’m not a lawyer.

“Language Testing” just published an article about using the TOEFL and IELTS tests to predict academic success at university. The authors’ meta-analysis of 32 studies indicates that there is a weak positive correlation with academic achievement and no significant difference between the predictive power of each test.

Note the following conclusion, though:

Although significant, the overall correlation was low; thus, practitioners are cautioned from using standardized English-language proficiency test scores in isolation in lieu of a holistic application review during the admissions process.

Perhaps the test creators would agree that the tests should be used only to assess a test-taker’s ability to use academic English, and not be used to predict their academic success. Indeed, there are other tests which can be used for the latter purpose.

ETS just announced a couple of deals for students in the USA.  You can read about them in their press release.  Here’s a summary:

  • Students who register to take the TOEFL iBT Paper Edition will get a voucher to take the test a second time for free.  Complete your registration before November 30. Read the terms and conditions.
  • Students who register to take the GRE will get a $40 discount on their TOEFL iBT registration.  Read more about the offer here.

The waiting period for TOEFL iBT scores changed in July.

Formerly, when the test was taken at a test center, scores would arrive “6 days after your test date.” When the test was taken at home, scores would arrive “6 to 10 days after your test date.”

Now, scores for both arrive “4-8 days after your test date.”

Those are all calendar days.

Scores from the paper test still take 11-13 business days to arrive.  

Check the TOEFL website for more information.

Sometimes when people attempt to make an account on the ETS website to register for the TOEFL or GRE they get this error message:

EIAS System error occurred – unable to process the request.

A user finally figured out what causes this.  The error occurs when certain special characters are used in your password.  To eliminate the error, only use the recommended special characters:  !@#$%^&*() .  Using other characters may cause the error to appear.

If this solution works for you, please leave a comment below.

Here’s a screenshot of the error:

 

Recent changes to the Duolingo English Test to TOEFL score conversion table have inspired me to begin a new series of blog posts.  Over the next year I’ll track whether schools adjust their score requirements in light of these changes.  In the chart below are 17 schools across the United States and Canada.  Also listed are their required Duolingo and TOEFL requirements as listed on August 11, 2022 and in the spring of 2022.  I used the Wayback Machine to get the older scores, so they are not all from a specific date.  I’ll check in with these schools now and then to see if the requirements have been adjusted.

A few things are worth mentioning:

  • Update:  I found a couple of schools in the UK.
  • There are a couple of community colleges at the bottom of the list.
  • The schools were selected mostly at random.
  • I have not listed IELTS scores, but I will create Wayback Machine archives along the way so perhaps those can be added later.
  • Many schools still list TOEFL PBT and CBT requirements.  Yikes.
  • I can’t think of any licensing boards or non-academic institutions that take DET scores.  But if you can think of any please let me know.
  • All scores are for undergraduate admissions.

Before we begin, note that:

  • So far, none of the tracked schools have updated their requirements.
  • However, the University of Pittsburgh recently updated its DET requirement!  The TOEFL requirement is currently 100, while the DET requirement is 125.  In don’t know exactly when it was adjusted, but in March of this year the requirement was 120.  This should not come as a surprise, as I believe that a relationship exists between that school and Duolingo.  In any case, the campus is within walking distance of Duolingo HQ.
  • A few of the schools have fairly high DET requirements and don’t need to adjust them to match the current conversion chart.  But most of the schools may wish to consider making adjustments to match the chart.

If any readers of the blog have an attachment to a particular institution, I will be happy to add it to the tracker.  Just leave a comment below.

Enjoy.

School

Spring 2022

DET / TOEFL


August 11
DET / TOEFL

MIT

120 / 90

120 / 90

U of Toronto

120 / 100

120 / 100

Cornell

120 / 100

120 / 100

UBC

125 / 90

125 / 90

Emory

120 / 100

120 / 100

U of Arizona

100 / 70

100 / 70

Carnegie Mel.

125 / 102

125 / 102

Brown

125 / 100

125 / 100

U of Utah

105 / 80

105 / 80

Rice

120 / 100

120 / 100

UCLA

120 / 100

120 / 100

Columbia

125 / 105

125 / 105

Dalhousie

115 / 90

115 / 90

City College of SF

85 / 56

85 / 56

De Anza College

95 / 61

95 / 61

Imperial College London

115 / 92

115 / 92

U of Chichester

95 / 79

95 / 79

 

Recently, the charts that convert Duolingo English Test Scores to TOEFL and IELTS scores were adjusted.  You can see the current charts right here.  An archive of the old charts is available via the Way Back Machine. 

For instance, previously a Duolingo English Test score of 125 converted to 103-107 on the TOEFL iBT.  Now, that converts to 93 -97 on the TOEFL iBT.

Likewise, a score of 125 on the Duolingo used to equal an IELTS score of 7.5, but now that is equal to an IELTS score of 6.5.

There are also changes to the Duolingo to CEFR conversion chart.

I suppose this is due to the inclusion of new question types on the DET.  I don’t know if any institutions have adjusted their score requirements at this time.  I suspect that many institutions are totally unaware of the change.

To illustrate, here is the top of the current Duolingo to TOEFL chart:

And here is the top of the old chart (forgive the broken image):

 

 

Update: It is working again!  Everyone can register.

I think the TOEFL registration system has stopped working again.

When I try to register for the TOEFL iBT Home Edition Test I get this error:

“No available slots in this time range” 

others have reported this error:

“The test center and/or test date/time are no longer available” 

I think it is still possible to register to take the test at a test center.

People on social media are reporting the same thing right now for the past 24 hours.  I think that the same error has been reported in the past.  It seems to get fixed eventually.  

Yesterday I wrote about ways to improve the pre- and post-test experience for test-takers.  I suppose “improve the backend” could be added to that list.  It seems to be that registration for the TOEFL is quite often offline for maintenance, or offline because of a mysterious error.  

Update: People on social media have indicated that it is not possible to register for a test at a test center at this time.

Update 2: Users attempting to register for the GRE get the same error.

 

As promised, here are my thoughts about how ETS can help test-takers to have an enjoyable experience throughout their TOEFL journey. Since I’m not qualified to speak about the content of the test, this article will focus entirely on the pre- and post-test experience.

Keep in mind that I’m not trying to pick on my friends at the Educational Testing Service.  I am a great admirer of that organization, its people and its mission.  But ETS is seems to be going through a period of transition (into something more streamlined and responsive) so perhaps my thoughts can be of use now. Needless to say, I’ve hesitated to share such ideas in public in the past.

Note also that these suggestions don’t just come from my experience.  They come from the opinions of the thousands of test-takers I’ve worked with directly, and from the comments of the thousands more who have contacted me via the blog and on social media.

Here goes.

  •  The TOEFL website may be considered old-fashioned. Key information is hard to find.

In the “For Test Takers” section of the website, critical information is spread across approximately 90 static pages, 17 embedded videos, a dozen PDFs and a few ZIP archives.  This is a somewhat dated approach to website design.  As a longtime user of the site, it seems like various UX elements have been grafted onto an original frame dating from 2011.

Consider the path a student must follow to get to a practice test:

Start on ets.org/toefl → scroll and click on “test takers” → click on “about the test” → click on “prepare for the test” in a sidebar → click on “practice tests” in the body of the page → click on “launch the TOEFL iBT Free practice test.”

Compare this to the Duolingo English Test website, where the same goal is accomplished with a single click from the homepage.

Next, compare this: https://www.pearsonpte.com/preparation/on-test-day

To this: https://www.ets.org/toefl/test-takers/ibt/test-day/

The difference is clear.  Pearson has a much more modern design.  Key information is presented on a single page, and superfluous information is omitted.  Meanwhile, ETS uses ten different pages (and a couple of videos) to share a similar amount of information.

Suggestion:  Scrap the website and start again.  The user experience is so dated that yet another patchwork revision would just make it worse.  Eliminate all of the extraneous stuff. Rewrite the stuff people do look at. Modernize the interface of the user account.  Use the new PTE Academic website (https://www.pearsonpte.com/pte-academic) as a model.  Get rid of all PDF and ZIP files.

  •  There is no FAQ or Help Center.

I am puzzled by the fact that ETS doesn’t provide an FAQ page for the TOEFL Test.

Answers to simple questions (like “when will my PDF score report be available for download” or “how do I reinstate canceled scores” or “what does scores not available mean” or “what the heck is an Error 476”) are either not available or are buried deep in the website. Some information is only available in the “TOEFL Bulletin” PDF file that is rarely read.  Students resort to making long distance calls to the customer support department for answers.

Here’s what Pearson offers:  https://www.pearsonpte.com/help-center/general-faqs

And Duolingo: https://testcenter.zendesk.com/hc/en-us

Calls to the support center could be drastically reduced by providing easy access to this kind of information on the website.

Suggestion:  Implement a modern “help center” page using something like Zendesk.  Or just create a static page that answers the most common questions. Talk to people within and outside the company about what questions they are asked again and again.  Remember that young people nowadays feel very uncomfortable talking on the phone. 

  •  The registration process could be overwhelming and stressful

Consider the process students must follow after selecting a test date and location:

  • An acknowledgement that they have read a huge chunk of provided legalese and that they have also read the 41 page TOEFL Bulletin.
  • A long pitch to sign up for the TOEFL Search Service.
  • A request for background information.
  • A request to select score recipients.
  • A baffling request for an Agent ID number and a review of the above.
  • An attempt to upsell them practice materials that scrolls on forever.
  • A shopping cart
  • A checkout screen
  • A payment screen

This can take from 10 to 20 minutes to get through.  Compare that to other tests, where registration is completed in just a few minutes.

Suggestion:  Move as much of this as possible to sometime post-registration.  When possible, move it to the post-test stage.  Run a study tracking how long it takes to sign up for the test.  Adjust as needed.

  •  The free practice material is limited

Note how the free practice test provided by Duolingo is a fairly accurate simulation of their test.  It even implements automated writing and speaking scoring. That was a game changer for Duolingo and likely played a huge part in its rapid ascent. It makes students really familiar with how the test works.  It makes test-takers feel that they are getting something from the testing company instead of being asked to give, give, give.

In contrast, the free practice test on ETS is not an accurate simulation of the real test, nor does it include automated scoring.  Additional content is provided in a zip archive (!!!).

Suggestion:  Provide a free online practice test that is different every time a student takes it.  Just pick and mix sections from those 65 TPOs that are currently sold to Chinese schools.  I can’t do math, but that sounds like it would make possible thousands of unique variations.  Implement the SpeechRater and e-rater in the speaking and writing sections.  This is all possible.

  •  The fee to send scores is perceived as onerous

Some test-takers feel that it doesn’t cost ETS anything to send scores to universities. So being charged $20 per school is problematic to them.

They also know that Pearson allows students to send an infinite number of scores to an infinite number of schools at no cost. Same for the Duolingo English Test. That makes them feel good about the TTX over there.  

Suggestion:  Eliminate this fee.

  •  The pricing of the Home Edition is perplexing

Test-takers have noticed that it costs $185 to take the TOEFL Home Edition from their bedroom in Sri Lanka, and $335 to take it from their bedroom in Switzerland.  They can’t wrap their heads around this fact.

Suggestion:  Fix this.

  • Some people find the TOEFL Search Service frustrating

For me, the search service only comes up in conversation when a test-taker asks “what the heck is going on with all these unwanted calls?”  No one has ever spoken to me about the search service in a positive light.  Some students have shared negative opinions of the marketing methods used by participating institutions.  Stopping these pitches currently requires a long-distance call to ETS.

Suggestion:  Make the search service “opt-out” by default, and don’t include it in the registration process.  Simplify the opt-out process.  

  •  OTI waiting times are sometimes quite long

Some students have told me that they have waited more than 100 days for their scores to clear the OTI’s review process.  That’s too long, and it creates a negative experience for them.  Reduce these waiting times. Consider bringing back the old position of “test-taker advocate” to help students who are dealing with exceptionally long waiting times.

Update: Just so I don’t forget, here’s my list of additional miscellaneous suggestions, as I remember them:

  • The old score reports were great.  They provided a level (not a score) for each writing task, and each natural pair of speaking tasks.  Students loved them as they provided a little guidance re: what to study.  Bring back something like that, even if it is just part of the ETS account.
  • Fix the GRE/TOEFL account login bug.  That’s still annoying.
  • Provide more descriptive error messages for billing failures.  Those are perplexing.  At least suggest a solution.
  • Fix the EIAS system error bug, or provide a proper error message.
  • Clear up the remaining issues in the Official Guide.
  • Remove the “cancel scores” option from the end of the test.  This is easier to do if score reports are all free.
  • Provide the score PDF at the same time as the scores are reported.
  • If the Home Edition is immediately cancelled because some background software is running, tell the test-taker what was running so they can avoid the problem next time

A few notes from the publishing world:

  1.  Book Depository now lists the 2023 edition of Princeton Review’s “TOEFL iBT” with an on-sale date of February 2023.  I think this one will be almost identical to the 2022 and 2021 editions.  Princeton Review sometimes fixes small typos, but doesn’t seem enthusiastic about making large revisions.
  2.  Book depository also has listings for new editions of all four TOEFL books from Harper Collins with on-sale dates in May and June of 2023.  I like those books, so new editions are certainly welcome.  The existing editions were published many years ago.
  3. Barron’s is now selling an ebook version of their most recent TOEFL iBT book!  That’s the first ebook edition of a major TOEFL book I have seen in ages.

It is about time for me to write a new “best TOEFL books” blog post.  Keep an eye out for that.

ETS has confirmed that students in Russia and Belarus can now register to take the TOEFL Home Edition and the at home GRE from within Russia and Belarus.  However, they may not take those tests at a test center. 

Here is the press release announcing this change:

Beginning August 4, ETS is updating its March 2022 policy and will now allow test takers in Russia and Belarus to register for the TOEFL iBT Home Edition, TOEFL Essentials and at home GRE tests barring any financial restrictions they may encounter.

Due to the ongoing conflict in Ukraine, students in Russia and Belarus remain unable to take these tests at a test center within these countries. Students who prefer to test in person are welcome to take an exam in other countries or regions barring any travel and financial restrictions they may encounter.

We remain firmly committed to supporting all learners on their education journeys and hope peace prevails as soon as possible.

Testing in Russia and Belarus was suspended in March by ETS with an announcement that the testing organization stood “in unity with the people of Ukraine with hope that peace prevails.”