Here’s something that was posted to Reddit a few days ago. The test-taker’s previous test was canceled because he may have received assistance during the test (no refund was given):

“I’m about to retake it. I got a completely empty room, nothing on the walls, with just a desk chair and computer pushed against the wall in the corner with the door right behind me. Should I put a mirror beside the door so they can see there’s nothing else in the room? I will also be diligent to not rest my face on my hand, roll my eyes, speak words to my self… I am truly terrified. I need this to go to grad school.

[…]

Anyone has any other advice to help me make sure I don’t get unfoundedly accused again?”

If the testing company is confident that there are no false positives they should keep doing what they are doing. If, on the other hand, there could be a few of those they should figure this out.

A sort-of rhetorical question: Do you know who really loves Reddit?

A sort-of insightful answer: Students from the country ETS is targeting above all others.

Hey, it’s my seventh favorite day of the year: the day of the new TOEFL Bulletin!  A new version, dated April 2023, is now available.  The only notable change is that payment can no longer be made using paper cheques or money orders.  Also, a separate bulletin for test-takers in India has been issued.

Note that after April 15, TOEFL and GRE test-takers in India will no longer be permitted to use their Aadhaar card as their primary form of identification.  After that date they must present a valid passport.  I hope news of this change is widely communicated, as the Aadhaar card is commonly used by Indian students I interact with.  Source.

Test-takers from India may now pay for the TOEFL test in Rupees!  That’s really great.  To date many students have been required to use third party agents to help them process payments so this change should make a big difference.  It also looks like the fee for test-takers in India is fixed in rupees, which means the cost of the test won’t be impacted by ever-changing exchange rates.

Note, though, that local taxes are now added to the fee so it is a bit higher than it was previously.

I hope this is offered in more countries one day.

The Japan office of ETS says that an announcement is coming from ETS HQ on April 11.

Huge changes are coming to the TOEFL iBT in July.  Here’s what you can watch for:

• The test will be shorter.  It will take just about two hours to complete the whole test.
• The reading section will have just two passages (twenty questions in total)
• The listening section will be the same (but no unscored questions)
• The speaking section will be the same
• The independent writing question will be eliminated, but the integrated essay question will remain.
• There will be a new “academic discussion” writing question.
• There will be no unscored questions.
• There will be no break during the test

I’m happy about all of these things.  I think test-takers will like them too.

If you are curious what the “academic discussion” question looks like, I’ve created and uploaded some samples for you.  Each contains a question and a sample response.  You might recognize it from the TOEFL Essentials Test!  Update:  I have also put together a guide to this question.

In addition to the aforementioned changes, the instructions given during the test will be streamlined and the test software will be improved.  Those will also result in some time savings.

Best of all, the user experience will be improved.  The test registration process will be made shorter and easier, and the user account will be modernized.  It should be a lot easier to find essential information in the account now.  Both of those things make me really happy.  As regular readers know, I’m really interested in improved test-taker experience.

This is scheduled to begin on July 26, but of course that date could change.  ETS will make an official announcement of all this stuff on April 11.  You don’t have to take my word for it… just check the TOEFL website on that date!

I’ve summarized these changes in a YouTube video as well!

I’ve spent some time this week trying to help students from Iran affected by score cancellations salvage their academic futures.  To be honest, I haven’t been able to help much.

Here’s something that bugs me: 

A student’s scores were cancelled.  Like many others, she was told:

“As a result of ETS’s rigorous score validation process, we have identified several factors that substantiate this score cancellation, in part because we detected unauthorized recording devices/software that were open during the test session which is a violation of ETS policy”

This could mean anything.  Maybe she set out to cheat and was caught.  Or maybe she forgot to turn off Skype after talking to her grandma.  Who knows?  ETS won’t provide any more information.  But, hey, both are valid reasons to cancel the scores, I suppose.

In any case, she suffered greater consequences than a cancelled score. A very fine university withdrew its offer of admission to grad school and  informed her that (due to the seriousness of the offence) she may never apply again.

I really hope that this institution was given all of the information that was needed to make such a life-changing decision.  I really hope that it knows that sometimes (often?) this particular rule is violated entirely by accident.  To react in such a way they darn well better have been told that the student set out to cheat.  And if this decision was made in error, perhaps the verbiage used to inform them of the violation needs to be adjusted.  Words have meaning and weight, damn it.

In the ancient past, test-takers could call ETS customer service and ask for a “test taker advocate.” I don’t know if they were effective advocates, but the position did exist.  It might be time to bring that position back.  Or to create some sort of ombuds position.  As it stands, students with problems like this often appeal to me for advocacy, and I’m feeling pretty helpless right about now.  

It is impossible to get a definitive count, but it seems like a significant number of GRE and TOEFL scores from Iran were canceled last week. The scores are from tests taken between November and February. A total of 413 people have signed a petition  regarding the situation, while a Telegram channel serving as a hub for updates now numbers 105 members.

I spoke to an individual with canceled scores (from December) who has already began studies in Canada. Not surprisingly, he is nervous about what happens next.

It seems that in each case, test-takers were notified that the cancellation was due to unauthorized software running during their test administration.

Affected test-takers have reached out to media outlets covering higher education (not me) and to parties in the USA they think will advocate on their behalf (not me). One imagines this story will be reported on in the near future.

Reports are starting to come in that a significant number of TOEFL and GRE scores from at-home tests taken in Iran were canceled last week. I woke up Friday to a handful of emails about the situation. A trusted contact in Iran I spoke to yesterday said there have been “more than thirty” cancellations. Others have mentioned larger numbers.

One of the individuals who emailed me said her scores were successfully reported back in January only to be canceled last week due to unauthorized software being detected during her test. On social media, people have talked about scores from five months ago being canceled.

If any journalists on the higher education beat are reading this, I hope you’ll do a little digging. It looks like a story worth telling. I know it’s difficult to report on Iran in 2023, but I can point you in the right direction if necessary.

I spoke to a fellow last week who had experienced difficulty with an at-home test.  He checked in with the proctor, showed his ID, did a room scan, showed his screen with a mirror… but immediately after the test commenced it was terminated because unauthorized software was detected running on his computer.

The test-provider wouldn’t tell him what software had been detected, and they wouldn’t refund his $215 registration fee.

But, hey, this guy was determined to take the test. He registered to try again the following week.  Sadly, the exact same thing happened.  He found himself down $430.

Picture this: he doesn’t have a score, his deadlines are fast approaching, and he still has no idea what software caused the termination.  No refund has been offered.

Needless to say, the guy decided to take a different test.

For the record, the average monthly wage in his country is $556.

I hear about this sort of thing quite frequently (though most people switch to a new test after the first termination).

I don’t think this guy really set out to cheat.  Some background process was running in Windows, but this dude has no idea what a background process even is.

I recently read that the “Skills for English” test has created a proctoring position called a “Check-in Specialist.”  This individual is responsible for examining the testing environment and checking the test-taker’s ID before the test begins. After these things are done, the rest of the proctoring is handed off to someone else.  I love it.  

Perhaps all testing organizations should consider creating such a position. In addition to the above, the specialist could be trained to detect unauthorized software and quickly shut it down before the test begins.  I mean… the proctors have the ability to remotely control the test-taker’s machine, so why not?

It’s important for the people who design online proctoring services to put themselves in the shoes of the young people taking their tests.  I’ve learned that a lot of young people today don’t really know about the finer details of desktop operating systems.  Many of them don’t know what the Windows Task Manager is.  Some don’t even know what the System Tray is. In 2023 everyone lives and dies by their smartphones, so that level of knowledge is no longer necessary. They think that programs stop running when they exit them.  So… stuff is left running when they take the test. A trained proctor could step in and compensate for  the test-taker’s lack of expertise.

People who like to know where all the money goes can now enjoy reading an audit of the Educational Testing Service for the year ending September 2022. Download it from the Federal Audit Clearinghouse, or just message me for a copy.  This audit contains information that will be published in the next 990 form release from ETS. Update: it is available via  Propublica now.

Revenues for the year were 1.06 billion dollars, down from 1.07 billion for the year ending September 2021.

The audit reports an operating loss of about 93 million dollars. I think that is the largest loss for all of the years I have records for (back to 2002). Note that ETS no longer develops or administers the STAAR exams for the State of Texas. That very large contract ($388 million dollars) has since been awarded to Pearson and Cambium.

There are a few mildly interesting things deeper in the audit:

Like everyone else in 2021/2022 ETS took a bath on its investments. Its assets are listed at 1.8 billion dollars, down from 2.0 billion.

We can deduce from page 12 that ETS paid 22 million dollars for a controlling (91%) stake in Toronto-based “Kira Talent.” To quote Wikipedia, they “[operate] a cloud-based holistic admissions assessment platform designed for use by academic admissions departments to assess and enroll students.” Sounds good to me.

I think we can deduce from page 21 that the HiSET exam was sold to PSI for 3.6 million dollars.

29% of revenues come from one “client.” This note puzzles me every year. My guess is that the client is the State of California, but I am probably wrong. That seems too high.

The price to take the TOEFL Test increased in many countries on February 1. I track prices in about 60 countries and spotted changes in the following (in parenthesis is the change in USD):

Azerbaijan (+15), Benin (+10), Bolivia (+15), Brazil (+5), Canada (+15), Cuba (+5), Egypt (+40), Ethiopia (+15), Guadalupe (+20), Guatemala (+20), Hong Kong (+15), India (+5), Iran (+20), Israel (+10), Italy (+35), Jordan (+20), Kenya (+5), Mexico (+15), Nigeria (+80), Palestinian Territories (+15), Paraguay (+10), Lima (+10), Saudi Arabia (+20), South Africa (+10), Spain (+20), Sweden (+10), Tajikistan (+5), UAE (+20), USA (+10).

Interestingly, the handful of price decreases I noted a few weeks ago were all reversed, except for the cut in Germany. That accounts for the very large increase in Nigeria.

I think increasing prices decreases equity in education.

I’ll put these up on the blog in a nice chart, as usual, when I return home from my holiday. I’ve spent altogether too much time on this island thinking about tests.