I want to draw attention to an article about remote proctoring in “Language Testing” by Daniel Isbell, Benjamin Kremmel and Jieun Kim. It’s the best article on this topic I’ve ever seen. Everyone involved in the development of standardized tests that use remote proctoring ought to read it.
I’ve written about remote proctoring issues here ad nauseum. I’ve tracked test-taker complaints about their experiences with remote proctors since 2020, and have shared many of them here. I’ve focused on complaints about online about tests being terminated at inopportune moments, and scores being canceled for reasons that aren’t explained properly. Test-takers often complain that they aren’t provided with evidence of malfeasance and that they are not provided with opportunities to appeal these decisions. I’ve focused mostly on TOEFL test-takers, but people do have complaints about other tests.
I’ve thrown around words like “dignity” and “respect,” but the authors of this article have found an even better one: “justice.” Here’s a relevant quote:
“When some test takers are accused of cheating, they might maintain that the accusation is false. False allegations of cheating could lead to more serious consequences than disregarding malpractice. To avoid such controversy, test providers should collect sufficient evidence of malpractice. Video and audio recordings of the test takers, computer screen, and testing environment can be recorded during remote proctoring. If malpractice did occur, test providers should be able to produce sufficient evidence. If malpractice cannot be established, or a test taker unintentionally committed a minor violation of test-taking rules, some leniency and a reattempt seem justified.”
This is a wonderful suggestion. Here’s another quote:
“The second warrant is that test takers and users have recourse to the test provider. When taking action against those suspected of malpractice, test providers should present test takers with solid evidence or reasons for score cancellation. Not knowing specifically what one has been accused of puts test takers at a disadvantage when making an appeal. Proactively providing test takers with reasonable criteria for score cancellation and comprehensible instruction for what is considered to be cheating would support both the fairness and justice of the test, for instance, specify the test takers’ right to voice their concerns about the testing process or the test results. For a test to be just, all test takers and test users should have easy access to clearly defined procedures for levying complaints or appealing malpractice-related decisions. Any test taker with low target language proficiency or disabilities, for example, should be able to understand the appeal procedure and make complaints to the test provider.”
I agree.
I’m not optimistic that testing firms will develop a justice oriented mindset any time soon. But we’ll see.