Regular readers of this space know that my pet peeve in standardized testing is the growing phenomenon of customer-no-service. Especially score cancellations that seem unjustified, unsupported or just plain mysterious.
In light of the recent SCOTUS decision re: affirmative action improvement in this area is more necessary than ever. Testing companies now have to work extra hard to ensure they aren’t canceling legitimate scores achieved by members of equity seeking groups. If nothing else, just imagine what the New York Times would write.
Anyway. With almost all of the big standardized tests gearing up for major changes in the months ahead, someone on a different social network than this one asked me what I would do to improve this situation. Here’s how I responded (I’ve removed company names):
Short Term:
- When tests are canceled due to the “possible” detection of malfeasance, a free re-test should be provided. When you tell test-takers that they “may” have done something wrong they see that as an expression that your accusation lacks complete certainty.
- Staff up the department in your organization responsible for academic integrity far beyond current levels. Give that department the time and money necessary to complete detailed (but timely) MANUAL reviews of all decisions to cancel test scores. AI is fine, but remember to place humans in the loop at all times.
- Don’t depend on a third-party for-profit proctoring service for anything other than proctoring. Don’t rely on them for decisions regarding test cancellations.
- Provide a robust appeals process when tests are canceled due entirely to statistical reasons.
Long Term (Pie in the Sky):
- Bring proctoring in-house.
- Collaborate with your peers in the testing industry to create a list of “best practices” for online proctoring. Or a code of ethics. Something that can be strictly adhered to so that test-takers get the best possible experience.
- Conduct regular independent audits of test cancellations. Give someone outside your organization the ability to examine individual files to ensure that your decisions are justified. This may seem like a bridge too far, but evidently it’s needed. At the very least create a committee of employees from outside of your academic integrity department who can do this. Give them the time and resources needed to do a good job.
- Eliminate multiple choice questions. This will eliminate a great deal of “analog” cheating in one fell swoop. Puzzling cancellations will also decline, obviously.