Earlier this week I attended an online “Ask Me Anything” about security and the Duolingo English Test featuring Basim Baig and John Woodcock. I’m glad I found the time to drop in, as I learned quite a lot. Like:
- 100% of DET administrations are reviewed by a human proctor after the fact. Ninety percent of tests are reviewed by one proctor, about 10% are reviewed by two proctors, and a small number are reviewed by more than two proctors. The proctors review one test at a time. The review includes watching a video recording of the test session. This info came in response to my own question.
- Without the need to livestream a video of the test to a human proctor during the administration, the test becomes somewhat more resilient to short connectivity issues.
- Some sort of firewall is used during the administration of the DET to prevent the test browser from communicating with anyone other than the Duolingo server.
- Keystroke biometrics are used to identify the presence of proxy test takers who have gotten involved in multiple tests.
Point four is interesting and I would love to hear more about it in the days ahead. Keystroke biometrics have been used to prevent cheating for quite some time, but their effectiveness isn’t fully understood. Indeed, I don’t even know exactly how reliable they are as a personal identifier (compared to, say, finger or palm prints). Perhaps this is something that could also be implemented in the delivery of tests taken wholly at test centers as a way to further reduce the scourge of proxy test taking.
I’m glad that test makers are a bit more willing to talk about this stuff in a public forum. Obviously this builds trust in their products, but it also can inspire competitors to up their game. I remember sitting in on a wonderful presentation about test security at Duolingo HQ in Pittsburgh some years ago. Everyone there was NDAed, but at the time I thought that a sort of reverse-NDA that punished people for NOT spreading the word would be more useful.
One day I’ll compile a list of questions that score users might ask their favorite test makers to suss out whether they take test security seriously. Some of this stuff will be on the list!