Last week I attended DETcon24 (that is, the 2024 edition of the Duolingo English Test Convention) in Pittsburgh, Pennsylvania. That’s the annual event that Duolingo holds to help stakeholders (from universities, mostly) become familiar with the DET. Since I’m not affiliated with a university, I was really flattered when the Duolingo folks extended an invitation.
While my scribbled notes can still be comprehended, I thought I’d share a few thoughts from the event. They are:
- The presentations on proctoring and security were my favorites. I’m NDAed, so I can’t write any details. But I’m certain that if the message about security can be repeated again and again with relevant decision makers in the room, the needle can be moved. The Duolingo folks need to find a way to communicate this stuff to more people. Perhaps a traveling road show.
- If you have an at-home test, I really think it is important to do security and proctoring in-house. This allows the tools to be more bespoke and customer friendly. And, of course, to provide additional security. Needless to say, Duolingo does it this way. Many of the issues I’ve written about here (cancellations for jagged profiles, cancellations for NVIDIA drivers, cancellations for memory usage) seem, in my opinion, to be examples of security being a blunt one-size-fits-all instrument. If you don’t do security properly people get annoyed and abandon your test (best case scenario) and innocent people suffer unduly (worst case scenario).
- The overall test volume seems to be higher than my estimates. Likely because the volume of vouchers given to institutions is quite high. I’ll see if I can repeat the exact number given for 2023.
- For some more info on the new writing tasks, read: “The Impact of Task Duration on the Scoring of Independent Writing Responses” now in preprint.
- I spoke on a panel. I was very nervous. I wore a new shirt.
- I think many decision makers still have an image of the Duolingo Test based on its 2019 iteration. Duolingo needs to do a better job exposing more people (and more types of people) to the most recent iterations.
- There was a comment about how and why Duolingo doesn’t really work with test prep providers. And about not selling test prep materials. Food for thought. Maybe testing companies shouldn’t sell prep for their tests. Someone made a comment about students having to unlearn IELTS prep. I laughed. But I love all my IELTS friends.
- Daniel Isbell has an article due out around the end of this year (but correct me if I’m wrong about the timeline). I won’t give away the topic but it will be an absolute must-read for anyone in the test prep space. If you are interested in this kind of stuff, check out his work. He’s the very best.
- There will be concordance matching the new individual subscores to subscores on other tests. The 2022 overall concordance will not change. I think that the sum of the subscores will match the overall score (which is not the case with the integrated subscores).
- An attendee made a comment about how institutions are loathe to change score requirements when concordance tables are changed, because they don’t want to be the only one. That explained a lot. I thought it was just inertia.
- I got a plush Duo for Mrs. Goodine. It is a surprise, so don’t tell her.