A Theoretical Assessment Ecosystem for a Digital-First Assessment—The Duolingo English Test
https://duolingo-papers.s3.amazonaws.com/other/det-assessment-ecosystem.pdf
The Duolingo English Test is a groundbreaking, digital-first, computer-adaptive English language proficiency test. In contrast to traditional assessments, the Duolingo English Test is a digital-first assessment, leveraging “human-in-the-loop AI” from end to end to automatically generate test items and score test-taker item responses.
This paper presents a novel theoretical assessment ecosystem for the Duolingo English Test. The ecosystem formalizes the theory underlying the test and key processes that support construct validity and a chain of inferences. The chain of inferences addresses digital considerations to ensure a valid, fair and reliable test score that can be used for the intended purpose of informing stakeholder admissions decisions at English-medium institutions.
The ecosystem leverages principles from assessment theory, computational psychometrics, design, data science, language assessment theory, NLP/AI and machine learning, and test security.