Evaluating a model for developing cognitively diagnostic adaptive assessments: The case of young children’s length measurement

We report on an innovative computer-adaptive assessment, the Comprehensive Research-based Early Math Ability Test (CREMAT), using the case of 1st- and 2nd-graders’ understanding of geometric measurement. CREMAT was developed with multiple aims in mind, including: (1) be administered with a reasonable number of items, (2) identify the level(s) of thinking at which the child is operating for each developmental progression (domain) of significant mathematics, and (3) provide information about specific concepts and skills that constitute the cognitive components of thinking involved in various tasks. CREMAT relies on Q-Matrix theory and a resulting blend of sophisticated psychometric and statistical techniques (Rule Space Method, poset models) to determine not only students’ overall performance but also their specific level of thinking along developmental progressions. Results indicated that the current tests range from satisfactory to excellent in their ability to diagnose 1st- and 2nd-graders’ understanding of measurement, but also suggest revisions and implications for the approach. We conclude with a description of why CREMAT is important for early childhood practitioners’ ability to formatively assess and document children’s progress in learning math.