Ordered Category Attribute Coding Framework for Cognitive Assessments

Abstract

Cognitive Diagnostic Assessment models define skills as binary. Examinees are described as either 'skill masters' or 'non-masters' and items as either requiring the skill or not. I propose an Ordered Category Attribute Coding (OCAC) framework, designed to enhance the diagnostic information provided by such models. This approach defines any skill, k, by the Mk steps taken to master it. Consequently, the entries of the categorical Q matrix represent skills' mastery levels required by test items and examinees' knowledge patterns represent their location on the learning path of each skill. The flexibility of the OCAC framework allows for a more informative, parsimonious and efficient representation of task requirements and examinee knowledge. The levels of required skills can be estimated simultaneously with the examinees knowledge states as well as noise parameters, with high recovery rate. The current work uses real and simulated data to test the framework's limitations and robustness to violation of underlying assumptions.