Test Discrimination and Test Construction for Cognitive Diagnosis

Abstract

While Cognitive Diagnostic Models, CDMs, can be useful in the analysis and interpretation of existing tests, little has been developed to specify how one might construct a good test using aspects of the CDMs. This paper discusses the derivation of a general CDM index and three attribute level CDM indices based on Kullback-Leibler information. These indices will serve as measures of how informative an item is for the classification of examinees. The association of each index with correct classification rates is validated to support their use in test construction. The effectiveness of the indices for test construction is examined for items calibrated using the Deterministic Input Noisy "and" Gate model (DINA) and the Reparameterized Unified Model (RUM) by implementing a simple heuristic to construct tests from an item bank. When compared to test constructed based on methods other then the Kullback-Leibler indices and randomly constructed tests from the same item bank, all index based tests show significant improvement in classification rates.