A class of sequential exploratory general cognitive diagnosis models using a polya-gamma data augmentation strategy

Jimenez, Auburn

Permalink

https://hdl.handle.net/2142/121507 Copy

Description

Title

A class of sequential exploratory general cognitive diagnosis models using a polya-gamma data augmentation strategy

Author(s)

Jimenez, Auburn

Issue Date

2023-07-13

Director of Research (if dissertation) or Advisor (if thesis)

Culpepper, Steven A

Doctoral Committee Chair(s)

Köhn, Hans Friedrich

Committee Member(s)

• Douglas, Jeffrey
• Kern, Justin
• Zhang, Susu

Department of Study

Psychology

Discipline

Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

cognitive diagnosis model, ordinal data, sequential process model, Bayesian estimation

Abstract

Cognitive diagnosis models (CDMs) are latent variable models which present a classification-based perspective on measurement in which observed response patterns are modeled conditional on a set of latent attributes. The classification-based frameworks are applicable in a variety of cognitive and non-cognitive domains, and provide a powerful tool for constructing assessments and measuring fine-grained information about the current working knowledge/skillset of persons. In this thesis, we will discuss advancements in the literature for the class of sequential, exploratory, general cognitive diagnostic models (SEGDMs) for analyzing multivariate, ordinal data under monotonicity assumptions on the latent attributes. Chapter 1: A prerequisite for valid inference of person proficiency is the accurate specification of the elements in Q, which describe the relationship between the observed items and the unobserved latent attributes. However, the construction of an accurate Q-matrix is a non-trivial process which is not necessarily feasible in practice in certain domains. Exploratory CDMs are valuable tools for inferring the underlying item-attribute relation in scenarios where Q is unknown. The issue of Q as a partial information structure is discussed and we highlight the need for joint estimation methods for Q and ∆ in order to make inferences about the dimensionality of the underlying structure and the form of the latent processes describing how persons select response options to an assessment item. Chapter 2: Much of the research in the CDM literature has been framed in the context of binary responses. The obvious limitation of binary models is the inability to analyze ordered item formats which provide additional information regarding the cognitive response process beyond that of a correct/incorrect item response classification. Recent research has evaluated a Polya-gamma data augmentation strategy for the efficient estimation of logistic response models using a Gibbs sampling method for binary data. We propose a Bayesian estimation routine for a SEGDM for ordinal data using a Polya-gamma data augmentation strategy for capturing within-item constrained latent processes. The proposed model decomposes ordinal response processes into a set of serial, conditional latent steps, enabling the joint inferring of latent structure and latent processes at the item-level of the item response process. We fit the ordinal model to sequential attempts data collected from a calculus-based probability theory course. Chapter 3: The model formulation in chapter 2 was capable of capturing response behavior characterized by a sequence of internal cognitive steps. However, the model assumed equality constraints on the functional forms for the model steps of a given item. The inherent limitation of the approach is that the model may be too restrictive in settings where response process differ across model steps of an item. In this chapter, we discuss a Bayesian estimation routine for modeling and inferring differential impact of latent attributes on step completion rates across steps of the model. The proposed SEGDM framework offers applied researchers a powerful tool for describing how individual differences in mastery status influences conditional comparisons of response options during the answer-choice process. Furthermore, the model offers a method of evaluating the appropriateness of theoretical assumptions using empirical data. We fit the model to the 2015 TIMSS student background questionnaire and the 2012 PISA problem-solving data. Chapter 4: For applied researchers, the issue of selecting an appropriate cognitive diagnostic modeling framework is paramount for understanding response behavior and making conclusions about person proficiency. However, this selection is not easy as the underlying cognitive processes involved in completing an item are often complex in nature. Simpler models offer ease in interpretation and fewer parameter to be estimated in scenarios where simpler processes characterize response behavior, yet are too restrictive modeling more complex response behavior. In contrast, more complex models offer flexibility, yet are overparameterized in scenarios where a simpler model framework adequately describes the response process. In this section, we discuss a Bayesian estimation routine for an innovated SEGDM framework which offers a middle-point between the models discussed in Chapter 2 and Chapter 3. The adaptive framework subsumes the two models as special cases, providing applied researchers a tool for understanding the dimensionality of the underlying structure, the impact of the latent attributes on the step completion rates, and whether this impact varies across steps of the model. We fit the model to the public-use approaches to learning and self-description (ALS) data collected from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-1999. Chapter 5: In the last chapter, we provide closing remarks, the overall contributions of the thesis, and potential routes for future research.

Graduation Semester

2023-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Auburn Jimenez

Owning Collections