The role of covariates on inferring the Q-matrix and learning trajectory

Yigit, Hulya Duygu

Permalink

https://hdl.handle.net/2142/114047 Copy

Description

Title

The role of covariates on inferring the Q-matrix and learning trajectory

Author(s)

Yigit, Hulya Duygu

Issue Date

2021-10-11

Director of Research (if dissertation) or Advisor (if thesis)

Culpepper, Steven A

Doctoral Committee Chair(s)

• Culpepper, Steven A
• Kern, Justin L

Committee Member(s)

• Douglas, Jeffrey A
• de la Torre, Jimmy

Department of Study

Educational Psychology

Discipline

Educational Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Bayesian
• restricted-latent class models
• cognitive diagnosis models
• variable selection
• learning trajectories
• Expectation–Maximization algorithm
• first-order hidden Markov model
• Markov chain Monte Carlo

Abstract

"Chapter 2: In learning environments, understanding the longitudinal path of learning is one of the main goals. Cognitive diagnostic models (CDMs) for measurement combined with a transition model for mastery may be beneficial for providing fine-grained information about students’ knowledge profiles over time. An efficient algorithm to estimate model parameters would augment the practicality of this combination. In this study, the Expectation-Maximization (EM) algorithm is presented for the estimation of student learning trajectories with the GDINA (generalized deterministic inputs, noisy, “and” gate) and some of its submodels for the measurement component, and a first-order Markov model for learning transitions are implemented. A simulation study is conducted to investigate the efficiency of the algorithm in estimation accuracy of student and model parameters under several factors—sample size, number of attributes, number of time points in a test, and complexity of the measurement model. Attribute- and vector-level agreement rates as well as the root mean square error rates of the model parameters are investigated. In addition, the computer run times for converging are recorded. The result shows that for a majority of the conditions, the accuracy rates of the parameters are quite promising in conjunction with relatively short computation times. Only for the conditions with relatively low sample sizes and high numbers of attributes, the computation time increases with a reduction parameter recovery rate. An application using spatial reasoning data is given. Based on the Bayesian information criterion (BIC), the model fit analysis shows that the DINA (deterministic inputs, noisy, “and” gate) model is preferable to the GDINA with these data. Chapter 3: The rise of online learning platforms requires new approaches for developing formative assessments that provide accurate, fine-grained information on student learning profiles. Restricted latent classification models (RLCMs) serve a central role in the development and implementation of formative assessments. The latent structure for RLCMs is defined by the Q matrix, which is a binary matrix that specifies the relationship between underlying attributes and observed responses. Recent research developed fully exploratory approaches for inferring the RLCM Q matrix. Although exploratory methods exist for uncovering the latent structure educational researchers are also interested in understanding the role of intervention effects and student covariates on item performance and skill mastery. Consequently, the purpose of our project is to extend the exploratory RLCM framework to jointly uncover the latent structure and assess the role of student covariates on item performance and attribute mastery. We consider a general modeling framework for including covariates and consider two special cases which correspond to different research settings. Our models provide researchers with tools for evaluating intervention effects aimed at enhancing learning outcomes and documenting the extent to which the relationship between the latent structure and responses is invariant to student background characteristics. We develop a new Bayesian formulation to estimate model parameters and report Monte Carlo evidence pertaining to accurate recovery of Q and other model parameters. We apply the methods to a dataset including 516 students' performance on a spatial rotation test (Culpepper & Balamuta, 2017). In addition, including covariates also benefits us by providing insights about the relationships between the covariates and the item success and attribute mastery probabilities. Chapter 4: In educational environments and online learning platforms, formative assessments can yield valuable information about students' knowledge profiles. Knowing which attribute a student has been mastered versus has not been yet will help educators provide well-targeted instructions. In this respect, exploratory restricted latent class models have significantly been used to estimate students learning profiles from their response patterns. Although students' response patterns are the primary source for estimating students' item performance and skill mastery profiles, students' covariates may also provide beneficial information in the process. However, one main challenge is to decide which covariates to include in the model when many covariates are available. Thus, this chapter applies a ""spike-slab"" variable selection algorithm on covariates in an exploratory RCLM, which simultaneously estimates a mapping between items and the attributes. We develop a Bayesian formulation to estimate model parameters while imposing a variable selection algorithm on covariates. We report Monte Carlo evidence pertaining to accurate recovery of Q and other model parameters while correctly identifying the active covariates from inactive ones."

Graduation Semester

2021-12

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/114047

Copyright and License Information

Copyright 2021 Hulya Duygu Yigit

Owning Collections