2026-09-20T06:42:09Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/42124 2023-07-11 col_2142_5131 col_2142_8800 com_2142_5130 com_2142_5132

Davidson, Frederick G. Davidson, Frederick G. Zhang, Jinming Bowles, Melissa A. Templin, Jonathan Tremblay, Annie Yi, Yeonsook 2013-02-03T19:16:27Z 2013-02-03T19:16:27Z 2015-02-03T11:00:32Z 2012-12 2013-02-03T19:16:27Z 2012-12 This dissertation is based on two major projects, cognitive diagnostic model comparison and implementing cognitive diagnostic assessment in an institutional test. In the first project, five cognitive diagnostic models are empirically compared for language test data under a unified general modeling framework. The models are applied to three sets of data, TOEFL Reading, TOEFL Listening and ECPE Grammar and examined in terms of their model fit to the data and functioning. The item-association root mean squared error values and multiple information criteria all indicate that the general model (LCDM) and the compensatory-RUM model are the best fit to all three test data sets used. The functioning of the models examined through multiple indices also unanimously confirms these fit indices. Based on these results, a discussion follows to argue that the general modeling framework is optimal for language assessment data due to its much greater flexibility. The behaviors of the compensatory RUM and non-compensatory RUM (Fusion) models are also compared and the differences are analyzed. Cognitive diagnostic assessment (CDA) has gained attention in language testing since late 90’s. A few models of this new assessment method have been applied to the response data of different language tests, yielding encouraging results in general. Most of these empirical studies used large-scale, standardized tests and retrofitted to these existing tests. The second research study in this proposal was an effort to go beyond this limited research context of previous studies of CDA in language testing. Using a new psychometric cognitive diagnostic model (Log-linear cognitive diagnostic model) and task type (elicited imitation task) as well as constructing the EIT test with a CDA implementation in mind from the outset (thus not retrofitting), this project tried to implement a cognitive diagnostic assessment in an institutional placement test. In doing so, the study employed a statistical method (resampling technique) as a way to resolve the issue caused by a small size of a test-taker pool at an institution, which is usually smaller than the optimal size for CDA implementation. It also tried to analyze polytomously scored response data. The study achieved a success at estimating with polytomous response data that were scored with a three-point scale (i.e., zero to two points). Though it was a limited success (in that more complex rating scale could not be analyzed), it was the first success in estimating with polytomous response data in the context of CDA research in language testing. The analysis results of the study also provide many insights and implications for the process of Q-matrix construction, grain size of attributes, appropriate task types and item types for cognitive diagnostic assessment as well as appropriate cognitive psychometric models for differing contexts of CDA implementation. The study also rediscovers or introduces the usefulness of bootstrap resampling method as an approach that is gaining popularity even in areas where only traditional quantitative methods are usually employed. Also, the networking occurred in this project between students in different specializations could be established as a new networking model in language testing. Considering such a collaboration is very much needed for implementing a relatively new measurement method in a specific knowledge domain, the co-work attempted in this project could serve as a model for implementing CDA in language testing. Item withdrawn by Mark Zulauf (zulauf@illinois.edu) on 2012-12-07T13:54:19Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 1 Yi_Yeonsook.pdf: 2634008 bytes, checksum: e8d4e1ecfce599960fefeebf2036cdc4 (MD5) Made available in DSpace on 2013-02-03T19:16:27Z (GMT). No. of bitstreams: 2 Yeonsook_Yi.pdf: 2810469 bytes, checksum: c5d8631f2b2714cb4a92e7f419bff55a (MD5) license.txt: 4060 bytes, checksum: 16b773af5d93263bfb1db73ba19b840b (MD5) Restriction data tranferred 2014-07-01T11:11:49-05:00 Original Data Group with Access UIUC Users [automated] Release Date: 2015-02-03 13:18:53 UTC Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system Item marked as restricted to the 'UIUC Users [automated]' Group (id=2) by Seth Robbins (srobbins@illinois.edu) on 2013-02-03T19:18:55Z Item is restricted until 2015-02-03T19:18:53Z U of I Only Restriction Lifted for Item 42071 on 2015-02-03T11:00:32Z. http://hdl.handle.net/2142/42124 en Copyright 2012 Yeonsook Yi cognitive diagnostic assessment cognitive diagnostic model comparison general modeling framework language test data Test of English as a Foreign Language (TOEFL) Reading and Listening university French placement test elicited imitation task resampling polytomous networking in language testing Examination for the Certificate of Proficiency in English (ECPE) Grammar Implementing a cognitive diagnostic assessment in an institutional test: a new networking model in language testing and experiment with a new psychometric model and task type text Educational Psychology 1616 Educational Psychology 0210 University of Illinois at Urbana-Champaign Dissertation Ph.D. PHD:Educatnl Psychology -UIUC 10KS0210PHD