Files in this item
|(no description provided)|
|Title:||The Reliabilities and The Cost-Efficiencies of Three Methods of Assessment for Writing Ability: An Empirical Inquiry|
|Author(s):||Bauer, Barbara Ann|
|Department / Program:||Education|
|Degree Granting Institution:||University of Illinois at Urbana-Champaign|
|Abstract:||Assessing achievement in writing ability from large numbers of essays is often an unreliable and time-consuming task. The extent to which these factors are problematic depends upon which conceptualization of this skill different raters adhere to during the grading process. The present investigation compares the reliabilities and cost-efficiencies of three methods of assessment, where each one conceptualizes writing ability differently.
Although it is generally accepted that analytical scoring methods are more reliable than holistic methods for grading essays, no evidence exists illustrating how the reliabilities for these two methods compare with those for the Primary Trait scoring system. Such a comparison required an empirical inquiry. In so doing, nine teaching assistants from the University of Illinois English Department were asked to grade 118 descriptive essays according to three scoring methods. After being divided into three groups of three different raters, each group of raters was trained to grade the essays according to one of three methods; that is, one group used the Diederich Expository Scale (an analytical method); a second group used a 4-point holistic method; and the third group used a Primary Trait scoring guide. Each group of raters then graded the essays according to its respective method. Time logs were also kept for each group's training and grading sessions.
This study yielded several major findings. An analysis of variance revealed inter-rater reliabilities of .874 for individual ratings and .954 for average ratings for the analytical method, .811 and .928 for the holistic method, and .633 and .838 for the Primary Trait method. Intra-rater reliabilities were also obtained, revealing composite correlations of .890 for the analytical method, .820 for the Primary Trait method, and .733 for the holistic method. The average time required to grade each essay was 4.14 minutes for the analytical method, 1.08 minutes for the Primary Trait method, and .96 minutes for the holistic method.
On the basis of these findings, it was concluded that the analytical method was the most reliable and the holistic method was the most cost-efficient in grading large numbers of essays.
Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 1982.
|Date Available in IDEALS:||2014-12-15|