Raters' different judgments of intelligibility in the English placement test at University of Illinois at Urbana-Champaign

Lee, Ji Eun

Permalink

https://hdl.handle.net/2142/50728 Copy

Description

Title

Raters' different judgments of intelligibility in the English placement test at University of Illinois at Urbana-Champaign

Author(s)

Lee, Ji Eun

Issue Date

2014-09-16

Director of Research (if dissertation) or Advisor (if thesis)

Davidson, Frederick G.

Department of Study

Linguistics

Discipline

Teaching of English Sec Lang

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.A.

Degree Level

Thesis

Keyword(s)

• English Placement Test (EPT)
• Language Testing
• Intelligilibility
• Group oral format

Abstract

This study investigated raters’ judgments of test-takers’ intelligibility on Phase I of the EPT (English Placement Test), and how their various teaching experiences and different native languages affected their judgments. The EPT is administered to incoming international students whose TOEFL scores do not meet campus or departmental cutoff scores at the University of Illinois at Urbana Champaign. Phase I of the EPT oral section is designed to measure students’ intelligibility, and employs a group discussion format. Nine raters with different teaching experience and native languages were interviewed, and their responses were analyzed qualitatively. Results show that all the raters interpreted oral intelligibility differentially, and raters’ different understanding of the constructs underlying intelligibility resulted in different rating scales. This result wasn’t affected by raters’ teaching experience or language backgrounds. Rather than these, familiarity with a specific accent or rater’s individual ideas about intelligibility had more influence on their judgment. When they were asked about which specific features contribute more to intelligibility, most raters responded that segmental factors and lexical stress were important factors for their judgment. This might be because English Placement Test is a tool for assessing the ‘word level’ intelligibility. Phase I of the English Placement Test uses a one-on-one interview format or a group-administered format at raters’ discretion. Most raters reported that they prefer the one-on-one interview format. They argued that even though the group-administered format had the benefit of reducing time and expense, it would threaten the validity of the test because of the possibility of copying another test taker’s response or of getting nervous due to the situation in which test takers had to speak in front of group members. Providing a clarified and unified definition of intelligibility in the recalibration session is suggested. Additionally, raters still need to have sufficient knowledge of what the group oral discussion format is and know how to manage this format.

Graduation Semester

2014-08

Permalink

http://hdl.handle.net/2142/50728

Copyright and License Information

Copyright 2014 Ji Eun Lee

Owning Collections