Files in this item



application/pdf9210987.pdf (4MB)Restricted to U of Illinois
(no description provided)PDF


Title:Station time requirements for performance-based examinations: Their influence on reliability and validity
Author(s):Shatzer, John Howard, Jr
Doctoral Committee Chair(s):West, Charles K.
Department / Program:Education
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Education, Tests and Measurements
Health Sciences, Education
Education, Educational Psychology
Abstract:Within the past fifteen years there has been increasing interest in using performance-based examinations to supplement or replace written examinations designed to assess clinical competence in medical education. Despite the widespread interest in this type of assessment method, obtaining reliable performance estimates remains a major obstacle in their use. For acceptable estimates usually 12 to 25 medical cases must be included in the examination. With each case taking between 20 and 30 minutes to complete, the administration of a performance-based test is unfeasible for many medical schools.
The purpose of this research was to determine whether an exam with shorter time allocations provides as reliable score estimates as an exam with longer stations. A second purpose was to explore the relationship between station times and the characteristics of the scoring information collected. Fifteen second-year medical students were administered 11 standardized patient cases in a multiple-station format. The length of each station (case) was set at 20 minutes. Scores were analyzed over 5-, 10-, and 20-minute intervals as an indication of performance at different station lengths. Performance at each station was scored by means of a checklist of medical history questions deemed important to ask by the examinee. Checklist elements were organized into one of three categories: elaboration of the chief complaint, associated symptoms, and associated medical history. These categories were used to determine examinee response patterns over the time interval studied. The highest generalizability estimate was obtained during the first 10 minutes of the eleven stations. A second study was conducted with 23 second-year students taking an 8-station examination comprised of cases from the previous test. Station time was set at 10 minutes with scoring at 5- and 10-minute intervals. The highest generalizability was again obtained at 10 minutes. An analysis of response patterns showed that examinees did not change their approach to asking history questions when faced with shorter station time allocations. Results are discussed in terms of their implications for making these examinations more feasible yet retaining their reliability and validity.
Issue Date:1991
Rights Information:Copyright 1991 Shatzer, John Howard, Jr
Date Available in IDEALS:2011-05-07
Identifier in Online Catalog:AAI9210987
OCLC Identifier:(UMI)AAI9210987

This item appears in the following Collection(s)

Item Statistics