IDEALS Home University of Illinois at Urbana-Champaign logo The Alma Mater The Main Quad

An approach to the evaluation of expert systems for the selection of reference sources

Show simple item record

Bookmark or cite this item: http://hdl.handle.net/2142/23247

Files in this item

File Description Format
PDF 9512565.pdf (6MB) Restricted to U of Illinois (no description provided) PDF
Title: An approach to the evaluation of expert systems for the selection of reference sources
Author(s): Su, Shiao-Feng
Doctoral Committee Chair(s): Lancaster, F. Wilfrid
Doctoral Committee Member(s): Smith, Linda C.; Allen, Bryce
Department / Program: Library and Information Science
Discipline: Library and Information Science
Degree Granting Institution: University of Illinois at Urbana-Champaign
Degree: Ph.D.
Genre: Dissertation
Subject(s): Library Science Information Science Artificial Intelligence
Abstract: Sixty students at the University of Illinois voluntarily participated in the research to select reference sources for ten test questions with or without the aid of the expert systems to be evaluated. The students were first divided into little-experienced and no-experienced groups; the students in each group were further randomly divided into RE, SF, and NO groups. The RE group students consulted Reference Expert to select reference sources, the SF group students consulted SourceFinder, and the NO group students consulted the online catalog (IO$\sp+$/IBIS), browsed shelves in the Undergraduate Library, or retrieved from personal knowledge.The results of the study suggest that the current expert systems for the selection of reference sources cannot perform as well as experienced subject-oriented reference librarians. The better system, Reference Expert, could only achieve 80 percent accuracy in aiding users to select appropriate sources. SourceFinder achieved 40 percent accuracy. A failure analysis is provided. The results indicate that the students without "expert system" aid can perform as well as comparable students aided by SourceFinder in the selection of sources. SourceFinder can lead its users to select reference sources at a 25% consistency level, while Reference Expert achieves a 57.5% consistency level. Students not aided by any expert system achieve only 10% consistency in source selection.The results of this research generally do not offer strong support for the belief that "expert systems" for selection of reference sources can provide much help to increase the accuracy of question answering in reference services. In fact, in the case of SourceFinder, students of little experience aided by the system performed significantly worse than did the comparable students using the online catalog or personal knowledge. It is not just a matter of "why bother" to build an expert system of this type; there also exists the possibility that exclusive use of such a system to select reference sources may worsen the existing "half-right" performance of reference librarians.
Issue Date: 1994
Type: Text
Language: English
URI: http://hdl.handle.net/2142/23247
Rights Information: Copyright 1994 Su, Shiao-Feng
Date Available in IDEALS: 2011-05-07
Identifier in Online Catalog: AAI9512565
OCLC Identifier: (UMI)AAI9512565
 

This item appears in the following Collection(s)

Show simple item record

Item Statistics

  • Total Downloads: 1
  • Downloads this Month: 0
  • Downloads Today: 0

Browse

My Account

Information

Access Key