Files in this item



application/pdfZHANG-DISSERTATION-2017.pdf (627kB)Restricted Access
(no description provided)PDF


Title:A formal framework for optimizing and evaluating interactive retrieval system
Author(s):Zhang, Yinan
Director of Research:Zhai, ChengXiang
Doctoral Committee Chair(s):Zhai, ChengXiang
Doctoral Committee Member(s):Han, Jiawei; Parameswaran, Aditya; Fuhr, Norbert
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Information Retrieval
Probability Ranking Principle
Interface Card Model
Abstract:The past decades have seen dramatic increase in the amount of information available to us, and the research area of information retrieval has served to help us access the tiny subset of information relevant to each of us more efficiently and more effectively. This thesis studies how to formally optimize as well as evaluate an interactive information retrieval system. First, we propose a formal general framework, the Interface Card Model, for optimizing interactive retrieval interface. We frame the task of an interactive retrieval system as to choose a sequence of interface cards to present to the user that can maximize the expected gain of relevant information for the user while minimizing the effort of the user, with consideration of the user's action model and any desired constraints on the interface card. We show that such a formal Interface Card Model can not only cover the classic Probability Ranking Principle as a special case by making multiple simplification assumptions, but also be used to derive a novel formal interface model for adaptively optimizing navigational interfaces in a retrieval system. Second, we propose a novel formulation of the Interface Card Model, the Interface Card Model with User States, for solving concrete interface optimization problems. The formulation is based on sequential decision theory, leading to a general framework for formal modeling of user states and stopping actions. Simulation and user study experiments demonstrate the effectiveness of the proposed model in automatically adjusting the interface layout in adaptation to inferred user stopping tendencies in addition to user interaction and screen size. Third, as a specific example of applying our proposed interface optimization framework in a larger scale real world application, we propose a Bayesian framework for user preference modeling and dynamically optimizing a faceted browsing system based on users' facet selection interactions. Finally, we propose a general formal framework for evaluating IR systems based on search session simulation that can be used to perform reproducible experiments for evaluating any IR system, including interactive systems and systems with sophisticated interfaces. We show that the traditional Cranfield evaluation method can be regarded as a special instantiation of the proposed framework where the simulated search session is a user sequentially browsing the presented search results. We further show that the proposed framework enables us to evaluate a set of tag-based search interfaces, a generalization of faceted browsing interfaces, producing results consistent with real user experiments and revealing interesting findings about effectiveness of the interfaces for different types of users.
Issue Date:2017-07-11
Rights Information:Copyright 2017 Yinan Zhang
Date Available in IDEALS:2017-09-29
Date Deposited:2017-08

This item appears in the following Collection(s)

Item Statistics