Assessing the Factors Leading to Adoption of Digital Libraries, and growth in their Impacts: The Goldilocks Principle

Paul B. Kantor
SCILS, Rutgers
kantorp@cs.rutgers.edu

Assessing the Impact of Digital Libraries is enormously complicated by the fact that the technologies and the resources on the associated networks are growing more than exponentially. Hence, all naive measures of digital library use and impact will grow. What is useful therefore, is to measure the factors present which influence adoption of or migration to a digital library, in an effort to identify those factors which accelerate and those which hinder the growth in beneficial impacts.

Some of the most important factors surely have to do with the management and organization of the transition process. These are best addressed through a detailed case study approach. With good luck, analysis of several such cases will provide some pointers for institutions seeking to establish digital libraries that are economical and effective. In this note we do not consider those management issues, but concentrate on the measurable indicators which should have good explanatory vaue.

We propose that the three major factors contributing to wide-spread adoption with beneficial impact are (1) richness and accuracy and accessibility of content (2) the technical infrastructure providing speed in access and transmission and ease of use (3) the user's perceptions of the importance of becoming a digital library user.

The first two issues can be assessed either by techniques of system description, or by energetic robots. The third is more troublesome because it requires us to probe the perceptions and motivations of the adopters.

I propose here that in studies and projects for which this kind of management oriented analysis is somewhat peripheral that very simple question forms can be adopted and used uniformly. With reference to the fairy tale character, Goldilocks, who found some porridge too hot, some too cold, and some just right, these questions are formulated on a 3 point scale rather than a more conventional 5 or 7 point scale.

Typical questions which can shed light on the perceptions of adopters, slow adopters, and non-adopters, are the following:

* In your judgment do the most productive workers in your particular field use digital libraries more than you do?

less than you do?

about the same amount as you do?

While keeping the answers the same the questions can be varied in a number of ways:

* Do you think that the most highly respected people in your field use digital libraries: (same list)

* Do you think that the people in your field whose work is of the very highest quality use digital libraries: (same list)

It is easy to see how to ring changes on this queston. But there will be great value in maintaining some core which is used in every survey.

If time and space permit the questions can be expanded to assess the user's general perception of the many kinds of electronic media. Thus there could be questions about use of electronic mail, use of web browsing in general, use of remotely analyzable data sets, etc. and etc.

I am led to this very lightweight easily portable format by reflection on the substantial complexities and difficulties encountered in studies such as the one now being conducted at Columbia University. In these studies we are attempting to assess the impact of a particular kind of digital library on users in the university community. We are also conducting parallel surveys of that community with regard to many kinds of computer usage, in order to situate the respondents to our deeper survey within that community. In general this would be out of scope as an addition to a digital library project which leads to the course suggested here.

Of course there is fertile ground for further research (and doctoral dissertations) in comparing individual perceptions about the activities in a field with the actual levels of use of both the respondents and members of the field at large.

In this brief position paper I would add only one more note: it is customary to assess many kinds of information services by asking the users how pleased, satisfied, or infuriated they are with the services. In this particular area that seems to be an extremely dangerous line of questioning because there are two factors involved which work against each other. With the progress of time, both the general socio-technical climate, and the content in a particular subject area grow. These two factors combine with positive synergy to increase the amount of use and the likely impact.

On the other hand, the growth in the general socio-technical environment seems nowadays to be accompanied by a super exponential growth in "hype". The effect of this is in many cases to raise user expectations even faster than the performance and value of systems can increase. In that case, perception of the value of the system itself will decrease from time to time, perhaps even for some time. One need only look at the widespread disparagement of Microsoft Windows 95 software, when compared to a simple enumeration of all the things that that software can do for the user, to see that this sort of mismatch is a real factor in today's human computer landscape.