EXPLANATION:

This document includes a brief listing of comments I made at the Allerton Institute at the Tuesday morning, October 31, 1995 session. My original pre-conference discussion document [which is a rather more personal statement] immediately follows these comments. For those who want more information I can send a bibliography of works relating to Sense-Making. I conclude the comments below with a few of the most pertinent references. In addition, also available in the Allerton Institute www site is a draft of a chapter entitled "Chaos, order, and Sense-Making: a proposed theory of information design" that after editing and re-writing will appear in a MIT Press book "Information Design", edited by Robert Jacobson. The paper elaborates on many of the comments offered below. The references which I have relied on heavily in developing my comments below are listed in the bibliography for this draft paper. Gladly welcomed will be any challenges, criticisms, questions, requests for more information, etc. from readers of these comments or the draft paper. Contact dervin.1@osu.edu.

COMMENTS:

1. For 22 years I have been working on a methodology whose explicit interest is to find a systematic way of applying user perspectives in the design and evaluation of information systems.
2. My assumption has been that there has to be a methodological coherence between the theory applied to design and evaluation and that applied to implementation both of the practice of research and the practice of information systems. This implies a coherence between our approaches to users and our approaches to systems which I believe we currently lack.
3. In this sense, then, my use of the term "methodology" does not refer merely to a set of method recipes but, if you will, to a theory of methodology and a methodology of theory. In turn, this use of the term "theory" goes far beyond the usual use of the term to indicate inductively derived or deductively applied generalizations regarding the nature of phenomena. I reserve the term "substantive theory" for this purpose and deliberately use the term "theory" here in a far more general as well as abstract sense -- in the sense of that which guides research and practice.
4. As is implied by the comments above, the methodology I have developed -- called the Sense-Making approach -- is a methodology between the cracks. It has explicitly moved between many of the polarities which dominate, even still, the social sciences and their derivative applied fields [as well, I might add, in many respects the humanities and even the natural/physical sciences]. The polarities I have tried to move between include, as examples: administrative vs. critical, qualitative vs. quantitative, theoretic vs. applied, micro vs. macro, foundational vs. contextual, social vs. psychological, structure vs. individual.
5. I am still interested in issues of prediction and control in information systems. HOWEVER, I speak of prediction and control in an entirely different way. Right now, most of our systems (and, thus, the theorizings that serve them) attempt to bend the user to the system. The standards for system development essentially lie outside the human domain -- for example, in so-called "knowledge" or "information" domains or in so-called "reality" domains. Underlying these standards is a philosophy -- a set of meta-physical assumptions regarding the nature of information, reality, systems, and users. The system is seen as mapping in some way reality and information is in some way the interface. This is a far too simplistic presentation but if the reader will understand it as such it will serve our purposes here. In this context, users are more an after-thought than a central thought. This shows in myriad ways. I would estimate that wittingly or unwittingly these assumptions permeate some 75% of our discussions. Look, for example, at our emphasis on the idea of typical routines when in fact research suggests that most of the most important information seeking instances involve the user entering the terrain of the non-routine, the anomalous, deviation from the normal, the rule. Or, look at the labeling of user's work as primarily "invisible work" and the implications of that label for our methodological choices both in research and in practice. Another example is the common use of the term "misuse" in our discussions with misuse being any use that deviates from our order.
6. This is not to imply that I think all modes of using systems are productive modes. Rather I am trying to highlight how philosophic assumptions permeate that inexorably intertwined net of meta-theory, substantive theory, methodology, and method. I think of these four terms -- meta-theory, substantive theory, methodology, and method as relating to each other on two-interconnected planes. On one plane we have meta-theory and substantive theory in an orthogonal relationship. On the second plane we have methodology and method. [I thank Peter Shields, Bowling Green State University, for his collaboration on the development of this two-plane model].
7. To complete the picture implied by my comments above, we need a second identical picture of two interconnected planes with the first picture labeled research and the second labeled practice. Now we must connect the two pictures.
8. This, then, is the terrain into which I am attempting to move. I am attempting, if you will, to change the nature of our discourse. For some of you a microscope metaphor may be more meaningful. In this sense, I am attempting to change our microscopes while understanding that a microscope is not merely a technology for observing, it is a technology embedded in a discourse involving meta-theory, substantive theory, methodology, and method.
9. An important aspect of my project focuses on how we position concerns for uncertainty, chaos, disorder, randomness in our theorizings. I am deliberately avoiding a precise detailing of the differences between the terms here and reaching for the more general problematic we face: our paradox if you will. On the one hand, users seeking information move through anomalous states. All living creatures facing anomalous states exhibit disorderly behavior. On the other hand, if we attempt to learn how to incorporate this disorderly creature -- the information- seeking user -- into our system design and evaluation we seem to sink into the post-modern abyss of chaos, disorder, solipsism.
10. One of my major points is that our analytics for focusing on users preordain this outcome. We have been attempting to pin the user down as an embodied entity with consistency (i.e. predictability) across time-space because, we assume, that will allow us the kind of control we require in system design. Our methodologies for addressing this embodied creature consist essentially of trying to create a particular kind of picture of the user -- a picture where we take users and categorize them into boxes which cohere around attributes -- demographic, personality, social, routine behaviors, and so on. Over here: the educated, the open- minded, the cognitively agile, the experienced, the fact- finder. Over there: the poor, the close-minded, the cognitively inflexible, the inexperienced, the example- seeker.
11. Research consistently shows that our attempts to pigeon-hole users in these ways is not very productive: we account for very little variance. Further, studies which have broken out of these restraining assumptions show that the pigeon-holings do indeed apply only to less important information-seeking instances. Bottom line, educated and uneducated, open-minded and closed, inexperienced and experienced and so on are externally imposed labels assumed to apply to embodied entities called users with consistency across time-space. But result suggest that this is not the most productive way to characterize what is going on. Humans are more flexible than this methodological approach suggests. This is not to imply that there is no inflexibility in human behavior. Rather, it implies that when you use a methodological approach that searches only for inflexibility that's all you can find and darn little of it.
12. If, however, one cannot find consistency across time- space what hope is there for systematic system design. One of my major points is that when it comes to addressing users we must refocus our attention from people to process, from users to usings. It is here that we can make a break- through in doing user-centered research that genuinely informs user-centered design and evaluation.
13. Further, we must also reconceptualize how we think of information, reality, and knowing/observing. In essence we must challenge our current ontological and epistemology assumptions. Normatively our work now accepts what I will label here as epistemological incompleteness and epistemological diversity. Thus, we have incorporated the idea that there is a gap between reality and observing and that gap is entirely accounted for epistemologically -- either by bias and error, or by difference. We are schizophrenic about which of these we choose -- some choose the former, some the latter, some both ricocheting between the two. Notice, however, that from the perspective of "difference" theorists "bias" theorists erroneously conceptualize difference as bias. And, from the perspective of "bias" theorists, "difference" theorists erroneously conceptualize bias and error as difference.
14. In this context, ontology -- assumptions about the nature of reality -- are most often bracketed [i.e. loosely defined as set aside] when we focus on users. But this setting aside is where serious mischief enters when it comes to system design and evaluation. This is because our conceptual perspective permits the systematic to be characterized only in users conceptualized as embodied persons while our results show enormous unsystematizable differences from user to user and user to system creator with these differences characterized either as error/bias or as epistemological perspective.
15. Setting ontology aside seems to result from two different methodological steps. One of these might be called a methodologivcal schizophrenia. On the one hand we have diverse users who as a body appear disorderly; on the other hand we have orderly systems. Somehow, we say to ourselves, we must learn something about those disorderly users to make them more orderly. Let's enter into their worlds bracketing our interest in the potentials we assume information systems offer for making reality-facing more effective. Obviously, what results here in research practice is only a temporary setting aside of ontological assumptions because when the results on users are "read" into system design ontological assumptions pertain again. Hence, a baggage of terms like misuse, invisible work, and so on.
16. A second way of setting ontological assumptions aside is by explicit theoretic mandate such as the mandates offered by phenomenologically oriented approaches which conceptualize phenomenological and cognitive processes as standing between observer and observed. Another such theoretic mandate comes from language-based theorizings which assume in various ways that it is language and discourse that is, in effect, reality for us language-using humans. Interestingly even these theoretically driven mandates end up frequently with only a temporary setting aside of ontological assumptions because when it comes to finding pattern in results and applying them to system design assumptions regarding ontology re-enter in one way or another (e.g. in consensus politics where the system meets the needs expressed commonly by the many and marginalizes the idiosyncratic needs of the few). The other outcome -- sometimes labeled the post-modern outcome -- is simply giving into to the chaos of it all by suggesting that reality if inherently chaotic and unmanageable and humans so incredibly diverse in their reality-handling approaches that nothing systematic can be understood.
17. None of these approaches incorporates ontological assumptions which have a coherent role in information- seeking and use studies. Using the idea of anomalous states again, it is as if we have relegated all of the disorder that an anomalous state implies to the user and none to reality. I propose that we must bring ontology out of the closet if we are to make progress in understanding users of information systems in such a way that our results can inform system design. In particular, I propose that if we posit reality as both ordered and disordered, or ordered only in part, we immediately are forced to reconceptualize what information seeking and use are about, and what role difference between humans plays in information systems. With this reconceptualization, what humans are doing when they seek and use something called information is building bridges to facilitate their movements (physical, cognitive, emotional, etc.) across a reality (time-space) that is assumed to be orderly only in part. Differences between humans are reconceptualized as differences in how humans build their bridges. Humans need access to the bridges built by others not merely because they are ignorant of aspects of reality but muddling through when at best you can only be partially instructed is helped by being informed about how others have muddled through.
18. By reconceptualizing our ontological assumptions we begin to address a more coherent way of conceptualizing order and disorder in our approaches. Order and disorder are given aspects of reality. A disagreement may result from difference, or it may result from bias and error, or it may result from people building different bridges as they muddle through. This reconceptualization mandates that our methodologies incorporate not only the possibility of epistemological disorder (i.e. difference either in perspective and/or bias) but ontological disorder (i.e. a gap-riddled reality).
19. Every fact becomes a muddling through, a bridge built in a time and space for bridging a specific gap. A fact implies a bridge that many have found useful. The time and space are inherently part of the bridge. A disagreement with a fact becomes a different bridge, not merely a disagreement, but a bridge built for a different time-space. Again, the time and the space are inherently part of the disagreement.
20. This reconceptualization, in turn, mandates that we attend in research and ultimately in system design and implementation to bridgings which, in turn, implies utilizing as core constructs: time, space, movement, and gap. Focusing on movement implies, in turn, a focus on the forces which impel or constrain movement (i.e. power, struggle, constraint, motivation).
21. In this context, information seeking and use becomes reconceptualized as the making and unmaking of order and chaos. Our focus of attention changes from people to process and the change in philosophic assumptions provides us with direction.
22. In this context, traditional questions about users -- e.g., what are their characteristics? what are their past experiences? what are their abilities? -- are too far away from the action (literally and figuratively). Instead, the action is usings, movings, bridgings. This does not imply that rigidity and inflexibility, habit and routine are tossed out as potentially useful concepts. Rather they become reconceptualized as repeatings and our attention is redirected to understanding when and under what conditions and with what consequences repeatings are used in bridge buildings.
23. A primary assumption -- born out by our research to date -- is that there are patterns in these usings, movings, bridgings, patterns which provide a systematic basis for system design and evaluation. The patterns here are conceptualized in my work as "verbings" -- situation- definings, strugglings, thinkings, feelings, bridgings, evaluating, and so on -- the movings that humans do as they make sense of their worlds. It is assumed that if we look at these movings in conceptual terms that focus on movement (i.e. time, space, movement, gap) rather than in conceptual terms that focus on entities (i.e. demography, networks, personality, topics) we can make a break-through in user- research.
24. In essence, this calls for a focus on the contiguities of doings (internal and external) that make up information seeking and use, this includes both the contiguities of situation-defining and step-taking (i.e. information seekings/creatings) as well as the contiguities of step-taking and consequence-interpreting (i.e. information usings, evaluatings).
25. Further, this reconceptualization implies understanding difference is improved by pitting differences against each other not in terms of their mere (and sheer) presence [i.e. the current focus on difference as a babel of voices] but rather in terms of what struggle, what muddling accounts for the difference. This necessarily implies a move in focus from conceptualizing users as solitary embodiments of characteristics and information domains as static representations of reality to conceptualizing users as a series of usings which make and unmake self, system, information, society.
26. Notice that a major implication of this line of reasoning is that the way we look at users (i.e. as sense-makings in progress) and the way we look at information systems must be grounded in the same methodological coherence.
27. This implies an information system which accounts for information as grounded in bridgings and strugglings and a user as moving through sense-making moments which are themselves grounded in bridgings and strugglings. Some bridges and struggles are amenable to information stuff we call facts. But this approach assumes that any fact is grounded and when we conceptualize facts in this context we open up the information system to the context and fluidity across time-space which so characterizes human potential.
28. This approach, then, attempts to study systematically that which has been assumed to be unsystematic -- human creativity, the human capacity to change and challenge and resist despite formidable odds. This approach assumes that there is something systematic to be studied in human struggling, human sense-making. This approach assumes that this potential is an important part of what information seeking and using is about. Further, this approach assumes that we can best understand both rigid and routine uses of information systems as well as fluid and flexible and anomalous uses of information systems by assuming the potential for the latter and seeking to develop a systematic understanding of that which we formerly saw as unsystematizable. This approach assumes that since the important instances of human information seeking and use are characterized by humans challenging themselves to flexibility and fluidity our information systems will be most used and most useful if they are designed on these methodological principles.
29. Here are a few bibliographic citations for those who would like to read more of how I have applied these ideas in the Sense-Making methodology including: detailing of the philosophic underpinnings of the approach; the development of analytics for designing data collection and analyzing results; the development of a variety of interviewing approaches; and the use of the approach in specific research situations; and, the application to information/communication system design.

Dervin, B. (1989). Users as research inventions: How research categories perpetuate inequities. Journal of Communication, 39, 216-232.

Dervin, B. (1991). Comparative theory reconceptualized: From entities and states to processes and dynamics. Communication Theory, 1, 59-69.

Dervin, B. (1992). From the mind's eye of the user: The sense-making qualitative-quantitative methodology. In J. D. Glazier & R. R. Powell (Eds.), Qualitative research in information management (pp. 61-84). Englewood, CO: Libraries Unlimited.

Dervin, B. (1993). Verbing communication: Mandate for disciplinary invention. Journal of Communication, 43, 45- 54.

Dervin, B. (1994). Information<---->democracy: An examination of underlying assumptions. Journal of the American Society for Information Science, 45: 6, July, pp. 369-385.

Dervin, B. (1989). Audience as listener and learner, teacher and confidante: the sense-making approach In Rice, R. and Atkin, C. (ed). Public Communication Campaigns, 2nd edition. Newbury Park, CA: Sage Publications, pp. 67-86.

Dervin, B. & Dewdney, P. (1986). Neutral questioning: a new approach to the reference interview. RQ, Summer, pp. 506-513.

A BRIEF BACKGROUND

For 22 years I have been working on development of a general methodology, called the Sense-Making approach. The approach encompasses a meta-theory as well as a variety of methods. It is general because it is designed to be used in any situation where we may define humans as making sense. In its brief statement here, this sounds tautological. However, Sense-Making rests on an elaborate meta-theory regarding the nature of reality, knowing/observing, the human subject, the relationship between human subject and reality, and the impacts/positionings of the structures (e.g. linguistic, institutional) humans create. Because of its philosophic grounding, Sense-Making is called a methodology between the cracks, being used theoretically and practically, critically and administratively, qualitatively and quantitatively. The most important of the premises of Sense-Making include: a) assumptions of partially ordered reality and partially isomorphic observing, mandating attention to the phenomenon sense-making as bridging gaps between order-disorder, disorder-order; b) a mandate to refocus attention from substance/nouns to process/verbs so that research can find pattern in what is now called error -- i.e. pattern in the ways people make sense; c) extrapolation from this of a set of assumptions regarding the design maximally useful and efficient information/communication systems: that these systems must treat "information" as tentative and in contest and humans as potentially flexible (but not always disordered); d) that flexibility can be studied systematically and that there are patterns in flexibilities if studied as embedded in movement and not embedded in people; and e) that in fact we can develop ways to make so-called chaotic humans the focus of information systems without reducing the system to the kind of babel-of-voices which now characterize communication/information systems which try to be user-centered.

CURRENT RESEARCH AND INTERESTS:

Currently, I am working with/advising doctoral students and colleagues who are using Sense-Making in the following research genres (as currently labeled): a) information needs and seeking: the information needs of citizens regarding environment and risk, people in interpersonal situations, students in fulfilling class assignments; b) cultural media studies: how males and females construct their images of female bodies in relationship to advertising; how journalists make sense of their encoding processes in institutional contexts; how welfare recipients make sense of their life situations; c) communication/ nformation system practice: the improvement in evaluations of system utility when the system is designed using Sense-Making premises; the use of Sense-Making as an approach to interviewing in the journalistic encounter; d) pedagogy: the use of Sense-Making tools as pedagogical approach; e) development communication: how Sense-Making premises bring some order to a contradictory body of literature; f) feminist studies: how participants in consciousness-raising groups make sense of and implement the process.

My own writing focuses on bringing together the 22 years of work in a series of books. One, solo-authored, focuses on presentation of the meta-theory, methodology, and method. The second, edited, brings together a series of exemplar and issue papers. The third, co- authored with Peter Shields, is a large scale quantitative and qualitative application to phone user perspectives, needs, uses, and assessments vis-a-vis the phone. Other books to follow these several years hence include: a translation of Sense-Making in terms accessible to practitioners; a methods book on the use of content analysis because it is so central to the use of Sense-Making; and a book on the application of Sense-Making theory to practice. Also, I have established a listserve for folks interested in Sense-Making and/or the issues/problematics to which it attends and I am experimenting with applying Sense-Making principles to this avenue of connecting with others as a way of reaping the benefits of well- designed academic exchanges.

WHAT STRIKES ME MOST IN MY WORK:

Strikes me, and frightens me sometimes, are two outcomes of the use of the approach. The first has to do with the productivity of its use in research. The second the productivity of its use in practice. Regarding productivity in research in study after study, doctoral students working with me have been able to enter contests in their particular literatures on the diagonal trajectory which Sense-Making mandates and make significant contributions to their literatures even at the early stages of their careers. To give two examples: a) in information seeking and use, studies show the conditions under which across time-space (e.g. demography), a priori time-space (e.g. interest, involvement), and time-space bound (e.g. situation movement) factors account for more variance, thus illuminating what had previously been contradictions in the literature; b) in alternative media practice, studies show the complexity in practices and how this complexity relates to the sense the practitioner is implementing even though previous work had labeled some of these practices as merely inappropriate.

Regarding productivity in design, our applications have been small scale but in every instance evaluations of utility as well as willingness to use the communication/information system improved. To give two examples: a) in an exploratory retrieval study, Sense- Making premises were used to add what traditionally is labeled "subjective material" to the traditional abstracts of a small body of studies relevant to two doctoral classes and a comparison showed the Sense-Making "abstracts" being judged much more useful; b) in a small scale study of use of an interactive system, Sense-Making principles were used to direct input from participants with 80% of the participant evaluations explicitly stating that this was the first such experience they enjoyed and were willing to participate in actively.

MOST DIFFICULT PROBLEMS:

There are four:

1. There are lots of folks citing my work and a substantial number using Sense-Making and I continue to struggle with how so many people can use the same thing so differently using such radically different philosophic assumptions. I have made some progress in understanding this I think and my 1994 JASIS paper resulted in part from that progress. Bottom line, since Sense-Making conceptualizes every approach to information design as verbs (e.g. factizing, opinioning, exampling), any one whose view of information seeking and use is directed to a small sub-part of what Sense-Making addresses can still use the approach. Of course, however, I am not always happy with the outcomes. But, on the other hand, sometimes the outcomes teach and challenge me.
2. Most people in the field of communication and many in other fields, like library and information science, who are study human information seeking and use, are entering their more humanistic foci from either language approaches and/or discourse approaches frequently embedding their work in assumptions regarding intersubjectivity. In essence what they do is place language/discourse as a all-constraining umbrella which binds the phenomenon sense-making and mandate all sense-making as intersubjective. My approach has explicitly taken a diagonal cut through these dominant paradigms and redefines these nouns to verbs: languaging, discoursing, intersubjecting (forgive me, trying to make a point in a brief space). It is in part a subtle point too complex to elaborate fully ere. But the bottom line is that I try to encompass not only those aspects of sense-making which are bound within structures (i.e. the verbs of being acculturated, or falling in line along with the verbs of rebelling and resisting) but also those, perhaps rare but I assume still pertinent, moments of sense-making that would more properly be labeled as wild abandoning, spontaneous creating. In this respect, I am informed [curiously] both by a lover of science (Richard Carter, U Washington) and by postmodernists. Bottom line for me is shall we drive our research based on a most abstract and all-encompassing meta-theory of sense-making or shall we embed it in assumptions which pertain to only some instances. To those who say "but, but information systems are about intersubjectivity," I necessarily reply "not just that, not if you look at what people are doing with those systems."
3. I become more and more aware of the political implications of my work. To state it briefly and too simply: If I am proposing a useful theory, then people (including so-called experts) can use information systems best when: a) those systems acknowledge the tentativeness and inherent so-called subjectivity of anything called information ); and b) those systems use design principles which attach the situatedness to the information and put information into contest. If this is useful, then information systems must invite disagreement even with so-called fact. Facts are usually attached to vested interests. Hence, political implications. Hence, picture me with furrowed brow.
4. Because I explicitly incorporate issues of power in my work, I am often labeled "radical" which I find most confusing. The evidence from Sense-Making studies is clear: people are often aware of how vested interests make that which is called information. Their questions and concerns regarding these issues are an important part of their information needs. In addition, even the best scientists are aware and science as an enterprise has built in a host of accountabilities (notice my explicit avoidance of the word corrections) regarding perspective, situatedness, etc. So what's so radical? Mind you, I understand the societal framework within which these judgments are made. Suppose again, here, I feel too alone.

MY INTEREST IN THE FORUM:

Methodology is my passion. My own substantive interests remain primarily in information seeking and use. I am housed in another field -- communication -- which no longer has a strong substantive focus in information seeking and use and has never had a strong interest in methodology. I look forward to learning a lot and hopefully making connections with others interested as I am in breaking out of the traditional assumptions which still guide/constrain our information/communication systems and our research approaches pertinent to them.

WHAT CAN I CONTRIBUTE:

I'd be honored to be given the chance to talk about user-centered evaluation and its, to my mind, absolutely necessary relationship to user-centered design. If that doesn't fit in then I will contribute as best I can by listening, connecting, and questioning.