Appraisal, Provenance, and the Computer Revolution:
An Examination of Organizational
Records in the Electronic Age
Harold E. Thiele, Jr.
Department of Library and Information Science
University of Pittsburgh
© 1998 Harold E. Thiele, Jr. firstname.lastname@example.org,
Department of Library and Information Science, University of Pittsburgh
Katharine Sharp Review ISSN 1083-5261, No. 6, Winter 1998
One of the by-products of the Second World War was the computer. Originally conceived
electro-mechanical device capable of performing the repetitive numerical operations needed to
produce ballistics tables, the computer slowly evolved into the multi-functional machine that
dominates offices throughout the world. The development of the multi-functional computer and
the introduction of high speed networking technology is changing how individuals and
businesses perform their daily operations, and, as a consequence, the type of records they are
producing. As the tsunami of technology surges onward, efforts are being made to understand
the changes and consequences of the computer revolution. These changes are calling into
question many of the traditional assumptions archivists and records managers use to appraise
and classify records. This article examines some of these assumptions and explores proposed
solutions to the problems relating to appraisal and classification of organizational records in
the electronic era.
Whenever I'm confronted with a choice between two evils, I kinda like to go for
the one I never tried before.
--Mae West in the film My Little Chickadee (cited by Burke, 1988, p.
- The basic assumptions and practices followed by archivists and records managers in
organizational records have their beginnings in Europe several hundred years ago. With the rise
of the European nation-states over the last 400 years, authority was transferred from localized
feudal bodies to more centralized administrations. This led to the rise of civil administrators and
the bureaucracy. One side effect of this was the development of written regulations and
documentation of decisions made based on these regulations (Lutzker, 1982).
- These basic assumptions grew out of an environment bounded by a print-based (i.e.,
and paper) universe. Print-based materials, at this time, were very expensive and not to be
wasted. In each and every case, the 'record' of the transaction became indistinguishable from the
print-based medium that contained it. Unbound documentary materials developed into
transportable forms that identified the type of transactions they were documenting. Invoices,
deeds, bills of lading, and passports are examples of these portable documentary types. Often
documentary materials were bound into books or journals dedicated to documenting a single
type of transaction. Examples are accounts receivable ledgers, day journals, and books of vital
statistics. These records of business transactions tended to remain physically in the office that
created them. This was also an era when it was assumed that an office would perform the same
function year in and year out. The function, the office and the records were inseparable.
- At the beginning of the Twentieth Century, Weber's analysis of the bureaucracy, with its
institutional hierarchy and administrator roles in decision making, was incorporated into the
archivist's consciousness (Lutzker, 1982). To the assumption that the function, office, and
record were indistinguishable was added the concept of a stable organizational structure. During
this period, the most efficient method of maintaining control over records was to keep each
office's records together based on organizational structure. Even if the records were relocated to
an archive or storage facility, they were kept together and identified by their submitting office,
and not by their function or activity. There was little difficulty in locating records dealing with a
particular business transaction or activity since they tended to be print-based documents
localized in one identifiable office.
- The advent of the electronic office has called into question many of these assumptions.
World War II, records associated with particular business transactions or activities are no longer
confined to an individual office within a single organization. Many of these transactions involve
multiple political entities, corporate bodies, and individuals. Some parts of the documentary
evidence are captured on print-based medium, while other parts are captured by electronic or
other forms of media. Within the distributed electronic office, the documentary evidence for the
business transaction may not reside within the office, the state, or even the nation state. Parts of
the documentary evidence may reside on many different computers in many different locales. In
addition, the electronic databases do not conform to the traditional definitions of various record
types. Where the information was maintained in separated record formats in the print-based
world, in the electronic world the information is merged into a single relational database, and the
information is produced according to specified reporting formats. Finally, unlike the print-based
records, one cannot define an electronic record just by looking at it. Electronic records require
interpretation by some additional software package before it can be understood.
- The increasing size and complexity of modern civilization is generating increasing
records. We are being overwhelmed by the records we create. Without appraisal and its
associated records destruction we would be buried in outdated records. Society records more
information than it needs. Much of it is of an ephemeral nature or highly repetitive and not
necessary to maintain (Boles & Young, 1991). By 1990 the United States was creating over
trillion documents each year (Toffler, 1990). The pace of record keeping is increasing. Roy
Williams of the California Institute of Technology estimates that the amount of printed
information is currently equivalent to 200 petabytes (a petabyte is a quadrillion bytes or 10E15
bytes). Williams also estimates that the amount of information stored electronically is expected
to exceed 500 petabytes by the year 2000 (Murphy, 1996).
- The sheer mass of records defeats any attempt to obtain an intellectual understanding of the
events being documented. There are over five billion documents stored at the National Archives
in Washington, D.C. There are an additional 19 million cubic feet of federal documents stored
in federal repositories located across the United States (Murphy, 1996). In addition to these
stored federal records, there are several times this amount stored in thousands of other
governmental (state and local), corporate, institutional, and private repositories located around
the country. It is estimated that only 1% to 5% of the total records created are worthy of
preservation as records of continuing value (Reed, 1993, p. 158).
- The appraisal process reduces redundancy and thereby "make[s] accessible and
archival and research purposes, the intellectual working tools of organizations. [The] . . . aim of
archival appraisal . . . should be to make archives eloquent and to facilitate research"
(Menne-Haritz, 1994, p. 530). Records are created as "intellectual working tools for the steering
and controlling of cooperative decision-making processes" (p. 541). It is only by removing the
outdated, redundant, and generally valueless records that this process becomes clear.
- The traditional definition of appraisal is the process of evaluating documents to determine
value and disposition (National Archives and Records Administration, 1993; Robek, Brown,
Maedke, 1987). Cox and Samuels (1988) use a much broader definition, "any selection activity
that enables archivists to identify recorded information that has enduring value, primarily for the
documentation of society" (p. 29).
The basic concept that underlies appraisal practice is record value. The concept
that certain values inhere in records, that these values are primarily defined by use, and that the
archivist should be able to identify and measure these values. (Ham, 1993, p. 7)
- Documents are evaluated according to a set of criteria and classified as to how long they are
be retained, where they are to be maintained, who shall have access to them, the conditions of
the access, and the manner of their disposal. Retentive value depends on "[the] continuing
importance of records, taking into account their administrative, legal, and fiscal use, their
research value, and their relationship to other records" (Dearstyne, 1993, pp. 1-2).
- The critical aspect of the appraisal process to keep in mind is its decisiveness. Because of
uniqueness of most archival materials, the archivist's decision about what records are to be
maintained and which are to be destroyed is irrevocable (Ham, 1993). The responsibility of the
archivists to select documents and records for destruction often upsets many people because of
the finality of the decision. However, it is only through the appraisal process that we are able to
exercise some control over the records we create.
- Appraisal is an iterative process that seeks to separate out records that have long term or
continuing value from the great mass of records that have only ephemeral value. This process
initially separates a collection of documentary materials into two distinct classes: records and
nonrecords. It is with this primary classification that the impact of the computer is first felt.
- A key issue that must be dealt with is the elusive concept of a record. Everyone 'knows'
record is, but no one seems to be able to provide a definition that satisfies everyone. The first
step in the appraisal process is identifying what is and is not a record. The better definitions
attempt to provide some form of functional definition. This said, the overall impression is that
the focus is still on record formats fixed on print-based media.
- Records management textbooks tend to define a record as any recorded information that a
business produces or receives, regardless of its characteristics, medium, or form (Robek, Brown,
& Maedke, 1987). The Society of American Archivists prefers a more restricted definition:
document created or received and maintained . . . in pursuance of legal obligations or in the
transaction of business" (Bellardo & Bellardo, 1992, p. 28).
- Most laws in North America take a similar approach to defining a record, but usually
extensive listing of various records types and add a long list of exemptions to the law. The
model example of this approach is from the Public Printing and Documents Act (1991) which
defines records as:
[including] all books, papers, maps, photographs, machine-readable materials, or
documentary materials, regardless of physical form or characteristics, made or received by an
agency of the United States Government under Federal law or in connection with the transaction
of public business and preserved or appropriated for preservation by that agency or its legitimate
successor as evidence of the organization, functions, policies, decisions, procedures, operations,
or other activities of the Government or because of the informational value of data in
Canada also employs a similar style of language in defining what is meant by a record but
exclude the 'evidence' section that is present in the United States Code (Canada, 1991; Canada,
- In the legal codes of the various states the term used is often 'public records.' A good
of this approach can be found in the North Carolina statutes:
'Public record' or 'public records' shall mean all documents, papers, letters, maps,
photographs, films, sound recordings, magnetic or other tapes, electronic data-processing
records, artifacts, or other documentary material, regardless of physical form or characteristics,
made or received pursuant to law or ordinance in connection with the transaction of public
business by any agency of North Carolina government or its subdivisions. (North Carolina,
1995, p. 132-1)
In this section, 'public records' are defined in a manner similar to that used to define 'records' in
the United States Code. As with the Federal statute, the emphasis is placed on the record format
(i.e., letters or maps) or on the media used to record the record (i.e., film or tapes). As one
delves into the laws governing records, fine distinctions are uncovered. In Minnesota,
distinctions are made between 'government records,' 'state records,' and 'local records.' In
addition, 'records' are defined to exclude data and information not considered to be part of an
official transaction (Minnesota, 1994, Chapter 138.17 Government Records; Administration,
- Record definitions are not limited to statutes. Governmental agencies and departments issue
rules and regulations governing records. As in the previous case, these definitions mirror the
Federal code approach of listing record formats or media. The EPA (1994) definition is an
this approach at the departmental level:
Official records are made or received in the conduct of Agency business. They
relate directly to
the functions of an office and they document those functions. They consist of recorded
information of continuing administrative, fiscal, legal, historical, or informational value,
including published materials, papers, maps, photographs, microfilm, audiovisual, optical disks,
electronic records (computer tapes, disks, floppy diskettes) or other documentary materials
regardless of physical form or characteristics, made or received by the Agency that evidences the
organization, functions, policies, decisions, procedures, operations or other activities of the
Government. (Attachment A)
- In addition, there is the concept of 'nonrecord materials' which refers to materials excluded
or not meeting the legal definition of records (National Archives and Records Administration,
1993). This usually refers to documentary materials of a personal nature, that are not related to
official business. Often there are extensive lists of materials exempted from the definition of
public record. For example, Title 1 V.S.A. 317 (Vermont, 1995) provides a listing of 22
exemption categories or classes that include, among other items, library materials and test
- The increasing use of computers to produce and maintain records is resulting in a
re-examination of the records concept. There is a growing awareness that the traditional
definitions of records, with their associated lists of media dependent record types, are becoming
outdated and unworkable. For example, references to account books or maps have no meaning
in an electronic world. In an electronic system, the financial information exists in a complex
relational database, from which an account book may be generated. A map doesn't exist in a
Geographic Information System (GIS) either. It is just a collection of data points and vector
references. This disconnection of the information from a specific media presentation is driving
a movement toward developing a functional definition of the record that will exist independent
of any media.
- David Bearman (1994), one of the leading proponents of a transaction based approach that
to have a dynamic records definition, states that,
[a] record is any communication between one person, and another, between a
person and a store
of information available to others, back from the store of information to a person or between two
computers programmed to exchange data in the course of business. The important aspect of this
definition is that a record is not a collection of data but the consequence of a business event.
Records 'occur' rather than 'are.' (p. 190)
This approach incorporates the concept that the transaction record provides evidence that the
activity has occurred and the manner in which it has occurred. As Roberts (1994) states,
"transactional records are recorded information [taken] . . . in the transaction of business or the
conduct of affairs and kept as evidence of such activity" (p. 17). He further reports that at the
Records Management Office of New South Wales, Australia they distinguish between electronic
documents and electronic records "by the transactional origins and evidential qualities of the
latter" (p. 21).
- Australia has made the most progress in adapting this concept of record into law. In 1995,
Standards Australia issued a draft standard for records management that included functional
definitions of records and record keeping. Record keeping is defined as "the making and
maintaining of complete, accurate and reliable evidence of business transactions in the form of
recorded information" (pp. 4, 7); transactional records are defined as "recorded information, in
any form, including data in computer systems, created or received and maintained by an
organization or person in the transaction of business or the conduct of affairs and kept as
evidence of such an activity" (p. 8); and a transaction is defined as "the smallest unit of business
activity" (p. 8). This standard was approved in 1996 for use in Australia. It has been submitted
to both the International Standards Organization (ISO) and the North Atlantic Treaty
Organization (NATO) as a proposed international standard. At this time it is currently being
reviewed by both agencies for adoption.
- Dollar (1992) clarifies the changes that the electronic revolution is making in the record
world. He points out that many of the traditional records definitions depend on the secondary
characteristics associated with paper records to define the record--characteristics that are no
longer available in the electronic world. A transactional definition that captures the evidence of
the activity taking place will ensure the future of record keeping activities. In the new electronic
world, the record is no longer tied to its container.
- Since records result from organized and purposeful activity, the Principle of
asserts that important information can be obtained from the context and relationships
surrounding the records. The Principle of Provenance developed out of administrative
rules in Denmark (1791), Naples (1812), the Netherlands (1826), and France (1841). The
first published theoretical expression of the principle was by Francesco Bonaini in 1867,
who "articulated the principle of maintaining all the documents of a body in their original
order" (Eastwood, 1992, fn 4, p. 13).
- The original French concept of respect des fonds had both external and internal
external dimension stressed identification of records with their office of creation, and the
internal dimension stressed maintaining the original record order. The Germans articulated in
Provenienzprinzip a greater precision in maintaining the integrity of records with
creator. This German articulation of the respect des fonds became known as
- The Principle of Provenance emphasis is on the origin of the records. It seeks " . . . the
preservation of the context of the records, that is their links to purpose, function and activity, to
the individual or parts of an organization which created them, and to other records created by
that individual or within that organization" (McKemmish, 1993, p. 10).
- The context of records creation is more important than the contents of the records
(Dearstyne, 1993; Reed, 1993). As Dollar (1992) succinctly puts it, "understanding the
provenance of records requires an understanding of the business functions and activities
underlying the information system that produced the records" (p. 49). Bearman and Lytle (1985)
maintain that the effectiveness of provenance "rests on a detailed understanding of both the
structure and processes of the organizations which created the records in question" (p. 16).
- In addition to the Principle of Provenance, archivists and records managers have
used the Principle of Original Order as part of the appraisal process. The Principle of Original
Order, the internal dimension of respect des fonds, maintains that records should be
kept in the
order in which they were organized or maintained by the creating agent. In this fashion the
original order "documents the sequential development of organic activity, reveals organizational
approaches, shows administrative processes, and in general documents relationships"
(Dearstyne, 1993, p. 130). It provides evidence of the manner in which a particular activity or
transaction was performed, the particular order in which a function was fulfilled (Brunton
PRINT- vs. ELECTRONIC-BASED
- The key impact that the electronic environment is having on appraisal is the dislocation of
'record' from its container. With print-based records it is comparatively easy to make use of the
Principles of Provenance and Original Order to appraise the records because
provenance related information in paper files is manifested in organization charts
procedures, policies, physical arrangement of the records, media characteristics such as
watermarks, formal elements such as letterheads, and in content/context elements such as names
of writers and recipients, dates, references to other documents or files, and the like. In addition,
individual documents may contain marginalia, initials, and similar contextual information that
reveal the information environment in which records were created, used, and maintained.
Because the physical, formal, and contextual components of traditional records are inseparable,
the physical location of one record vis-a-vis another record denotes how records were created
and used at a given point in time. (Dollar, 1992, p. 49)
- Electronic records, on the other hand, provide no physical expression that can be used to
determine their provenance. The most glaring difference between electronic and print-based
records is that an electronic record cannot be viewed without the aid of some intervening
software and hardware. Without the correct software and hardware, the electronic data cannot
be seen to determine whether or not it is a record.
- The second area of concern is that an electronic record is stored as a randomized pattern of
throughout the storage area, intermixed with randomized bits from other records and nonrecords.
The provenance or the original order of these bits cannot be determined. Computer memory
management systems compress data that is not in active use and or restructures the data in some
other fashion to maximize the use of storage space. In networked environments, it is often
difficult to identify which linked computer system has which part of the record.
- A third area of concern is the current process of merging information from many offices into
centralized data warehouse. Since each bit of information may or may not be used by a wide
variety of offices, there is great difficulty in identifying which office has custody over which bit
of data. Custody is an important aspect of appraisal because it often determines the retention
period for maintaining records and their associated data.
- A fourth area of concern is replacement of earlier data with newer data. In many data
warehouses, no record is maintained of old data and their links to previous activity.
Geographical Information Systems (GIS) are a good example of this. As new data comes in, the
old data is often overwritten or deleted from the system. It is often impossible to recreate the
status of the system as it existed months or even weeks earlier. Without being able to
reconfigure the system to an earlier status, it is impossible to reproduce the information and its
presentation that project managers used to made decisions. This means that there is no source
record available for the decisions that affect entire populations, regions, and ecosystems.
- A fifth area of concern is the question of whether the preservation of the electronic record is
long enough so that it can be appraised. Unlike print-based records, electronic records require
early and active intervention to ensure their preservation and maintenance. If these records are
not upgraded whenever the system is changed, they have a good chance of becoming unreadable,
and therefore unappraisable. If these records are not transferred to fresh media, the media that
they are recorded on will probably degrade and become unreadable.
- A sixth area of concern is the ease of duplication and lack of authentication associated with
many computer systems. Individuals are able to get copies of as many records as they want.
Changes can be made very easily to electronic records, and in most systems are undetectable
and/or untraceable. It usually is impossible to tell which copy is the "original" and which is the
TRADITIONAL APPRAISAL CATEGORIES
- Until the Twentieth Century there was little need for appraisal methodology. With the
development of the modern nation state and its attendant bureaucracies, the mass of records
threaten to overwhelm resources. While work on appraisal methodology had begun in Europe
before the beginning of the Twentieth Century, it wasn't until the 1940s that systematic work
began in the United States. The pioneering work of Philip C. Brooks, G. Philip Bauer, and
Theodore Schellenberg defined the basic criteria that were to guide archivists around the world
in appraising records for the next forty years (Boles & Young, 1991).
- In the traditional appraisal process, two characteristics are considered to be of importance.
primary characteristic is the evidential or documentary value of the record. The secondary
characteristic is the informational value of the record.
- The evidential value of the record is reflected in the evidence it provides on how an office or
individual conducted its business. The three major types in this category are records that: 1)
have continuing administrative, legal, or financial use; 2) protect individual or community civic,
legal, property, or other rights; or 3) document the development or evolution of the creating
body or individual (Reed, 1993).
- The informational value of the record is determined by its information content and the
probability that it will be of use to the originating organization or to a variety of researchers
- This traditional methodology depends on a bottoms-up approach to the examination of the
records and their classification for retention or destruction. The records come to the attention of
the archivist near the end of the their life cycle, when their usefulness to the creating
organization is greatly diminished. The most widespread complaint about this approach is that
the records are appraised separately from their social and organizational context (Reed, 1993).
- In the 1970s Brichford expanded on this traditional model and added additional criteria and
categories, including an archival category. This category looked at the relationship of the
records in question to other records and the costs associated with processing, preservation and
storage (Boles & Young, 1991).
- In 1985, Haas, Samuels, and Simmons challenged the archival world to examine the
process and communicational patterns associated with their institutional enterprises.
To appraise effectively, archivists need to understand that the nature of the
technological process and the complex patterns of communication and funding affect the
existence and location of records. . . . For archivists this complex environment of internal and
external associations requires comprehension of a universe of interconnected documentation.
- In 1989, Bearman, picking up where Haas, Samuels, and Simmons left off, challenged the
approach to appraisal based on value concepts. Arguing that it is physically impossible to
appraise the volume of records using traditional methodology and criteria and that no
cost-benefit justification can withstand close scrutiny, he argued for a risk management
approach to selecting records. The question to be asked is not "what benefits would derive from
retaining records," but instead should be what is "the probability of incurring unacceptable risks
as a consequence of disposing of records?" (p. 10). He further stresses the need to focus less on
the records, but more on the activities they document:
We will only be able effectively to appraise larger volumes of records if we focus
methods on selecting what should be documented rather than what documentation should be
kept, and develop tactics for requiring offices to keep adequate documentation, rather than trying
to review what they have kept to locate an adequate record. (p. 15)
- Bearman and Hedstrom (1993) suggest that "all archival activity focuses on the business
of organizations, the requirements they present for accountability" (p. 97).
- Samuels (1992) argues that institutional functional analysis is the first step in all archival
processes, including appraising. The functional approach "frees the observer from focusing on
the particulars, such as the name of the office that created the record, and promotes greater
understanding of the purposes for which the records were created" (p. 4). The focus needs to be
on what the organization does rather than on who does it. Institutional functional analysis shifts
the focus of appraisal from the record to the functional context in which the record was created
(Cook, 1993). The analysis takes place on two levels, the generic and the specific. In the
generic analysis, all the functions are treated equally. After this is accomplished, a specific
institutional analysis is done where the functions are assigned values, which shift the appraisal
focus. Functions are generally grouped into three broad categories: core functions, support
functions, and sustaining functions (Cook, 1993).
- Cook (1992a) argues forcefully that the "concentration of archival appraisal on groups of
and on the search for 'value' in them may well be misdirected . . . [and] the focus of appraisal
should shift from the actual record to the conceptual context of its creation" (p. 38). Using a
macro-appraisal methodology, the conceptual purpose of the records is identified: ". . .
macro-appraisal emphasizes . . . the archival value of the location or site or circumstances of
records creation rather than the value of the records themselves. It assesses . . . in a global way
rather than dealing directly . . . with individual records" (p. 53). He advocates a focus on the
functions causing the creation of a record and on the structures affecting its creation rather than
on the record itself.
- Sanford and Thiele (1996) have presented arguments in favor of a macro-functional
the appraisal of government records. Based on their experience with the Vermont Information
Strategy Plan (VISP) project in Vermont, they were able to show that functional appraisal using
business process analysis methodology was effective in identifying business functions and
locating key collection sites. The problem was that the time and expense needed to do the
analysis placed the techniques beyond the reach of most archival institutions. Instead they
argued for a generalized macro-functional approach where broad areas of government activities
are identified and categorized, and the appraisal process is used to acquire documentation in
each of the functional areas based on weighted criteria. Modified forms of the business process
analysis are used to identify business transaction collection sites.
- Cox (1994) notes that the major change in scope of archival appraisal in the United States is
from an institutionally bound focus to a multi-institutional focus. The development of the
documentation strategy model is a result of this change of focus. Cox identifies twelve archival
appraisal principles, and argues in support of a documentation strategy model to guide the
selection of records. Contending that the most important records are the ones that are used the
most, and therefore the ones most likely to be lost or damaged, he argues that archives cannot
wait until the end of the document life cycle to select records, but instead must become involved
nearer the time of their creation. He further states that the documentation strategy concept can
go a long way in meeting the representativeness issue raised by historian Yates. Cox also argues
that archivists cannot depend on historians as a source of criteria for making judgments about
long term value because the historians' identification of valuable documentary materials vary as
the political and social winds change the focus of their research interests. Instead, the effort
must be to provide some documentation of all the functions that government has engaged in.
DOCUMENTATION APPRAISAL STRATEGY
- Documentation strategy grew out of efforts in the 1970s to document social movements,
minority issues, and other issues not adequately covered in most archival collections. The
concept was developed by the efforts of the Joint Committee on Archives of Science and
Technology (JCAST), the Minnesota Historical Society, the State Historical Society of
Wisconsin, and the Bentley Historical Library during the late 1970s and early 1980s. It was first
defined at the 1984 meeting of the Society of American Archivists in a series of papers delivered
by Samuels, Hackman, and Aronnson (Cox, 1990).
- The concept is a response to the growing complexity of society and the failure of traditional
appraisal methodology to document adequately the activities, interrelations, and interactions of
governmental bodies, private enterprises and individuals. To document even a simple activity or
achievement requires the records of many organizations, government agencies, and individuals.
Adding to this growing complexity has been the electronic revolution which makes it possible to
create, manipulate, duplicate, modify, and erase records with considerable ease (Cox, 1990;
- The documentation strategy model focuses on current activities or issues. It seeks to enlist
aid of record creators, administrators, historians, archivists, and other interested parties in
identifying what areas of a business process or activity need to be documented and collected in
order to ensure an adequate record is obtain. As a result of this analysis, efforts are made to
ensure that adequate records are created and collected. The focus is less on what records exist
and more on what records should exist. This analysis is used to classify where and which types
of records are to be collected, and the physical examination of existing records is bypassed
- Samuels (1992) argues that this approach seeks to reduce the subjective guessing that
appraisal uses and replaces it with clear documentation objectives that are based on a thorough
understanding of the function, phenomenon, activity, or institution being documented. This
approach seeks to document the entire spectrum of the activity rather than depend on the bits
and pieces that make their way to the archives. It mandates an activist role for the archivist to
ensure that adequate documentation is being produced at the critical record points. This is
critical when dealing with non-print records. By being able to identify in advance the areas that
need to be documented, efforts can be made to ensure that the appropriate steps are taken to
ensure the upgrading and preservation of these records.
- Cox (1990) argues that the use of documentation strategies as appraisal strategies has
for the following reasons: 1) the traditional systems are decentralized, uncoordinated, reactive,
passive and duplicative; and 2) in modern society, creation and use of information is spread
among a diverse group of records creators. Documentation strategies allow a rational approach
to dealing with the super-abundance of records and information.
- Samuels (1992) uses functional analysis to identify seven functional areas that university
activities can be separated into. She identifies, weighs, and classifies activities the university
community engages in. Further analysis allows her to select the critical areas that need to be
documented and the type of records that need to be collected based on this classification.
Identifying documentation that other repositories are collecting from her list, she is able to
concentrate her efforts on classifying and collecting the necessary documentation to fill in the
BUSINESS PROCESS ANALYSIS
- Business process or transaction analysis in general refers to the same process. A
systems analysis from computer science and business modeling from management, this process
identifies all the activities involved in the performance of a business activity. It has proved to be
a potent tool for restructuring business because it follows the business activity across
organizational boundaries, and identifies the inputs and outputs at each stage of the activity. The
key archival and records management concept behind process analysis is that records are created
only when they are required to move the process forward or to document some transaction or
activity. Appraisers first need to examine the business process being documented to determine
whether or not archival type records are being produced. If the analysis indicates that the
business process has the potential of producing large quantities of archival records, then more
detailed examination of those records can be scheduled. If the analysis indicates little potential
of archival records being created, then the business process can be placed low on the schedule
for detailed examination (Reed, 1993).
- Business process analysis has an added benefit for the archivist. Because of its emphasis on
business functions and activities and not on organizational structure, it enables an appraisal
across structural lines. Functions which appear as minor activities of many agencies when
examined using traditional methodologies, gain in importance when appraised across structural
lines using business process models. Some functions that appear to be of major importance in
traditional appraisal appear much reduced in importance when examined against the entire
functional activity of an organization. The identification of common functions across structural
lines allows the development of standardized classification and treatment of similar record
types. These standardized approaches often show up in revisions of General Records Schedules.
The General Records Schedules provide a unified classification structure for a wide variety of
record types with built-in disposition recommendations.
- The business process analysis also allows the appraisal of functions that overlap
agencies. Records in different organizations that deal with different aspects of the same
function can be identified and appraised together (Reed, 1993). Rick Brown (1996) of the
National Archives of Canada used immigration to illustrate the importance of this aspect of
appraisal. He pointed out that no one agency in Canada has immigration as a major function. If
appraisal is based on agency major functions, most records dealing with immigration would be
destroyed without examination. With the business process analysis, all the offices dealing with
immigration were identified, and the critical record collection points scheduled for examination.
- The most intensive effort at using business process analysis in government was undertaken
the State of Vermont. Using techniques developed by the James Martin Company, Vermont
analyzed the executive branch. They identified seven major functions and distributed
government agencies under those functions (Vermont. Government Consulting Group, 1992).
Subfunctions were identified and further studies undertaken. The emphasis was placed in two
areas: revenue collection and client services.
- The revenue collection appraisal grouped four major revenue collection or generating
together. The resulting report established a consolidated view of the revenue collection process,
identified areas of major record keeping importance, and developed a unified terminology. The
process went far enough to identify data entities and allow for the development of a new unified
computer system devoted to revenue collection. The data tables provide much of the necessary
metadata information needed to appraise the electronic records. The major problem with the
study is that there was no input from the State Archives so that appropriate provenance metadata
could be included in the system as a design feature (Vermont. Government Consulting Group,
1994). Even without the provenance data, for the first time in Vermont history there is a
complete functional breakdown of the revenue collection function in state government.
- This study also showed how expensive it is to carry a business process analysis all the way
to the data element level. It took four full-time employees two years to perform the analysis.
The State of Vermont did end up saving money because of this study, because it enabled the
state to streamline the revenue collection activities into one computer system. The lesson for
archivists is that they need to assist other agencies in their efforts to computerize their functions
so that the archives can obtain this type of detailed information.
- Macro-functional analysis, or macro-appraisal, is a top-down analysis of the functions and
record creator. The leading proponent of this approach is the National Archives of Canada.
Faced with an explosion of records to be appraised and very limited resources, the National
Archives concluded that the traditional appraisal methodology was totally inadequate. Opting
for an approach that combined functional analysis with structural provenance the Canadians
classified each agency into one of four categories based on its importance and likelihood to
provide significant archival quality records.
- The criteria for classification of institutions under this approach were: 1) importance of
institution within the government; 2) breadth and diversity of institution's mandate and
functions; 3) size and complexity of institution; 4) identification of anticipated duplication of
some function in other institutions; and 5) important archival records in several media or
multi-media formats. Gaps in National Archives holdings or records at risk were not considered
to be relevant criteria (Canada, 1990).
- The 174 government institutions were classified into four categories: 1) institutions with
impact on Canadian society or government and anticipated high archival payoff; 2) significant
and separate institutions not covered by category 1; 3) smaller institutions with narrow focus;
and 4) institutions of limited or marginal nature (Canada, 1990).
- With the failure of the VISP effort to reorganize information handling along functional lines
Vermont, the State Archives assessed the value of VISP as a macro-appraisal tool. It was
recognized that the business process approach had high value for appraising government records
on functional process lines, but was less successful as a macro-appraisal tool. Moving away
from the traditional functional division of government into three co-equal branches, the State
Archives sought to identify functions common to all levels and branches of government. They
identified six major or macro-functional areas and were able to classify all the government
structures across these macro-functions:
- address needs, responsibilities, and grievances;
- adjudicate offenses, disputes, and disagreements;
- cultivate inter- and intra-governmental relationships;
- manage resources (stewardship);
- provide security and protection; and
- promote the social contract.
They discovered that this approach provided a framework that links the executive, legislative,
judiciary, bureaucracy, and extra-governmental sources together. It also revealed previously
unidentified relationships among the many units of government. The archives is developing
agency histories to include these functional links as a classification tool (Sanford & Thiele,
- Alabama is also moving toward a macro-functional analysis of government as a way to gain
control over records, especially electronic records. Using a process similar to one used in
Vermont, the Alabama Government Records Division identified eight broad functions of
government and created a matrix of the agencies performing each function:
- economic development;
- client services;
- administrative support operations;
- public advocacy;
- policy and statute development; and
- law enforcement and emergency powers. (Alabama. Government Records Division,
Then by linking the record keeping warrants (i.e., statutes and regulations) with the transactions
involved, appropriated retention schedules and requirements will be issued. Alabama also plans
to identify transactional areas where records are not being created and mandate their creation.
Based on this analysis, the archives will then be able to classify transactions and prioritize record
appraisal and collection activities (Alabama. Government Records Division, 1996b).
- The majority of the commentators dealing with appraisal, provenance, and the electronic or
computer revolution agree that the traditional bottoms up approach is inadequate to handle the
growing mass of records. The thousand or so professional archivists in the United States and
Canada are unable to control this approaching tsunami. To add to the problems, the electronic
revolution has changed many of the ground rules.
- The electronic revolution began in earnest with the development of simple copying
technology--the old thermofax machines followed by the wet toner copiers and finally the dry
toner copiers. These machines made it possible for multiple copies of records to be produced
and distributed across the records landscape. Where earlier archivists had to deal with a single
copy of a record located in one office, current archivists are now faced with redundant copies,
often running into the hundreds, distributed among multiple offices. Still, there was the
potential to manage this with the traditional approach, because these print-based copies still had
most of the provenance signatures attached to them.
- Electronically-based media has changed this forever. Unlike print-based media (paper,
parchment, and vellum) which has an expected life span running into the hundreds of years, the
electronic media (audio tape, video tape, magnetic tapes and diskettes, optical discs, etc.) has a
hoped-for life expectancy of 20 years. Experience shows that the life expectancy of the denser
storage media is closer to 5 years or less. The realization that electronic records are unstable to
the point that they might disappear before they can be appraised has galvanized the archival
profession into examining its core assumptions.
- The first assumption to be addressed was that records were appraised at the end of their life
cycle and classified into two major categories: records that were to be retained and maintained
for some specified period of time, and everything else that was to be disposed. The likelihood
that records recorded on electronically-based media would not survive long enough to reach the
appraisal stage indicated that the appraisal process needed to be moved forward toward the
beginning of the record life cycle. Many authors have argued that appraisal should take place at
the moment of creation: "Now is the time for archivists to be present at the creation of
documents, to ensure that they are designed not only to serve immediate administrative ends but
also administrative/historical research for policy planning and development" (Taylor, 1984, p.
30). As Dearstyne (1993) points out, "this type of advocacy . . . has brought counter appeals for
caution and adherence to traditional missions" (p. 229). Most of the driving force for moving
the appraisal process toward the beginning of the record life cycle is coming from archivists at
major institutions and archives where they are feeling the brunt of the growing record mass.
- The second assumption to be called into question is Schellenberg's (1956) approach of
records based on their evidential or informational value. Schellenberg favored physical
examination of the records or use of sampling techniques to make appraisal decisions. Under
this approach, the decisions were based on the contents of the records themselves. The general
understanding that emerged during the 1980s and 1990s is that this approach is unworkable with
both print-based and electronic records. There is no way to examine physically over a trillion
documents each year to make appraisal decisions.
- A consensus has emerged that the most efficient way to appraise and classify this growing
of records is to use some form of functional analysis. Functional analysis allows the archivist to
identify critical business functions that need to be documented. Using the results of this
analysis, classification categories can be established that allow the placement of record groups
into different classes. Several different approaches have been suggested, including the
Canadian, Vermont, and Alabama models.
- The approach furthest along in its implementation is the Canadian model. The Canadian
breaks with archival theory and differs from most efforts in that it classifies the institutions
based on their potential to produce archival quality records, not the records themselves. This
process classifies most Canadian institutions into one of four categories using a series of
functional requirements and value judgments. The argument is made that institutions create the
records and the National Archives of Canada must deal with institutions, not with records.
Priorities will be determined by detailed research on history, functions and mandates of the
institutions, information systems, and profiles involved (Canada, 1990).
- Both the Vermont and Alabama models make use of macro-functional analysis to identify
functional areas. They then use business process modeling to identify processes that are likely to
produce archival quality records, and also locate junctures where the probability is highest that
such records will be produced. After mapping the governmental structures against the functional
process model, they are able to classify institutional locations where there is a high probability
of quality records being located.
- These functional process models are interested in documenting the fact that some
process or activity has occurred, and are less interested in the contents of that interaction. They
view government as a process, and see records as objects documenting the workings of the
process. According to this view, the only record content of importance, in an archival sense, are
the records where the process broke down or encountered difficulties because they showcase the
full workings of the process under pressure.
- The third assumption shattered by the electronic revolution was that "the container is the
record." Before the electronic revolution, the print-based document containing the recorded
details of the transaction was the record; they were inseparable. Great weight was placed on the
form and structure of the record. With the advent of the electronic record all this has changed.
The record is no longer tied to its container. An electronic document exists as scattered data bits
throughout the memory storage unit of the computer. These bits can be reassembled into a wide
variety of presentation forms, all of which contain the recorded information. This has led to the
development of transaction based definitions of records which is having a major impact on
appraisal and record classification.
- One of the goals of appraisal is to separate records from nonrecords. The transactional
records definitions require a deeper understanding of the functions, processes and
interrelationships among record creators in order to achieve this aim than the traditional
approaches do. These transactional based definitions identified a third class of electronic
information objects--the information database. The information database or data warehouse
contains information that has value and should be appraised for continuing retention. It is not a
record, nor is it a nonrecord.
- The fourth assumption that needed modification was the Principle of Provenance. In the
print-based world, the physical copy of the record was encrusted with provenance signatures.
The office of the record creator was identified, the official responsible, date, location, and so
forth. In addition, the office and filing system where the record was stored also provided
detailed provenance information that aided in the appraisal and classification of the records in
question. Even the copying machine did little to disrupt this system. The electronic record
shattered the tie between the record and its provenance.
- What the electronic record requires is the active introduction of provenance signatures to the
record. They can no longer be allowed to just happen. One of the motivations behind the
movement to functional analysis as part of the appraisal process was to begin tying context (i.e.,
provenance) back to the record. By linking the business process and its associated structural
members to the record, provenance was restored to the appraisal equation. The development of
detailed institutional histories which are linked to functional and process descriptions provided
needed information to aid the classification process. These histories also act as authority files
which track the movement of functions and related business processes among different
organizational units. All the archives using the functional approach to appraisal are developing
these histories in considerable detail (Alabama, 1996; Canada, 1990; Sanford & Thiele,
- The last assumption to be discussed is the importance of the Principle of Original Order.
Considered to be the internal portion of respect des fonds, it has not fared well.
print-based world, the original order could be determined from the filing rules and the physical
presence in filing cabinets. Appraisal and classification could make use of the order and
arrangement developed and used by the originating office. All this is impossible in the
- In the electronic office data bits cannot be examined to determine original order. These bits
strewn randomly throughout the electronic storage space by the memory management system.
The memory management system continually reorganizes the structure of the memory spaces to
maximize the storage space. In systems working with a data warehouse, networked or not, there
are many links to data items in the warehouse. There is no simple way to deduce original order
or filing structure from the electronic office.
- The computer and its associated networking technology is putting severe stress on archives
appraisal methodology. The computer, with its ease of duplication, has multiplied the number of
records. Network technology has enabled copies of records to be held in computer databanks all
over the world. All this has had a profound effect on appraisal.
- The first effect has been upon the concept of the record. Since the data elements of the
are scattered throughout the electronic media and are reassembled into a variety of formats
through the use of external programs, the record has been disassociated from its container. Also,
the record has become a distributed entity, focusing on the shift from the contents of the record
to the transactions for which the records are evidence.
- Appraisal is the key to handling records, especially in the electronic world. Less than 1% of
content in records justifies preservation, so appraisal and classification systems that capture the
context and process most of the institutional activity will be sufficient. Appraisal needs to be
shifted towards the moment of record creation.
- Macro-functional analysis has the most potential for becoming an effective appraisal and
classification tool. Properly done, it ensures representative documentation across institutional
structures and also identifies functions that cross structural boundaries. Because it focuses on
functions and processes and requires good institutional histories as part of the process, it
promotes the development of efficient and effective classification schemes.
- Business process analysis describes the inner workings of the process in question, and also
identifies the entities and flow patterns involved. It locates critical junctures which allow
focused appraisal and collection activities, but has the severe drawback of being a very labor
intensive process. Archivists are going to have to piggyback their efforts onto information
reengineering activities being done for other purposes in order to acquire this knowledge.
- Provenance is still important. A strong case can be made for it becoming more important in
identifying the context surrounding electronic records. A major effort will be required to link
provenance information to the record as it is created in the electronic environment, since this
type of linkage needs to be part of the system design. This will require the development of
sophisticated metadata systems to ensure that this critical information is systematically attached
to the records as they are created. In order to ensure the proper appraisal and classification of
records, archivists and records managers are going to have to move to the forefront of their
institution's record creation activities.
- The burgeoning mass of organizational records and development of the electronic records
environment is forcing archivists and records managers to re-examine their basic assumptions.
The electronic environment has disassociated the record from its print-based container. The
record needs to be redefined in functional terms. This disassociation has also increased the need
for provenance signatures that are attached to each record. The traditional bottoms up approach
to appraising records based on record contents is unworkable in the face of the ever increasing
numbers and complexity of modern records. Archivists and records managers need to use top
down functional approaches that ensure adequate documentation of organizational activities and
functions. They also need to appraise records nearer the moment of their creation.
Alabama. Department of Archives and History. Government Records Division. (1996a).
Functional analysis of Alabama government. Montgomery, AL: Government
Alabama. Department of Archives and History. Government Records Division. (1996b).
government records program: Defining a new records appraisal strategy. Montgomery,
Government Records Division.
Bearman, D. A. (1989). Archival methods (Archives and Museum Informatics
No. 9). Pittsburgh, PA: Archives and Museum Informatics.
Bearman, D. A. (1994). Electronic evidence: Strategies for managing records in
organizations. Pittsburgh, PA: Archives and Museum Informatics.
Bearman, D. A., & Hedstrom, M. (1993). Reinventing archives for electronic records:
Alternative service delivery options. In M. Hedstrom (Ed.), Electronic records
program strategies (pp. 82-98, Archives and Museum Informatics Technical Report No.
Pittsburgh, PA: Archives and Museum Informatics.
Bearman, D. A., & Lytle, R. H. (1985). The power of the principle of provenance.
Bellardo, L. J., & Bellardo, L. L. (1992). A glossary for archivists, manuscript
records managers. Chicago: The Society of American Archivists.
Boles, F., & Young, J. M. (1991). Archival appraisal. New York:
Neal-Schuman Publishers, Inc.
Brown, R. (1996). Macro-appraisal theory. Paper presented at the 1996 Annual
Meeting of the
National Association of Government Archives and Records Administrators (NAGARA),
Washington, D.C. July 17-20, 1996. Session #12: Macro Appraisal in Practice.
Brunton, P., & Robinson, T. (1993). Arrangement and description. In J.
Ellis (Ed.), Keeping
archives (2d ed., pp. 222-247). Port Melbourne, Victoria, Australia: D. W. Thorpe in
with The Australian Society of Archivists.
Burke, F. G. (1988). Commentary [to Cox & Samuels (1988)]. American Archivist,
Canada. (1990). National Archives of Canada Act. Government-Wide Plan
Disposition of Records, 1991-1996, 22 October 1990. Ontario: Disposition Coordinating
Canada. (1991). National Archives of Canada Act. Report of the Multi-Year
Disposition Plan Work Group, March 7, 1991. Ontario: Disposition Coordinating
Canada. (1992). Management of Recorded Information, Directive 7-5,
June 1992. Ontario: National Archives of Canada.
Cook, T. (1992a). Mind over matter: Towards a new theory of archival appraisal. In B. L.
(Ed.), The archival imagination: Essays in honor of Hugh A. Taylor (pp. 38-70).
Association of Canadian Archivists.
Cook, T. (1992b). The concept of the archival fonds: Theory, description, and provenance in
post-custodial era. In T. Eastwood (Ed.), The archival fonds: From theory to practice
(pp. 31-85). Ottawa: Bureau of Canadian Archivists.
Cook, T. (1993). Taming the information behemoth: Institutional Functional
analysis as a tool for appraising the records of modern society. Draft. 5 November 1993.
(Typescript sent to the VISP Project, Vermont State Archives, by T. Cook in 1994).
Cox, R. J. (1990). American archival analysis: The recent development of the archival
profession in the United States. Metuchen, NJ: Scarecrow Press.
Cox, R. J. (1994). The documentation strategy and archival appraisal principles: A different
perspective. Archivaria, 38(Fall), 11-36.
Cox, R. J., & Samuels, H. W. (1988). The archivist's first responsibility: A research
improve the identification and retention of records of enduring value. American Archivist,
Dearstyne, B. W. (1993). The archival enterprise: Modern archival principles,
management techniques. Chicago: American Library Association.
Dollar, C. M. (1992). Archival theory and information technologies: The impact of
technologies on archival principles and methods (Informatics and Documentation Series,
ed.). Macerata, Italy: University of Macerata.
Eastwood, T. (1992). General introduction. In T. Eastwood (Ed.), The archival fonds:
theory to practice (pp. 1-14). Ottawa: Bureau of Canadian Archivists.
EPA. (1994). Summary of current records management policy on electronic mail.
Attachment A of Notice 88-1 Safeguarding and Disposition of Official Records
(Draft). Handout from Working Meeting on Electronic Records Management,
Pittsburgh, PA. April 8-10, 1994.
Haas, J. K., Samuels, H. W., & Simmons, B. T. (1985). Appraising the records of
science and technology: A guide. Cambridge, MA: Massachusetts Institute of Technology
Ham, G. F. (1993). Selecting and appraising archives and manuscripts. Chicago:
Lutzker, M. A. (1982). Max Weber and the analysis of modern bureaucratic organization:
toward a theory of appraisal. American Archivist, 45(2), 119-130.
McKemmish, S. (1993). Introducing archives and archival programs. In J. Ellis (Ed.),
archives (2d ed., pp. 1-24). Port Melbourne, Victoria, Australia: D. W. Thorpe in
with The Australian Society of Archivists.
Menne-Haritz, A. (1994). Appraisal or documentation: Can we appraise archives by
content? American Archivist, 57(Summer), 528-542.
Minnesota. (1994). Minnesota statutes annotated (Volume 10B, Sections 127 to 143). St.
Paul, MN: West Publishing.
Murphy, C. (1996). Backlogs of history. The Atlantic Monthly, 277(5), 20-22.
National Archives and Records Administration. (1993). A Federal records
(2d ed.). Washington, DC: National Archives and Records Administration.
North Carolina. (1995). General statutes of North Carolina. Chapter 132, Public
Charlottesville, VA: Michie.
Public Printing and Documents Act, 44 U.S.C. § 3301 (West 1991).
Reed, B. (1993). Appraisal and disposal. In J. Ellis (Ed.), Keeping archives (2d
ed., pp. 157-206). Port Melbourne, Victoria, Australia: D. W. Thorpe in association with The
Society of Archivists.
Robek, M. R., Brown, G. F., & Maedke, W. O. (1987). Information and records
ed.). Encino, CA: Glencoe Publishing.
Roberts, D. (1994). Defining electronic records, documents and data. Archives and
Samuels, H. W. (1992). Varsity letters: Documenting modern colleges and
Metuchen, NJ: Society of American Archivists and Scarecrow Press.
Sanford, G. D., & Thiele, Jr., H. E. (1996, July). Government
macro-functions. Paper presented at
the 1996 Annual Meeting of the National Association of Government Archives and Records
Administrators (NAGARA), Washington, DC.
Schellenberg, T. R. (1956). The appraisal of modern public records.
Washington, DC: U.S. Government Printing Office.
Standards Australia. (1995). Records management. Part 1: General requirements
Strathfield, New South Wales, Australia: Standards Australia.
Taylor, H. (1984). Information ecology and the archives of the 1980s. Archivaria,
Toffler, A. (1990). Power shift: Knowledge, wealth, and violence at the edge of the
21st century. New York: Bantam Books.
Vermont. (1995). Vermont Statutes Annotated. Title 1 through Title 3.
Charlottesville, VA: Michie Publishers.
Vermont. Government Consulting Group. (1992). Vermont Information
Montpelier, VT: Government Consulting Group.
Vermont. Government Consulting Group. (1994). Revenue Collection
Plan. Montpelier, VT: Government Consulting Group.