State Records of South Australia MANAGING ELECTRONIC RECORDS ISSUES a discussion paper

The issues

1 Records management system design and creation

In recent years there has been a convergence of views between the records management and archives professions on both being involved in the earliest stages of a record’s life. Previously, the conventional wisdom was that the two groups met over disposal of records, but the archivist did not seek to influence the nature of current record-keeping - neither the retrieval (or filing) system used in the classification of records, nor any indexing or numbering. In particular, the archivist did not seek to define the circumstances where records should be made. Records were seen as a by- product of administrative activity - it was their very incidental character which was seen as giving them the crucial feature of impartiality.

The increasing volume of records prompted a change in view. The difficulties of ensuring timely and appropriate disposal of records was not just a factor of volume; it was also a result of the current record-keeping practices. Subsequent retrieval of records, outside the agency which had made them and which often depended heavily on its own inhouse knowledge to remember where significant papers had been filed, became a real nightmare. But it is the advent of electronic records which has been the catalyst for the general realisation that the design of the record-keeping system, and defining what records it should capture, are critical points of involvement for archivists if important records of Government are to survive.

1.1 Appraisal strategies - micro versus macro

The State Records Act requires the preservation of "official records of enduring evidential or informational value". This implies selecting certain records and discarding others, or appraisal. Indeed, appraisal is at the very core of what State Records does, to ensure the on-going preservation of the State's documentary heritage. Appraisal is the process of determining the value of records - not in the sense of monetary value, but with regard to the information contained in the records.

Currently State Records uses what is known as a micro level appraisal regime. This involves examining samples of records in agencies, working out their place within the broader administrative, legal and cultural context of the department, the public service and the State and then assigning retention periods. Such an approach is time consuming and rarely allows the archivist to develop a broader picture of all of the records being generated by an agency.

State Records is having difficulty meeting the demand for the appraisal of reasonably stable format records in large numbers. Given the speed at which electronic record systems can create and adapt records, it is projected that we will be less likely to cope with the appraisal of electronic records as they become more widely used.

The very nature of electronic records could imply that no appraisal at all is warranted. The major justification - or at least the most often perceived benefit - for appraisal of paper-based records is to reduce their bulk, to reduce the cost of ongoing storage. In addition, the systems used by agencies to manage their records are often inadequate for identifying relevant information by enquirers outside the agency. To rectify this, supplementary descriptive work has had to be done by archivists after the transfer of records into their custody.

With electronic records, physical bulk is not an issue. Increasingly large amounts of information can be stored electronically, and retrieval of potentially relevant references is both rapid and bountiful. Paradoxically, it is this over-abundance, well-known to anyone using the search engines of the Internet, which is the strongest argument to eliminate unimportant electronic records in a timely way.

To meet this challenge State Records is evaluating a model of appraisal used in the Netherlands and Canada. This model is known as macro, or functional, appraisal. It involves the systematic appraisal of all of the business activities of a department and the assigning of values to those activities. The record disposal schedule developed ensures that only those records which document higher level functions are kept permanently, while those documenting lower level functions are kept for shorter periods of time. The key difference to this approach is that the function, or business activity, is the main focus for appraisal and not the records themselves.

It has been suggested that, since most electronic systems are designed to meet certain business function requirements, a functional appraisal strategy could be more efficient for handling electronic records. Most archivists accept that it will not be possible to appraise individually all of the electronic records being generated by a department. This is due to the speed and volume by which the records are generated and adapted, and the pace with which such record systems are replaced. A project is currently under way at the University of Indiana which is examining the application of functional appraisal strategies to the electronic record generating systems in use by that University.

Issues for consideration

1. How similar (and valid) are the reasons for appraising electronic records to those applying to traditional paper-based records?
2. Should State Records adopt a functional appraisal methodology to appraise all of the output records of an agency, regardless of the manner in which they were generated.
3. What transitional strategy is appropriate to adopt? For example, State Records could adopt a mixed appraisal strategy which would involve the following:

   (a) continuing with a micro appraisal approach of the traditional format records

   together with

   (b) a macro appraisal strategy intended to analyse the functions ofelectronic systems and then assigning appraisal criteria to those functions - the value or relative importance of these functions being reflected in the electronic records which are retained.

1.2 Ensuring adequate records are generated by electronic systems

State Records and agency staff will need to collaborate at the conception stage of an electronic system to ensure that all electronic systems in use do create and store records, where appropriate. Functional requirements for the management of electronic records systems should be addressed in the design and specification of those systems to ensure that the content, context and structure of the records created or retained provide reliable evidence of the creator’s activities and that permanent records are identified and preserved. This stage is critical.

The South Australian Government has a number of software products which have been mandated, and are being used, across the entire public sector. Examples of these systems include Masterpiece and Concept. There would seem to be scope for State Records to be involved in the design (if not the implementation) of such whole-of-Government systems: doing so would be an initial stage in the process of ensuring that they generate adequate records. Such experiences would complement our involvement in the design of agency operation specific systems. This approach is similar to that currently in use for the development of general (i.e. across-Government) and agency-specific records disposal schedules. Thus, it could have the advantage of ensuring, for a smaller initial effort, a widespread change - and improvement - in systems which are in general use, and which are generating large numbers of records

Requirements for preserving permanent value records should be incorporated into the systems design. Similarly systems which have temporary value records should be designed to preserve these records for as long as is necessary for current business transactions and for the retention times specified in any relevant disposal schedules. As with paper records, surveys have shown that only around 10% of electronic records systems are required to be kept permanently. Systems will need to be reviewed once they have been in use to see if they conform to expectations at the conception stage. Many systems will have a mix of temporary and permanent value records.

A further complication arises in the area of databases, from the software’s ability to manage varied data elements and the possibilities which exist (and are in use) to integrate data from differing database systems. These extremely useful functional abilities raise some serious issues for determining what is to be regarded as the record. In addition the varied formats of data which can be managed in any one single database make for complications in the description of that database, the articulation of what are records, and the management of essential metadata elements.

Experience shows that often the records management concepts involved in the design and implementation of electronic systems are a final, last minute consideration - if at all. Given the pace of change in electronic systems and the abilities which they offer for the creation and adapting of records, it is crucial that the record generation and management issues become one of the primary system design concepts.

Such a role is discussed in the State Records Policy and Procedure Manual (1996) but it needs further discussion and refinement. One option is for direct involvement by State Records itself in the initial stages of system design discussions. Another would be to require the issue of a certificate of compliance from State Records before the systems could be implemented. It would seem desirable for State Records to promote the involvement by all South Australian public sector agencies of their records management staff in the initial stages of system design discussions. This approach would ensure that the systems were designed with the records generation and management concepts as a primary element of the design. The general consensus within the records management and archival communities is that such an approach is essential to ensure that clearly identified records are generated and managed during the life of the system and not as a final afterthought when the system is to be replaced.

However, within the public sector, there are electronic systems currently in use which did not have the generation and management of records as primary system design concepts - but, nonetheless, do generate and contain records. Such legacy systems pose a problem for State Records, since modifying them is likely to be both time-consuming and expensive. It is vital for State Records to develop a range of solutions which can be applied - preferably by the agency rather than ourselves - so that such systems do generate adequate records.

The ability of electronic systems to integrate and manage data from differing applications has some significant consequences for records. Electronic mail is important here, since these systems are very efficient generators of what can be regarded as records and are very adaptable at combining data from different applications. The use of electronic mail for the conduct of official business increases daily, so it is imperative for State Records to define when an e-mail message becomes a record, how it should be stored, managed and described, and how such records will be stored and made accessible in the long term. The options range from purging the lot, keeping the lot, or a structured program of identifying and storing selected messages or messages from selected parts, positions or persons in an agency.

Issues for consideration

1. To what extent should State Records be involved in the system design stage for all electronic systems being implemented and designed by government agencies?
2. What other strategies are viable to ensure that the record-making and subsequent management is assured, for example, the routine involvement of departmental records managers in the design of systems?
3. What impact might come from State Records adopting a role of assisting in the design of "whole of government systems" initially and expanding into agency operation specific systems at a later date?
4. How is State Records to handle the issue of existing systems which were not designed to capture and manage records?
5. How will State Records deal with those agencies which have complicated databases systems in use which incorporate data elements from different systems and in widely differing formats?
6. How will State Records ensure that e-mail is seen and managed as an official record?

Contact the State Records of South Australia

Previous

Next