State Records of South Australia MANAGING ELECTRONIC RECORDS ISSUES a discussion paper

The Issues (continued)

3 Preservation of electronic records

A number of sections of the Act make direct reference to the preservation of records by State Records, and these are set out below:

Objects of Act

5. (1) The objects of this Act are -

(a) to establish State Records -

1. as the principal repository for official records that are no longer required for current administrative purposes; and
2. to ensure that official records of enduring evidential or information value are preserved for future reference; and

Mandatory transfer to State Records’ custody

19. (1) Subject to this section, an official record must be delivered into the custody of State Records (unless sooner delivered to State Records or disposed of in accordance with this Act) -

1. when the agency ceases to require access to the record for current administrative purposes; or
2. during the year occurring 15 years after the record came into existence,

whichever first occurs.

Keeping of official records in premises other than State Records’ premises

22. (1) On the recommendation of the Manager, the Minister may, in relation to official records that have been received into the custody of State Records, make arrangements with the Commonwealth, another State or any other person for the keeping and use of records in premises other than premises under the control of the Manager or in premises jointly controlled by the Manager and the Commonwealth, the other State or other person.

The implications of section 19 (1) on any future strategic directions for State Records, in terms of the management of electronic records, are significant

Subsection (a) refers to agencies transferring records, including electronic records, when they are no longer needed for administrative use. Given the pace of change in electronic systems, both software and hardware, it is more than likely that requests to transfer such records will arise well before the 15 year period elapses. The South Australian public service has an extensive range of electronic systems in use and is rapidly developing and purchasing such systems. If State Records were to try and accept, and verify and describe, all of the information held in systems which become out of date under this subsection we would not have the time or resources to conduct any of our other business activities.

Subsection (b) refers to a mandatory transfer period of 15 years, with exceptions to this being possible at the Manager's discretion. It is extremely unlikely that any one electronic records system will remain unchanged for the time envisaged in this subsection. However, such cases are possible and pose the questions of how to transfer the records on such an old system and who would be able to supply State Records with a detailed knowledge of the system and its changes, to enable us to effectively record, describe and manage the system. More significant, however, is that this provision requires State Records to assume custody of records unless the agency seeks postponement or exemption.

3.1 The requirements for preservation of electronic records

The key issue here is how to protect the content, structure and context of electronic records over time. If the medium itself cannot be preserved, how do we ensure that the information - and the context of that information - survives in an intelligible and accessible way?

3.1.1 The software/hardware museum concept

Very early in the discussions of electronic records it was realised that the special properties of electronic records would require a different storage solution to those of traditional format records. Since electronic records are dependent on software and hardware to be accessible, some archival authorities attempted to preserve all of the software and hardware to ensure that the records could be used. It rapidly become evident that the pace of technological change meant that any archives attempting to follow this path would rapidly consume its entire operating budget in keeping the systems running. In addition, the software and hardware manufacturers soon made it clear that they had little, or no, intention in keeping old systems running once they had developed new products. There are already instances in the United States and elsewhere where old electronic records can no longer be accessed due to the redundancy of the software and hardware. State Records has considered the implications of trying to keep the software and hardware for all electronic systems running and has come to the view that such an option is not realistic.

3.1.2 The single format option and Open Systems

It is not at all uncommon for software and hardware vendors to assure customers, and archivists, that their products are completely "backwardly compatible". While it may be true that some software products will enable users to access, convert, read and use records which have been created in superseded products, this does not in itself equate with the archivist's view of on-going access to records. This is because the integrity of the record is paramount. If this cannot be assured, access becomes pointless. And, of course, any one format must itself rapidly run into the stumbling blocks of software and hardware obsolescence.

Another possibility, based on technology, is the development of "open systems". There is no doubt that there has been progress in this area, and reduced difficulty in aligning requirements for data capture, storage, linkage and retrieval. But the evolution of standards is slower than the pace of technological change.

3.1.3 Flat files

One technique has been to migrate records into software-independent files, so-called "flat files" or ASCII files, and then to manipulate these datafiles using standard software packages. This worked well for electronic databases, the raw material for social science research. But the all-important context of electronic records is lost. A promising alternative seemed to be to use optical media or high density tape, and then utilise increasingly powerful free-text search tools. But while context is preserved in the tapes, the relevant record may simply never be discovered by free-text searching, and links with other relevant records cannot be made.

3.1.4 Functional specification

There is now work being undertaken both in Australia and North America to develop an alternative (and potentially superior) approach. Perhaps, not surprisingly, given the significance noted above of being involved in the design of electronic records systems, the solution for maintenance is also related to good design. The work seeks to define the functional requirements for electronic record-keeping and then to design software technical specifications that will ensure that the computer systems themselves make and preserve the contextual links so critical to understanding the significance of a record. The requirements include that the records are comprehensive, uniquely identifiable, unable to be altered, auditable and authentic. It is about managing business processes rather than the individual pieces of documentary record.

Since this work is incomplete, it is not yet clear how far these requirements can be sufficiently standardised over subsequent systems to ensure that the characteristics of records migrated into them are retained and are accessible. In the short-term, of course, there is a need to provide some assurance and guidance over how electronic records can best be managed, until it is clear what a viable long-term strategy is. For State Records, this again raises the question of what standard should be set.

3.1.5 Mandatory data integrity transfer standard

There is little doubt that some agencies will need to transfer custody of electronic records. State Records will need to provide for this custody - either itself or using a third party. From this perspective, it looks prudent to be precise and definitive on how such electronic records are to be formatted, prior to transfer. This is the course adopted by the National Archives of New Zealand which has recently specified exactly what format the records must be in before they are transferred, with a view to ensuring that the data is transferred in a standardised format to ensure its integrity. On the other hand, New Zealand Government departments are also encouraged to sign agreements with the Chief Archivist which focus more on preserving the functional characteristics of records rather than their format.

It is the latter aspect which eases what could otherwise be a very significant risk. It was unnecessary for the National Archives in New Zealand to become well acquainted with all of the electronic record systems in use there. Despite the whole-of-Government approach in South Australia, there is still considerable diversity in the public sector here, and extensive documentation of all the record-keeping systems and their data elements would be very large task. In any case, it would seem better to determine what an appropriate standard would be to ensure the long-term survival of electronic records which are of permanent value and look for comment from agencies as to their ability to meet it.

3.1.6 The advisory data integrity standard

The second possibility is for State Records to issue an advisory data integrity standard for use by South Australian public sector agencies to help guide them through the steps to ensure that their electronic records were to stay inviolate and accessible. This may assist agencies to understand what needs to be done more quickly than a format transfer standard, as it can be more readily related to their current business needs. Such a standard is closer to the role adopted in New South Wales: this spells out the issues which need to be considered in ensuring the integrity of electronic records in the long term. Such a standard could be made a binding requirement on all public sector agencies as a set of issues for consideration in the long term management of their electronic records.

Issues for consideration

1. How should State Records identify the key metadata elements for the ongoing preservation of electronic records?
2. How diverse are the electronic systems in use in South Australia?
3. Would State Records be in a position to make such a standard binding on the public sector?
4. Whether mandatory or voluntary, how would State Records ensure that the standard was being followed?

Contact the State Records of South Australia

Previous

Next