The future of records preservation
This online resource captures current good practice in the area of digital records preservation but it is clear that as we work in this area the technology and institutional practices continue to move on at a pace. This section brings together some of the trends and initiatives we may see in the future.
1. ‘Hands free’ transfer
Though the EDRMS preservation task force has seen a few examples of direct integration between an EDRMS and a digital preservation system, this is still an area for future development. The extent to which transfer between an EDRMS and the digital archive can be fully automated is something to explore in the future and it would be helpful for the digital preservation community to see more case studies of how this might work in practice. Could we see a future in which a document is created and correctly described and classified by the user and that this is ultimately transferred into the archive without any manual intervention from the archivist or records manager? Is this something the community would like to see, or is trust in the technology never going to reach these levels
2. EDRMS as digital archive? (or vice versa)
Instead of aiming for effective integration between a record keeping system and a digital archive, could we have one system that does it all? Some EDRMS providers have been making moves into the digital preservation space, adding in features for ‘archiving’ of records in order to meet the need of customers who do not have digital preservation systems in place. Similarly, providers of digital preservation systems may be providing bespoke features for customers that are more traditionally seen in the records management space. It is conceivable that a supplier will manage to create a combined records management and digital preservation system in the future. One could imagine dragging and dropping records from a records management area to an archive area and seamlessly bypassing some of the challenges and complexities of integration and transfer that are detailed in this online resource. Again, trust in the technology is required in order for the community to embrace these developments. It is advised that practitioners working in this space continue to articulate their particular requirements and ask questions of solution providers about how certain functionality is implemented before making decisions on whether a system meets their needs.
3. Beyond the EDRMS
Though the EDRMS Preservation Task Force came together specifically to talk about preservation of records from EDRMS, it was clear once we started to scope our work, that records are not exclusively stored in formal EDRM systems. Within many organizations, other systems are being used to manage records and/or documents. It was noted that many organizations are moving away from a structured and controlled EDRMS to systems that are less formally managed. Sharepoint is one such system that falls outside of the tight EDRMS definition but is widely used to store records. Taking this further, Office 365, Microsoft Teams and Google Drive are now becoming widely adopted and in some organizations are replacing the EDRMS. Though it is certainly a challenge to preserve records that sit within a structured and controlled EDRMS, it is perhaps more of a challenge to work with records that reside in a less rigorously controlled system.
Though some of the advice and guidance contained within this resource will be applicable to record keeping systems that are not considered to be EDRMS, it is recognised that the community will face emerging challenges as the preservation of records within these (more loosely controlled) systems become a priority.
Several presentations that have touched on this issue are listed below:
-
John Sheridan, The National Archives UK suggests that we need to be ready to accept a more chaotic approach to record keeping going forward - ‘Taking hold of the record’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021 (DPC members login to view the recording)
-
James Lappin, University of Loughborough discusses the tensions between manageability and reliability in record keeping systems and the move towards Microsoft and Google cloud suites - ‘The evolution of records systems in the digital age’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021(DPC members login to view the recording)
-
Paul Young, The National Archives UK reports on his investigations and experiments into the export and preservation of records held in Google Drive - ‘Records in the cloud: The challenge of transferring records from Google Workspace’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021 (DPC members login to view the recording). Paul also summarises some of this work in a blog post entitled ‘What’s Up, (with Google) Docs? – The Challenge of Native Cloud Formats’
-
Patricia Sleeman, UNHCR touches on the challenges of the preservation of Office365, describing them as “tanks without drivers, rolling away with all our information” - Winners Webinars, 10th December 2020 (DPC members login to view the recording)
Processing the lessons learned
Whichever approach has been taken to preservation, it is important that lessons learned are captured and shared. It may be that any problems and pitfalls you have encountered in this process can be headed off in advance next time you need to carry out a similar task. Also consider whether the path to digital preservation can be smoothed for digital content residing in other systems. You should now have a line of communication with colleagues in various departments who should be receptive to talking to you about how things have gone and may be able to work with you to enact changes to improve the process.
Here are a few points to consider:
-
How will you report on your preservation work and share lessons learned with colleagues?
-
Can any updates to preservation and/or records management policy and procedures be suggested to improve things for next time? Can you use them to influence or improve on the record creation or management process?
-
Can any lessons learned related to the record keeping system itself be absorbed - what would you look for in a future record keeping system to make this process easier next time? How can you influence this process? Note that many of the questions included within this resource may also be considerations when looking for a new record keeping system. The DPC Procurement Toolkit may also be a helpful resource to consult.
-
Are there parts of your work that can be shared with the wider digital preservation community? Sharing of case studies and lessons learned are always welcomed. Do contact the DPC if you would like to provide a blog post or a case study to link from this resource
Taking action
Hopefully your business case or bid for resources to enable you to preserve the records has been successful and you are now in a position to start putting your plans into action. This is the biggest and most challenging step in the process, but all the groundwork you have laid up until this point will stand you in good stead. As this is a substantial piece of work it is recommended that detailed planning is carried out and project management techniques are used to manage and keep track of timelines, goals and resources. Ensure that you tap into support and guidance that is available in your organization. |
“Good and thorough planning are essential and it is never too early to be planning the transfer from one system to another - every system is a legacy system waiting to happen.” Rachel MacGregor, University of Warwick |
Kyle Browness from Library and Archives Canada gave a talk at the DPC’s Connecting the Bits unconference event in June 2020 entitled ‘The challenges and lessons of processing records from an EDRMS‘. He describes Library and Archive Canada’s approach to the project and gives an example of phases and deliverables.
The steps you will take will vary depending on the preservation approach that has been selected. For example, if the selected preservation approach is to export or transfer records to a digital archive the following steps will need to be carried out. Specifics will vary depending on the organizational context and the technologies in use.
|
As the steps listed above are very generic, it may be most helpful to you to find out in more detail how others have tackled this challenge. Some case studies are provided below but note you may find it most helpful to find someone who is using the same record keeping and/or preservation system as you and gather more details from them as to how they approached the exercise.
-
Elvis Valdes Ramirez from the International Residual Mechanism for Criminal Tribunals describes in this case study how they established an export and transfer workflow for records held within their EDRMS and an application that was developed to automate the packaging and structuring of metadata and associated objects for ingest.
-
James Doig from the National Archives of Australia provides an overview of work that has been carried out to transfer records from their EDRMS to the digital preservation system. This case study discusses some of the stages in the process of establishing a transfer methodology and some of the specific challenges that were encountered.
-
Kyle Browness from Library and Archives Canada gave a talk at the DPC’s Connecting the Bits unconference event in June 2020 entitled ‘The challenges and lessons of processing records from an EDRMS‘. He provides details of some of the decisions that were made in order to establish a transfer workflow, including those that relate to the metadata that will be exported alongside the records.
-
In a talk at the EDRMS Preservation Briefing Day, Zsuzsanna Tözsér Milam from the European Central Bank described their semi-automated workflows for the transfer of records from an EDRMS at ECB and the rationale for the approach they have taken. ‘Preserving records at the ECB’ a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021.
As can be seen from the selection of case studies presented, there is no one way of tackling this challenge and no one export tool or standard in use. It should be noted that work is currently underway by the DILCIS Board to create a Content Information Type Specification for ERMS (Electronic Records Management System) and an export tool to go alongside this. A talk by Karin Bredenberg (Kommunalförbundet Sydarkivera) at the 2021 Briefing Day on EDRMS preservation can be viewed for further background on this initiative.
The winding road to a CITS ERMS - Karin Bredenberg (Kommunalförbundet Sydarkivera)
Remember that whichever approach you have taken, you will also need to establish what to do with any documentation and information you have gathered to inform your work. Consider which information should be preserved alongside the records and metadata to provide further context for future users.
Making the case
By this stage of the process you will have already spent a considerable amount of time communicating with colleagues, gathering information and coming to a decision about the best course of action. Before you can move forward with the approach you have selected, you may need additional resource to carry out the work. Even where resource is not required, having buy-in from senior colleagues will be important to ensure that they understand what you are doing and why.
The DPC’s Digital Preservation Business Case Toolkit provides a template and step-by-step guide for building a business case. Some of the preparatory work will already have been carried out (for example finding out about the content, assessing the risk and considering preservation approaches). Your main challenge will be in identifying what you and your organization need to achieve your aims, calculating costs and effectively communicating these needs.
In order to help establish methodology and costs you may find it helpful to carry out a small pilot project. This should help inform the business case and add weight to your arguments. A pilot project may be a substantial piece of work in itself and it may not be possible to do this without additional resource.
Selecting a preservation approach
As with most digital preservation challenges there is no one-size-fits-all solution to the preservation of records from an EDRMS or other record keeping system. The approach that you might take to address the problem will depend on a number of different factors, including:
-
Whether the system is still in current active use
-
Whether the system is still supported
-
The level of digital preservation experience within your organisation
-
The software, tools and technological skills available to you
-
The value of the content and length of time it needs to be kept
-
Current and future users and use cases for the records
-
The preservation policies of your organization
-
The size and complexity of the challenge (and the frequency at which it arises)
-
The risks highlighted in the steps above and risk appetite of your organization
The information gathering you have carried out in the steps above will help inform any decision you make on preservation approach and next steps. A number of different courses of action could be taken
-
Transfer records to a digital archive
-
Emulate the system or application that holds the records
-
Leave in current system and manage ‘in situ’
-
Migrate records to a new record keeping system
-
Making the decision
Whilst the first two approaches are more clearly focused on digital preservation, the others, focusing on management outside of a digital archive may be appropriate decisions in some contexts. It should be noted that these four approaches are not mutually exclusive. In some cases more than one approach may be taken. For example:
-
You may plan to migrate records from a legacy system to a new records management system to facilitate current access and management as well as transferring them to a digital archive for long term preservation;
-
You may plan to manage the records ‘in situ’ for the time being, with a view to taking further preservation action at a later date (for example in response to increased risks to the records).
A presentation Tim Gollins gave at the DPC briefing day on Preserving Semi-Current Records (DPC Members login to view recording) may also be helpful in aiding decision making. In his talk, Tim focused on two options, continuing to manage the records in a record keeping system or moving them to a digital archive. He looked at how much functionality was required in the use of those records and for how long this was needed, and came up with a simple matrix to help inform decision making. Do watch the full presentation to view this matrix and hear these ideas explained in full.
'Preserving Semi-Current Records: EDRMS Task Force - Reflections' - Tim Gollins from the National Records of Scotland,
At the end of her presentation at the DPC’s EDRMS Preservation Briefing Day, Zsuzsanna Tözsér Milam described a decision tree she has developed to support decisions about whether to transfer records to a digital archive or manage in situ (Members login to view slides and presentation)
Lorna Williams from The Bank of England has blogged about their decision making process to select a preservation approach for their EDRMS.
Transfer records to a digital archive
This approach involves moving the records from the record keeping system to a digital archive. There are two variations on this approach:
-
Establishing an automated transfer process that moves the records and associated metadata from the records system to the digital archive (perhaps on a regular schedule).
-
Establishing a manual or partially automated process involving export of the records, preparation for ingest and import into the digital preservation system.
Why might you choose this approach?
There are many reasons why you may consider this to be the best approach. For example,
-
Where the records no longer need to be maintained in a system that has records management functionality
-
Where the risks of leaving the content where it is are considered to be too high
-
Where you have an institutional policy to manage all content of long term value centrally in the digital archive.
Further examples and guidance on this approach are included in the later sections of this resource.
Emulate the system or application that holds the records
Emulation involves recreating the environment required to access legacy systems and content over time. Unlike migration approaches, the original records would not be altered over time, rather, the right environment would be maintained to enable the content to be accessed in its original format. In the context of EDRMS preservation this would involve emulating the record keeping system itself and allowing users to continue to access the records within their native environment.
Why might you choose this approach?
There are a number of reasons why emulation may be a preferred approach, for example:
-
Where there is considered to be value in retaining the experience of accessing digital records within the original record keeping system.
-
Where there are features of the records themselves (for example, relationships between records, structure or metadata) that could not be adequately captured through export, or replicated in the access system associated with the digital archive.
-
Where emulation is part of the preservation strategy of the organization.
Though many archives may find this approach too technically complex (or not a viable option at scale), initiatives like Emulation-as-a-Service Infrastructure are reducing the barriers to emulation.
In a talk at the EDRMS Preservation Briefing Day, Euan Cochrane from Stabilize introduced emulation as an option for the preservation of record keeping systems. A case study was provided by Sally Vermaaten and Andrew Bowers of the Gates Archive who described how they were using emulation to work with a legacy record keeping system and shared lessons learned and future plans ‘New opportunities in whole-system preservation for EDRMSs’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021 (DPC members login to view the recording)
New opportunities in whole-system preservation for EDRMSs - Euan Cochrane (Stabilize), Sally Vermaaten and Andrew Bowers (Gates Archive)
Manage ‘in situ’
This approach involves leaving the records where they are (for a period of time). It is not quite the same as a ‘do nothing’ approach as it involves a careful assessment of how safe the records are currently and may lead to some changes to the current environment to give greater assurance that they can be left alone for a defined period. The tools mentioned in the ‘Risk Assessment’ section above can help both with assessing the risks to the digital records in their current location, and in defining and suggesting actions that can help mitigate some of those risks. Remember that this approach may only be temporary and you may still plan to take further preservation actions at a later date. Time taken gathering information and engaging with colleagues up to this point will have been time well spent and the knowledge gained should be retained for future reference.
Why might you choose this approach?
There are many reasons you may choose this approach but it will typically be a balance of a number of factors. Some of these factors are listed below:
-
Where a risk assessment of how safe the records are currently has revealed a situation that is considered satisfactory (taking into account the risk appetite of the organization) or where actions for reducing those risks have been highlighted and can be enacted.
-
Where the records are nearing the end of their life and are not considered to have longer term value - it may not be sensible to go through a potentially costly and time consuming transfer exercise, only to delete them from the digital archive a few years down the line.
-
Where there is digital content elsewhere within the organization at higher risk that must take precedence over these records at this point in time.
-
Where there is no digital preservation solution in place at the organization and no immediate resource to establish one.
-
Where the organizational policy sets out a decentralised approach to digital preservation with a specific focus on managing digital content ‘in situ’ where appropriate.
In a talk at the EDRMS Preservation Briefing Day, Esther Maes from TU Delft presented a short case study on their decentralised approach to the preservation of records, focusing on conversations with system owners and a methodology for assessing risk. ‘Decentralized archiving of records at TU Delft’, a presentation at Unbroken records: A briefing day on Digital Preservation and EDRMS, 20th May 2021 (DPC members login to view the recording).
Decentralized archiving of records at TU Delft - Esther Maes (TU Delft)
Migrate records to a new record keeping system
This approach involves migrating the records to a new record keeping system to continue their active life. Though not a preservation action as such, this work should very much be carried out with preservation in mind if assessment of the records in earlier stages of this process has demonstrated that the system holds content of long term value.
Why might you choose this approach?
You may choose this approach in the following circumstances:
-
Where the records are still in active use or still being actively updated and records management functionality is still required.
-
Where the software holding the records is coming to the end of its life or a support contract no longer exists.
-
Where the digital archive is unable to provide the required management or access to the content.
If the records have long term value it is important that preservation considerations are taken into account during the process of migrating to a new record keeping system. Important features of the records, including their context and metadata must be maintained so that it can be captured and preserved at a later date. Note also that documentation should be kept of the system migration as this will be helpful in informing future preservation work.
The National Archives UK provides a useful guide to Migrating Information between Records Management Systems including a section on defining digital continuity requirements.
Though it may be colleagues who will lead this piece of work to select and implement a new system, you are in a good position to influence this work and ensure that digital preservation considerations are taken into account, both in decisions made and in the configuration of the new system. The metadata section [link to metadata work] and some of the lists of questions included within this toolkit may be helpful in highlighting some of the features, functionality and metadata requirements required to facilitate future preservation of the records.
Assessing the risks
Once you have gained a good understanding of both the system, the content and the records management processes and procedures that are in place, you may find it helpful to assess the current risks to the digital content within the record keeping system. Some of the information you have gathered in previous steps may already have given you an indication of risk, and you may already have enough evidence to make a decision on preservation approach and to back up these recommendations. There are however tools and models that you can use which will help you to better understand and communicate some of these risks. Doing so may help you to:
-
Reach a decision on the best preservation approach to take
-
Understand how urgently you should take action
-
Provide evidence to back up your case for action
-
Develop shorter term mitigations to help manage risks
There is no single risk assessment tool suitable for assessing digital content in record keeping systems specifically, though there are methodologies in use more generally for assessing digital preservation competency and risk. Some tools and models that you may want to consider are described below but this is not an exhaustive list. You may prefer to select one that your organisation is already familiar with. Tests by members of the EDRMS Preservation Task Force suggest that there may be a benefit in trying out a combination of the methodologies listed below. Combined, these methodologies can provide a really clear understanding of the risks and help to frame and evidence the case for action.
Note that it can be particularly helpful to be able to compare the assessment of the record keeping system with an assessment of the digital archive. This gives a quick and easy understanding of how much risk can be mitigated by moving the digital content to the archive. In particular, a comparison like this is helpful to show colleagues who are not as familiar with digital preservation and may not have a clear understanding of how a record keeping system differs from a digital preservation system.
A postgraduate dissertation from Zsuzsanna Tözsér Milam investigates the use of models such as the NDSA Levels of Preservation (version 1) for assessing the risk to content in an EDRMS, using the European Central Bank as a case study: Digital preservation - Requirements for preserving authentic, reliable and usable records in an EDRMS over the long term: A case study (2014).
DiAGRAM tool (the Digital Archives Graphical Risk Assessment Model)
![]() |
The Digital Archives Graphical Risk Assessment Model (or DiAGRAM) tool was created by The National Archives UK and the University of Warwick as an output of the ‘Safeguarding the Nation’s Digital Memory’ project in 2020. The DiAGRAM tool has specifically been designed to help organizations understand and quantify digital preservation risks and to use this information to facilitate evidence-based decision making. The tool takes users through a set of questions which need to be answered in order to produce a visualisation of results and risk score. A score is given for the two main nodes of the underlying model:
|
DiAGRAM provides a helpful way of assessing and quantifying risk to content within a record keeping system. There are various ways it can be used as described below. Modeling the current situation is the obvious first step but further models can be created to explore other scenarios:
-
Model the current situation (records within the record keeping system): use DiAGRAM to assess risk to content within the record keeping system if it is left as is.
-
Model the digital archive: use DiAGRAM to model risks to content within the digital archive (assuming you have one in place). This gives a useful point of comparison to the scores obtained in the step above and allows you to see how much risk could be mitigated by transferring content to the digital archive.
-
Model a future scenario: use DiAGRAM to make changes to your initial model, considering the changes you could put in place to reduce risks to the digital content within the record keeping system. This step can be repeated as many times as is appropriate to explore numerous risk mitigations.
The DiAGRAM tool is freely available to use. It has been created in the first instance for a UK audience but may still be relevant and helpful to those outside the UK. Some initial preparatory work is required before using the tool itself. Users are encouraged to download the questions before using the model in order to ensure all relevant information has been gathered. It should also be noted that the DiAGRAM tool requires users to have completed some sections of the NDSA Levels of Preservation and DPC RAM so that this information can be incorporated into the model.
The DiAGRAM tool is user-friendly but adequate time should be set aside to use it. Users report that the most challenging aspect of the tool is finding accurate answers to the questions to input into the model. Time should be spent talking to colleagues in IT and other sections to find out the answers, but the groundwork carried out in earlier steps of this process and lines of communication that have been set up will be invaluable in moving things forward.
It should be noted that the DiAGRAM tool does not cover all possible risks to digital content and therefore should not be used as the only input to inform decision making. It does not include, for example, questions around vendor support or the risks of continuing to store records in a legacy system.
NDSA Levels of Preservation
The NDSA Levels of Preservation is a well-established and widely used tool for rapidly assessing digital preservation capabilities. Focusing on the technical aspects of digital preservation, it consists of a simple grid covering five key areas of digital preservation (storage, integrity, control, metadata, content). For each of these areas, an organization simply needs to define which level they are working at, using the prompts that are included within the model. The Levels are quick and easy to use and a spreadsheet tool is available to record and visualise results. The tool and associated documentation are available here: https://osf.io/m2fek/
The NDSA Levels have been designed to measure digital content that is already in a digital archive, but particularly with the latest revision to the Levels (2019) it is possible to use it as a means to assess content that resides elsewhere. Tests by the EDRMS Preservation Task Force showed that the Levels could be used to assess content within a record keeping system though it was noted that not all areas of the Levels were applicable outside of a digital archive (in particular where standard ingest procedures such as the use of write blockers and virus checking on ingest were directly referenced). Tests of using the Levels in this context demonstrated that it was possible to score relatively high on the ‘Control’ row, as many record keeping systems do this well. The ‘Content’ row (which focuses on understanding and maintaining the content over time) tended to score lower as this is not an area that is typically covered by record keeping systems.
DPC RAM
![]() |
DPC’s Rapid Assessment Model (DPC RAM) is a maturity model for digital preservation. Specifically designed to assess the digital preservation capabilities of an organization it covers 11 areas and includes both organizational capabilities (such as organizational viability, policy and strategy, IT capability) and service capabilities (such as bitstream preservation, content preservation, metadata management). Like the NDSA Levels of Preservation it is focused on measuring digital preservation capabilities and has not been designed to enable the assessment of content that resides in a system outside of the digital archive. It is included here for two reasons. |
-
Firstly the DiAGRAM tool (above) requires the user to enter a score for the ‘Content preservation’ and ‘Metadata management’ sections of DPC RAM. Even if you do not feel it is appropriate to carry out a full assessment using DPC RAM, you will need to think about these two sections when using the DiAGRAM tool.
-
Secondly if you are already using DPC RAM to monitor your digital preservation maturity over time, you may find it helpful to assess the records in the record keeping system as another stream of content using just the five Service Capabilities of RAM. This could provide a good point of comparison to your known digital preservation capabilities and may provide a constructive way of talking about the differences between the functionality of a record keeping system and a digital archive with colleagues.
Understanding the metadata
“One of the main areas to think about is metadata. It is important to consider this from the start. If migrating into a new EDRMS it is important that you don’t lose or corrupt metadata which could provide important context. It’s also important to think about how you are going to export metadata when moving to another EDRMS or to be preserved in a digital archive.“ Paul Young, The National Archives UK |
Spend some time investigating the metadata that is stored alongside the records in the record keeping system. Note that it may not be immediately obvious what metadata exists - some of it may only be available to certain users, or may only be available in the back end. It is advisable to carry out some research to fully understand what is stored within the system and how you might access this. Carrying out a test export of records and their metadata may be helpful. Note that some metadata fields may not be included within a standard export - for example, an EDRMS may store a checksum for a digital object but this may be held behind the scenes and not made accessible to users. Talk to those who develop, support, configure the system to find out more about metadata and export capabilities. Find out whether the system automates the completion of some metadata or has mandatory fields which a user must complete. If the onus is on the creator to complete certain fields, are there conventions to follow and is there compliance? Assess the accuracy of metadata and the impact of errors e.g. spelling mistakes, which may affect the accuracy of searches that you carry out when trying to discover records for export. |
The EDRMS Preservation Task Force discussed and defined a set of metadata fields that may be captured from a record keeping system in order to preserve the records, their meaning and context. This work took as its starting point a draft minimum metadata set shared with us by Kyle Browness from Library and Archives Canada. There was discussion about whether to assign a priority rating to metadata fields (e.g. optional/recommended/mandatory) but it was recognised that different organizations working in different contexts may not require the same fields. It was agreed instead to include information on the rationale for capturing certain types of metadata, as this will help users of this resource to consider whether a field is required in their context or not. Different functionality within record keeping systems and differences in local implementation and configuration also mean a one-size-fits-all approach is not appropriate. Note too that additional factors which need to be considered in decision making include:
-
Does the record keeping system capture/store this information?
-
Can this information be extracted from the record keeping system?
-
Can this information be stored within the digital archive?
Hugh Campbell from Public Record Office of Northern Ireland (PRONI) discusses their approach to metadata in this case study. He notes some of the decisions that were made about which metadata fields to keep and some challenges that have been faced, as well as sharing their metadata template.
In a talk at the EDRMS Preservation Briefing Day in May 2021, Kevin De Vorsey and Bethany Cron from NARA presented on ‘Federal Electronic Records Modernization Initiative : lessons learned in electronic records preservation’. Their presentation gave an overview of work to ensure that the records they receive from federal departments are suitable for preservation, including guidance relating to metadata (Members login to view the presentation).
Federal Electronic Records Modernization Initiative : lessons learned in electronic records preservation - Kevin De Vorsey and Bethany Cron (NARA)
Understanding the digital objects
It is important to gather information about the individual files or digital objects that make up the records. This will help inform your preservation work and will enable you to flag some of the more specific preservation challenges you face. Depending on the record keeping system in use, there may be no easy way to gather the necessary information. Ideally you would be able to run a tool such as DROID across the contents of the record keeping system to characterise the files and gain an understanding of the content, but depending on how the digital objects are stored, this will not always be possible. You may alternatively be able to find out about the digital objects from metadata that is held within the system itself, or by asking questions to users and administrators of the system. Gather what information you can but accept the fact that you may not be able to gain a full and clear understanding of the situation until the records have been exported and are held outside of the system. |
“EDRMSs do not have digital preservation built into them, even though long-term temporary records need to be kept for 75+ years. In Australia we point agencies to sustainable formats that have the best chance of remaining accessible over time, but the business needs of agencies may require, in particular cases, formats that aren’t sustainable in the long term.” James Doig, National Archives of Australia |
File formats
Try to establish which file formats are included within the system. Are there any restrictions on formats that a user can create and/or upload or are users able to save records in whatever format they like? If you are unable to run file identification tools over the records stored within the system, find out if the system metadata includes any potentially useful information such as MIME type or file extension.
Complex records
Find out whether the system includes complex records that consist of multiple digital objects - for example, a webpage, an email with attachments or a geospatial record consisting of multiple files. If this is the case you will need to be particularly careful that your preservation methodology maintains the relationships between the files and their metadata and be prepared to test and check that they are captured and preserved satisfactorily.
File migrations
Find out whether any file format migrations have been carried out within the system and if so, whether details have been logged. Most record keeping systems won’t carry out file format migration but some examples have been noted of systems creating PDF versions of documents or altering the file format of emails. If file formats have been changed within the system, find out whether this is recorded or logged anywhere and whether this information can be extracted alongside the records.
Legacy formats
Find out when the digital objects were created or last modified? Have they been accessed recently? Again, some of this information can be gathered using characterisation tools such as DROID, though you may find that more detailed audit logs are available within the system metadata. Perhaps your investigations and conversations up to this point will have highlighted problems in accessing records of a certain age or held in legacy file formats, and this information will help you further understand risks and challenges related to specific records.
Use the information that you have gathered to better understand some of the preservation risks to the content and to gain a basic understanding of additional preservation actions you may need to take once you have these files in the digital archive.
In his case study from the National Archives of Australia, James Doig stresses the importance of gaining this understanding of the digital objects prior to transfer in order to flag up and understand some of the issues that you will encounter.
Understanding the records and the processes that created them
“I think the main principle I will take from this work is the significance of robust records/information management practices. Record keeping strategies and policies and the retention triggers that can be built in to an EDRMS can, in the long run, streamline preservation activities. Activities such as transfer and ingest to a preservation system, can be time consuming (depending on the workflows in place), so ways to streamline or automate actions will help in the long run.” Nicola Steele, Grosvenor Estates |
Different record keeping systems facilitate different ways to manage records and these systems may be configured in different ways to fit organisational needs. Gather information about how the records have been managed, described and stored and what processes and procedures are in place. Finding the right people to talk to and ask questions of is key to moving forward. The questions you ask can also be usefully supplemented by reading relevant information management and data governance policy and procedure documents in order to understand how users of the system are encouraged to work.
“Even though EDRMSs usually have (by definition) disposal and appraisal functionality built into them, it is still very much a manual and resource-intensive process to determine archival value records, export them, and transfer them to the archive. Current and future work must look at automating value determinations through machine learning and AI techniques." James Doig, National Archives of Australia |
One of the biggest preservation challenges you may face is establishing which records are of long-term value and should be preserved, but if the system and procedures for using it have been well managed, you may find this problem has already been solved. It is worth spending time understanding how this has been applied to establish whether any additional appraisal work is needed prior to preservation.
If documents are not managed in such a formal way, for example they are shared across platforms in Microsoft Office 365, analysis for preservation is harder and you will need to consider how you identify and safeguard the records of long-term or permanent value.
Some record keeping systems allow sensitivity classifications to be applied to documents. Find out who has applied them and whether they have been used accurately and consistently. Consider whether your preservation approach is suitable for all classifications of sensitivity or whether certain classes of record will need to remain within the system.
“It is not always clear what the ‘record’ is in an EDRMS. Agencies create systems to meet their often unique business requirements and while it may be understood that their contents contain permanent records that will eventually be transferred out, they may also contain information that is not appropriate for transfer. It is a laborious process to pick through a complex system to determine which bits are really of interest.“ Kevin De Vorsey, National Archives and Records Administration |
This section includes further questions to help identify the records management processes in place and what impact they will have on the records and metadata that reside in the system.
Questions to ask include:
|
Understanding the technical infrastructure
You will need to take some time to understand the nature of the system or application in which the records are held. Note that some of the key contacts you have highlighted in the step above will be invaluable in answering all the many questions you will have. Gaining an understanding of the technical infrastructure in place will be invaluable when you start to consider preservation approaches but will also help with assessing risks to the content as mentioned in a later step of this process.
The questions you will need to ask include:
|
Make sure you document the answers to these questions and keep copies of any relevant documentation and system information you gather.
You may also want to start asking questions about export options, finding out about how records (and metadata) can be extracted from the system. Some detailed questions are included below. As noted above, this is not a linear process and you may want to hold fire on some of these questions until you have a clearer idea of the risk and your preservation approach.
Ideally, the record keeping system should include a means of exporting the records and associated metadata from the system in a practical, configurable and comprehensive manner. Unfortunately, some systems have limited options and functionality for export and this important requirement is not always seen as a priority at the point of procurement. Off-the-shelf systems often provide limited export options, while custom installations may lack export functionality altogether requiring that a programmer be engaged to extract records and associated information from whatever underlying database was used.
Questions related to export include:
|
Subcategories
Template for building a Business Case
This section provides guidance on the content that will be useful to include in your business case, but it will likely need to be adapted to the structure used in your organization’s template.