Posted in Records management, Electronic records, Digital preservation, XML, Retention and disposal, Conservation and preservation, Exchange 2010, Information Management, Exchange 2013, Access controls, Exchange Online

The enduring problem of emails as records

Ever since emails first appeared as a way to communicate more than 30 years ago they have been a problem for records management, for two main reasons.

  • Emails (and attachments) are created and captured in a separate (email) system, and are stored in mailboxes that are inaccessible to records managers (a bit like ‘personal’ drives).
  • The only way to manage them in the context of other records was/is to print and file or copy them to a separate recordkeeping system, leaving the originals in place.

Thirty-plus years of email has left a trail of mostly inaccessible digital debris. An unknown volume of records remains locked away in ‘personal’ and archived mailboxes. Often, the only way to find these records is via legal eDiscovery, but even that can be limited in terms of how back you can go.

Options for the preservation of legacy emails

The Council on Information and Library Resources (CLIR) published a detailed report in August 2018 titled ‘The Future of Email Archives: A Report from the Task Force on Technical Approaches to Email Archives‘.

The report noted (from page 58) three common approaches to the preservation of legacy emails:

  • Bit-Level Preservation
  • Migration (to MBOX, EML or even XML)
  • Emulation

In a follow up article, the Australian IDM magazine published an article in March 2020 by one of the CLIR report authors (Chris Prom). The article, titled ‘The Future of Past Email is PDF‘, suggested that PDF may be (or become) a more suitable long-term solution for preservation of legacy emails.

Preservation is one thing, what about access

There is little point in preserving important records if they cannot be accessed. The two must go together. In fact, preservation without the ability access a record is not a long different from destruction through negligence.

Assuming emails can be migrated to a long-term and accessible format, what then?

No-one (except possible well-funded archival institutions perhaps) is seriously likely to attempt to move or copy individual legacy emails to pre-defined and pre-existing containers or aggregations of other records. This would be like printing individual emails and storing them in the same paper file or box that other records on the same subject are stored.

Access to legacy emails in an digitally accessible, metadata-rich format like PDF provides a range of potential opportunities to ‘harvest’ and make use of the content, including through machine learning and artificial intelligence.

These options have been available for close to twenty years in the eDiscovery world, but to support specific legal requirements.

Search, discovery and retention/disposal tools available in the Microsoft 365 Compliance portal, along with the underlying Graph and AI tools (including SharePoint Syntex) provide the potential to manage legacy content, including emails.

The starting point is migrating all those old legacy emails to an accessible format.

Posted in Electronic records, Exchange 2013, Products and applications, Records management, Retention and disposal, SharePoint 2013

Recordkeeping functionality in SharePoint 2013 – What’s New?

By all accounts so far, SharePoint 2013 does not have a lot of new records management functionality, but the new functionality it now has is an important step forward.  It includes:

  • Records management in the cloud, but a parallel universe to existing on-premises deployments of things like the Records Centre, Content Type Hub, and Managed Metadata.
  • The ability to apply retention policies to entire sites.
  • Better integration with Outlook, including site mailboxes.  All other email will still need to be managed.
  • Better support for eDiscovery.

Records management features in Office 365

It will now be possible to do most on-premises records management activities online in Office 365 (i.e., in SharePoint 2013 online). This includes the ability to create and use Records Centres, Content Type Hubs, Managed Metadata, and in-place records management. (See Don Lueders’ interview with Adam Harmetz from Microsoft here for more information.)

However, as Mike Alsup from AIIM points out here, on-premises and cloud-based environments will remain completely separate. It will not be possible to send records from Office 365 to the on-premises Records Center, and vice versa.

Retention policies for entire sites

This is a welcome feature as the only realistic alternative in SharePoint 2012 was to make a site read only and then ‘archive’ it.

See Microsoft’s introduction and overview to this issue for more information.

Better integration with Outlook

This is a very interesting move by Microsoft who have made it very clear that they believe content should be kept where it belongs – email in Exchange and documents in SharePoint.

In SharePoint 2013, teams can have a site mailbox that is visible from Outlook as well as the site. Documents attached to emails can be saved easily to a SharePoint document library. With retention applied to an entire site, this means that the emails in the site mailbox will be kept – in its business context – along with the rest of the relevant content.

This leaves ‘Messaging Records Management’, a feature that has not changed in Exchange 2013, to cater for all other emails.

Better discovery

SharePoint 2013 includes new eDiscovery sites. Some might say this is not a records management feature but a feature for lawyers. In any cases, both are likely to find this new feature compelling.