I’m often asked (and sometimes challenged to confirm) if SharePoint can manage records. The question is often based on a unspoken second element – like ‘xyz’ system?
It might be (and sometimes is) argued that any system can manage records if the records stored in the system provide evidence of business activity and are considered information assets. But recordkeeping is more than just keeping any records in any system.
The international standard for records management, ISO 15489-1-2016 states: ‘They (records) can be distinguished from other information assets by their role as evidence in the transaction of business and by their reliance on metadata. Metadata for records is used to indicate and preserve context and apply appropriate rules for managing records.’
So, records are not just distinguished by their evidentiary role, but also by their reliance on metadata.
Record types
In my opinion, it is a mistake to focus solely on SharePoint when the question is asked – can it manage records? The question assumes that one application will be used to store all the records – or at least the so-called ‘unstructured’ records – created by the organisation. Any discussion of SharePoint must include and take account of SharePoint Online as part of the Office 365 ecosystem.
The term ‘unstructured’ in this sense generally means email and any kind of digital record that can be saved to network file shares. The latter generally includes documents (in multiple formats), images, and other digital record types.
However, ‘what gets saved on a network files share’ generally overlooks the fast-increasing volumes of information that would not normally be ever stored in a file share. Simple examples of records that are never (or may never be) saved in network file shares (or dedicated recordkeeping systems) include tweets on Twitter, messages sent by personal messaging apps, and social network type information.
I’m always impressed (albeit a bit sceptical) when I hear that organisations state they are capturing all these types of information in their recordkeeping system. That’s pretty impressive.
Records in Office 365
Many organisations have moved (or are moving) their Microsoft enterprise licencing to Office 365, Microsoft’s subscription-based service. Office 365 includes a range of applications that create or store records including:
- Exchange (email, calendars, Groups, Planner)
- Office (used to create the content)
- SharePoint / OneDrive for Business* (document libraries and lists)
- Microsoft Teams
- Sway
- Skype for Business
- Stream (video)
*OneDrive for Business is a SharePoint-based service.
SharePoint is only one part of this information rich ecosystem, and really shouldn’t be thought of as a single destination for the storage of records. Yes, it can be used to store and manage records, but you need to stand back a little to appreciate the full picture.
How Microsoft (may) have approached recordkeeping in Office 365
Until recently, and in the on-premise world, records were stored and managed separately in Exchange and SharePoint, each with their own recordkeeping capabilities, quite independent of each other.
Over the past two years, Microsoft developed a unified strategy for recordkeeping in both systems, presumably based on the likelihood that most corporate records would continue to be stored separately. The requirement for additional recordkeeping metadata in either system would remain optional – see below.
In mid 2017, Microsoft introduced a centralised way to create, manage and apply retention policies to content stored across (no longer separately in) Exchange, SharePoint and OneDrive for Business. These new policies are created as labels in the Office 365 – Security and Compliance Portal under the slightly misleading section called ‘Classifications’.
These new retention policies can be applied across Exchange, SharePoint and OneDrive for Business. In SharePoint, they can be applied to a site, a document library (preferred), list, or individual documents. They remove the requirement to manage retention separately in Exchange and SharePoint.
But what about the metadata?
As noted above, the international standard for recordkeeping states that metadata ‘is used to indicate and preserve context and apply appropriate rules for managing records’.
So why or how is metadata optional in Office 365? I think this is for two reasons:
- Making any form of metadata mandatory will turn users off using the system.
- Metadata may not be the ‘be all and end all’ for context-based discovery.
‘Context’ essentially means that records relating to a given context (e.g., ‘noise complaints’) can be identified, retrieved and managed in that context. For example, emails relating to a meeting; the meeting agenda and minutes may be stored in one location but more often than not the emails remain on the Exchange server. Another example might be emails relating to the development of a new policy; again, these are more often than not stored separately from the system used to store and manage documents.
Regardless of where they may be stored, metadata should provide and indicate the context of any records that may be created. Years of EDRMS use suggests that users generally don’t like to add additional metadata to records.
So how does Office 365 do this?
In most organisations, the only ways to apply recordkeeping metadata to an email is to save it to an EDRMS or in SharePoint. Most organisations will rarely configure Exchange to include the capture of metadata.
As with an EDRMS, the metadata options in SharePoint are more or less unlimited but careful thought needs to go into what metadata should be applied, and how. For example, metadata can be set:
- In the Managed Metadata Service (MMS)/Term Store, including with hierarchical models
- As site columns
- As library columns
Regardless of the option selected, metadata may be set as a default on each SharePoint document library column. That is, when a record (including an email) is saved to a specific library, it can be assigned specific metadata that is to be assigned to all documents in that library.
Applying metadata in this way, especially as site columns, means that information can be retrieved in context.
It should also be kept in mind that Microsoft Office documents saved to a SharePoint document library also retain their metadata in the document properties, even when the document is exported, a kind of ‘metadata payload’.
Is metadata still relevant?
In the mid 1990s, Yahoo introduced a new portal that allowed users to browse the nascent internet based on pre-defined categories. That is, a form of metadata tagging was applied to all content that allowed the user to browse to where they wanted to go to.
The problem with this idea was that it assumed everybody would understand the categories. Google’s response to this was to provide a single search box allowing users to retrieve whatever they were looking for – subject, of course, to the way the algorithm presented the information to the user based on their understood context.
Adding metadata to indicate the context of a record works as long as the context is still valid – both for the content and the user. Or, to put it another way, the way in which I might describe a record with metadata may be different from the way you want to access that record, because your context is not the same as mine. There may be a range of information that I want to find that hasn’t necessarily been recorded in the context in which I am looking for it.
Some years ago I was curious why users in one business area could not find many records relating to a specific subject – noise complaints in a city area well known for its nightlife. In most cases, they were searching for records containing complaints about noise in that specific area, recorded in the title or metadata of the record.
When we asked them to ignore the metadata and search by the content of the records they found thousands of records, all described in different contexts – building approvals and inspections, delivery of services, police liaison, visitor numbers and public feedback. All these contexts were quite valid, but they were not the context of the user searching for the records.
The lesson learned was simple – my context is not necessarily your context. Records, especially digital records, could relate to any context including future and unpredictable contexts.
Context-based information and eDiscovery
For some users, one of the most ‘startling’ features of Office 365 is Delve and the related Discover option in the user’s OneDrive for Business. Both are based on the underlying Office Graph that learns a user’s context based on their interactions (or ‘signals’) across the Office 365 environment and presents potentially relevant content (to which they have access) from SharePoint or another user’s OneDrive.
I used the term ‘startling’ because, for most users, the idea that you can find out what others are working on seems intuitively to be some kind of breach of privacy (even though they have access to that content already). And yet, what it is doing is letting a user know, based on her or his context, what may be of interest from potentially quite different contexts. It does this based on the interactions between users.
Office 365 also includes a powerful eDiscovery capability that allows the user (if licenced to do so) to find all information across Exchange and SharePoint relating to a specific context regardless of where it is stored, and quarantine that information as required in a case file. While metadata may assist in the process, it is not essential.
But what about all the other records?
So far I have not said anything about the records produced by and stored in the other Office 365 applications such as Teams, Planner, Skype for Business and so on. Or about the management of records produced in third-party social media or messaging applications.
The Office Graph already takes into account the interactions between users to present potentially relevant information stored in SharePoint or OneDrive for Business. At some point in the future, Microsoft may include the information in the various other Office 365 applications.
As for social media, the preferred model may be to capture the feed of that information in an Office 365 service – Microsoft Teams, for example, can receive a feed from third-party applications including Twitter. The answer to the use of third-party messaging applications is to use applications that have at least the same or, preferably, better functionality. Teams and Skype for Business are in this space.
Summary
If you have got to the end of this article, thank you for reading.
In summary, my main point is that when thinking about SharePoint for recordkeeping it is a good idea to consider it in the context of the broader Office 365 ecosystem and its recordkeeping capabilities, not as an isolated application capable of storing and managing records.
Records about the world 