Audit trails for records in SharePoint Online

As evidence of business activities, records should be authoritative.

According to ISO 15489:2016, the characteristics that define authoritative records are authenticity, reliability, integrity and usability. These characteristics may be met by either:

• Securing or locking a record to prevent unauthorised changes (‘declaring a record’).
• Recording the details of any events – the ‘event history’ – relating to the record to show ‘who did what, when and (possibly) why’.

According to several publicly accessible records management standards, examples of event history metadata include when records are: created, captured, viewed or accessed, edited/modified, copied, downloaded, transferred or exported, reviewed, sentenced, migrated, converted, and deleted.

This post briefly notes, for comparison purposes, how traditional EDRM systems record event histories. It then describes the event history information captured by and recorded in SharePoint Online, and (separately) in the Microsoft 365 unified audit logs. The post underlines the need for records managers to understand what event history is captured (in either SharePoint and/or the unified audit logs) and how that information can be accessed and managed, if required, for the life of the records.

TL:DR: Key points that records managers need to know

• All metadata relating to a record stored in SharePoint is destroyed when the record is destroyed. A limited set of metadata is retained for destroyed records that were subject to disposition review labels. To ensure all the required information is retained, it may be a good idea to export all required metadata from a library – including for review – and store them separately before the records are destroyed.
• The Microsoft 365 unified audit logs record details about a wide range of activities across the platform and contain a LOT of content not relevant to managing records. As the logs may only be retained for three months or 12 months depending on your licence, it may be a good idea to export sub-sets of the audit logs on a regular basis – for example, all audit records for specific sites for a given period (week or month) and then store them with the site or separately.
• Alternatively, consider acquiring a third-party system that can capture and retain the event history for records stored in SharePoint.

Event history metadata in EDRM systems

Traditional Electronic Document and Records Management (EDRM) systems have three ‘tiers’. The back-end or third tier usually consists of two parts:

• A database where the metadata about the record, including event histories, is stored.
• An associated file store where the digital objects are stored.

This (common) architecture model allows metadata, including the event history, for records to be retained even after the original digital objects have been destroyed.

Event history metadata in SharePoint Online

SharePoint Online automatically records the ‘created’ (date), ‘created by’ (person), ‘modified’ (date) and ‘modified by’ (person) details for every item in a list or library as shown in the screenshot below.

The version history shows who modified the item and allows for previous versions (revisions) to be restored.

Unlike EDRM systems, SharePoint stores both the item and its metadata in the same content database. This means that all the metadata for each item, including its event history, will be destroyed when the item is destroyed, unless it is exported in advance.

(Note: Some records may be tagged with retention labels that include a disposition review. Some metadata about the record (e.g., Name, Location, Author, Destroyed Date, Retention Label) is retained after the record is destroyed. Records managers should check to see if this metadata meets their business requirements to retain the details of destroyed records.)

If there is a requirement to retain the complete set of metadata for each record, including the created and modified details, it should be exported (via a view) from the library before the records are destroyed.

Event history metadata in the Microsoft 365 unified audit logs

The Microsoft 365 unified audit logs record details of a wide range of activities across Microsoft 365, including certain records-specific activities such as ‘deleted file’.

The size of the logs can be very large even for a single day. The logs are only accessible for up to three months with an E3 licence, or 12 months with an E5 licence. This has implications for compliance if an organisation needs to retain event histories for longer. See below for suggestions on how to address this.

An audit retention policy may be created to retain parts of the audit logs (e.g., all SharePoint activities, or just selected activities) for up to 1 year, although in an E5 client tenant the option to retain them for 10 years is visible.

The unified audit logs are accessed through the Audit section of the Compliance center. The Compliance center can only be accessed by user accounts that have been assigned specific roles, including Global Admin, Compliance Admin, Compliance Data Admin, Records Administrator or a customised role. Additionally, the logs can only be accessed by user accounts that have been assigned to the Audit Logs role via an Exchange admin role.

The Audit section of the Compliance center presents the following interface.

Most of the audit information relating to items stored in SharePoint (and OneDrive) is found in the following ‘activity’ sections:

• File and page activities, including: accessed file, checked in file, checked out file, copied file, deleted file, deleted file from recycle bin, deleted file from second-stage recycle bin, downloaded file, modified file, moved file, performed search query, previewed file, renamed file, restored file, uploaded file, viewed page.
• Folder activities, including: created folder, copied folder, deleted folder, modified folder, moved folder, renamed folder
• SharePoint list activities
• Sharing and access request activities, including: shared file, folder or site
• Synchronization activities, including: downloaded files to computer
• Site permission activities
• Site administration activities (less likely to be useful)

The results of audit log searches can be viewed directly or exported to csv file. The results for all results include a range of metadata detail including the following ‘common’ elements:

• Date (event date, in UTC)
• IP Address (of the person who performed the activity)
• Users (e.g., agent, the email address of the person who performed the activity)
• Activity (e.g., Uploaded file)
• Item (e.g., ‘example.docx’)
• Detail (e.g., Uploaded to “Documents” library)
• Id (a unique GUID for the activity)
• User Agent (details of the application used to create the item and the computer)
• CreationTime (date the log item was recorded in UTC date and time, usually the same as the event date)
• Operation (e.g., ‘FileUploaded’)
• Workload (e.g., OneDrive, SharePoint)

Each activity includes additional elements specific to that activity.

The rest of this post focusses on examples of event history activities in the audit logs that are required for recordkeeping purposes.

Created/captured/uploaded

The audit logs records the creation of a new item in SharePoint or OneDrive as ‘Uploaded file‘, regardless of whether it was uploaded or created via the ‘+New’ menu option.

When created from the ‘+New’ menu option, Office documents are assigned the default name ‘Document’ (with a number suffix for each new document). If they are renamed, this is recorded in the logs as ‘Renamed file‘. This is important to note as a search in the logs for the new name will not return the ‘Uploaded file‘ result.

Note that the details do not include any other metadata when it was created, such as Document ID or other added metadata.

Viewed/previewed/reviewed/accessed

The audit logs records the details of any access to a ‘file’ stored in SharePoint as ‘Accessed file‘. A file in this case includes not only individual items (such as records) but also pages.

SharePoint and OneDrive provide the ability to ‘preview’ a file, for example when a page with images is opened (the images are ‘previewed’ rather than opened and viewed), a mouse hovers over a file name. This action is recorded in the logs as ‘FilePreviewed‘ although this is not listed as an activity; it only appears in the exported logs, for example to find events relating to a specific user.

(Note that ‘seeing’ items shown as tiny images (e.g., when the library viewed is changed to Gallery view) is not recorded as ‘FilePreviewed’ because they are too small to be legible.)

Edited/modified

The logs record when an item is modified in SharePoint as ‘Modified file‘.

However, the audit logs record ‘Modified file’ for each version. If the file is renamed, the only way to know that the ‘Modified file’ reference relates to the same document is through the ‘ListItemUniqueID’ in the log data. This ID is the unique GUID for the item. This number is also visible in the ‘Copy link’ reference URL.

Copied/Moved

The logs record the activities of ‘Copied file‘ and ‘Moved file‘, including details of both the source and destination locations.

Note that ‘Copied file’ may include the activity of deleting an item when a retention policy has been applied. This is because the item is not actually deleted but copied from the source library (where it is ‘hidden’ but becomes visible for 93 days in the Recycle Bin) to the Preservation Hold library (the ‘destination’).

Downloaded

The logs record who downloaded something from SharePoint or OneDrive, and when, as ‘Downloaded file‘. It includes all the metadata shown above including the IP address of the person who downloaded the item.

Transferred or exported / migrated / shared

The meaning of ‘transferred’, ‘exported’ and ‘migrated’ is understood to be the process of moving content to another organisation, rather than internally.

Neither SharePoint nor OneDrive have the option to ‘transfer’ or ‘export’ natively to another location. Instead, it is possible to download the content (which would be recorded in the logs as ‘Downloaded file’) and (separately) export the metadata of any SharePoint library or OneDrive content. This information can then be transferred or exported as required.

Alternatively, an authorised third party could be given access to SharePoint or OneDrive and from there could ‘download’ the content. This would also be recorded in the audit logs as a ‘download’ activity.

Any sharing from SharePoint or OneDrive is also recorded in the audit logs, mostly via the activity of ‘Shared file, folder or site’. They may also be recorded as ‘Created a company shareable link’, ‘Created an anonymous link’, ‘Created secure link’, ‘Created sharing invitation’.

Sentenced

There are two ways to sentence a record stored in SharePoint:

• Make it subject to a ‘back-end’ retention policy (‘destroy automatically’ or ‘do nothing’). Although they work against individual records, they do not ‘tag’ them; nothing is visible in the metadata of the record.
• Make it subject to an explicit retention label (‘destroy automatically’, ‘do nothing’, ‘subject to disposition review’). When applied, these labels are visible in the ‘Retention label’ column. The name of the label (but not the details) can be exported with other metadata from the library.

The logs include, in the ‘file and page’ list of activities, the activity ‘Changed retention label for a file‘. This displays either or both the ‘SourceLabel’ or the ‘DestinationLabel’ element with the retention label name.

The logs also include a section named ‘Retention policy and retention label activities’ with a series of activities that mostly relate to the actions of creating, configuring, updating and deleting retention labels and policies, and creating, editing, changing or removing adaptive scopes.

Converted

The Microsoft 365 audit logs do not record when a record is converted from one format to another. The only clue to this will be the title of the document and the name of the person who accessed/modified it and the the title and name in the converted document.

Deleted

The Microsoft 365 audit logs include the activities of ‘Deleted file‘, ‘Deleted file from recycle bin‘ and ‘Deleted file from second-stage recycle bin‘.

Tests on two separate tenants suggest that ‘Deleted file’ did not record anything. The reasons for this anomaly are not yet known.

Other audit events

Other ‘file and page’ activities captured by the Microsoft 365 logs that are not listed above are:

• Checked in file
• Checked out file
• Discarded file checkout
• Changed record status to locked
• Changed record status to unlocked
• Deleted file marked as a record
• Detected document sensitivity mismatch (triggered when a document uploaded to a site has a higher priority sensitivity label than the one applied to the site).
• Detected malware in file
• Recycled all minor versions of a file
• Recycled all versions of a file
• Recycled version of file
• Renamed file
• Restored file (from Recycle Bin)
• Viewed page
• View signaled by client (a user’s client, such as website or mobile app, has signaled that the indicated page has been viewed by the user. This is often followed by a ‘PagePrefetched’ event for a page).
• Performed search query (does not show the details of the query)

Accessing and retaining event history information

Exporting from SharePoint

As noted above, details of who created and modified a record in SharePoint, and any other metadata relating to the record, can be exported to Excel at any time from a view. Views may need to be created for this purpose as the default view does not show much information.

Event history and other metadata exported in this way should be stored separately from the site as the export file could potentially be subject to a ‘back end’ retention policy that is not visible to the site members (site owners can see it via Site Contents).

Exporting from the audit logs

The audit logs in Microsoft 365 are extensive and even when exported on a daily basis could be very large. Where will this data be stored? Records and information managers may only need the details of specific activities.

In practical terms, only some of the event history in some SharePoint sites is likely to be of any real value from a recordkeeping point of view. From personal experience, the most useful extracts are likely to be a monthly download of all event history for specific sites. These downloads can either be stored in a (possibly restricted access) library on the site (to keep the records with the metadata in the same location – recommended) or in a separate site.

Additional, specific requests for extracts from audit logs can be made to IT as required. For example, details of a user’s activity, or a specific activity such as ‘Deleted file’ or ‘Downloaded file’.

Of course, audit logs provide the details of what happened in the past. Consideration may also be given by IT to implementing more proactive monitoring of ‘live’ events through the Microsoft 365 ecosystem.

For more information

Further details about the Microsoft 365 audit logs may be found in the Microsoft web site ‘Search the audit logs in the compliance centre‘.

Feature image – Photo by cottonbro from Pexels

Advertisement

• Auditing
• Compliance
• Information Management
• metadata
• Microsoft 365
• Records management