Posted in Compliance, Exchange Online, In Place Records, Information Management, Records management, SharePoint Online

In place vs in place management of records in Microsoft 365

The ability to manage records ‘in place’ in SharePoint has existed since around 2013. But this is not the same thing as leaving records where they were created or captured and managing them there – ‘in place’.

This post explains the difference between the two ‘in place’ options. In brief:

  • The Microsoft ‘in place’ model is based on making the distinction between non-records content and content declared as records (as per DOD 5015.2), that may be stored in the same SharePoint site, or using Exchange in-place options.
  • The other ‘in place’ model is simply based on leaving records and other content where they were created or captured, and managing it there – including (where necessary) by applying the ‘in place’ options in the previous point.

The Microsoft in-place model

SharePoint

The Microsoft in-place model for managing records in SharePoint is based on the requirement to comply with the US Department of Defense (DOD) standard titled ‘Design Criteria Standard for Electronic Records Management Software Applications’, usually known by its authority number – DOD Directive 5015.2, Department of Defense Records Management Program, originally published in 11 April 1997.

Section C2.2.3 ‘Declaring and Filing Records’ of the standard defines 26 specific requirements for declaring and filing records, including the following points:

  • The capability to associate the attributes of one or more record folder(s) to a record, or for categories to be managed at the record level, and to provide the capability to associate a record category to a record
  • Mandatory record metadata.
  • Restrictions on who can create, edit, and delete record metadata components, and their associated selection lists.
  • Unique computer-generated record identifiers for each record, regardless of where that record is stored.
  • The capability to create, view, save, and print the complete record metadata, or user-specified portions thereof, in user-selectable order.
  • The ability to prevent subsequent changes to electronic records stored in its supported repositories and preserving the content of the record, once filed
  • Not permitting modification of certain metadata fields.
  • The capability to support multiple renditions of a record.
  • The capability to increment versions of records when filing.
    Linking the record metadata to the record so that it can be accessed for display, export.
  • Enforcement of data integrity, referential integrity, and relational integrity.

Microsoft’s initial guidance on configuring in place records management describes how to activate and apply this functionality primarily in SharePoint on-premise. It is still possible to apply this in SharePoint Online (but see below). The SharePoint in place model refers to a mixed content approach where both records and non-records can be managed in the same location (an EDMS with RM capability):

Managing records ‘in place’ also enables these records to be part of a collaborative workspace, living alongside other documents you are working on.

The same link above, however, also refers to newer capability that was introduced with the Microsoft 365 Records Management solution in the Compliance admin portal. This new capability allows organisations to use retention labels instead to declare content as records when the label is applied, which ‘effectively replaces the need to use the Records Center or in-place records management features.’

The guidance also noted that, ‘… moving forward, for the purpose of records management, we recommend using the Compliance Center solution instead of the Records Center.’

Exchange

A form of in-place management has also been available for Exchange on-premise mailboxes, with in place archiving based on using archive mailboxes – see the Microsoft guidance ‘In-Place Archiving for in Exchange Server‘.

One draw-back of this model is that the (email) records in these mailboxes were not covered by the same DOD 5015.2 rigor as those in SharePoint, but they could at least be isolated and protected against modification or deletion, for retention, control and compliance purposes.

Archive mailboxes, including ‘auto-expanding archives’, also exist in Exchange Online. In the Exchange Online archiving service description, it is noted that:

Microsoft Exchange Online Archiving is a Microsoft 365 cloud-based, enterprise-class archiving solution for organizations that have deployed Microsoft Exchange Server 2019, Microsoft Exchange Server 2016, Microsoft Exchange Server 2013, Microsoft Exchange Server 2010 (SP2 and later), or subscribe to certain Exchange Online or Microsoft365 plans. Exchange Online Archiving assists these organizations with their archiving, compliance, regulatory, and eDiscovery challenges while simplifying on-premises infrastructure, and thereby reducing costs and easing IT burdens.

The new ‘in place’ model

A newer form of in-place records management has appeared with Microsoft 365.

Essentially, the new model simply means leaving records where they were created or captured – in Exchange mailboxes, SharePoint sites, OneDrive or Teams (and so on). and applying additional controls where it is required.

The newer model of in place records management is based on the assumptions that:

  • It will never be possible to accurately or consistently identify and/or declare or manage every record that might exist across the Microsoft 365 ecosystem. For example, it is not possible to declare Teams chats or Yammer messages.
  • Only some and mostly high value or permanent records, will be subject to specific additional controls, including records declaration and label-based retention.
  • The authenticity, integrity and reliability of a some records may be based more on system information (event metadata) about its history, than by locking a point-in-time version.

Microsoft appear to support this dual in place model with their information governance (broader controls) and records management (specific controls, including declaration) approach to the management of content and records across Microsoft 365, as described in the Microsoft guidance ‘Information Governance in Microsoft 365‘, which includes the graphic below, modified to show the relationship between the two in place concepts.

The relationship between Microsoft’s ‘in place’ focussed records management, and managing everything (including records) in place.

In place co-existence

Can both in place models exist? Yes. There is nothing to prevent both in place models existing in the same environment, in which some records may need to be better managed or controlled than others, but it is important to understand the distinction between the two when it comes to applying controls.

Image: Quarantine Building, Portsea, Victoria Australia. Andrew Warland 2021

Posted in Access controls, Conservation and preservation, Digital preservation, Electronic records, Exchange 2010, Exchange 2013, Exchange Online, Information Management, Records management, Retention and disposal, XML

The enduring problem of emails as records

Ever since emails first appeared as a way to communicate more than 30 years ago they have been a problem for records management, for two main reasons.

  • Emails (and attachments) are created and captured in a separate (email) system, and are stored in mailboxes that are inaccessible to records managers (a bit like ‘personal’ drives).
  • The only way to manage them in the context of other records was/is to print and file or copy them to a separate recordkeeping system, leaving the originals in place.

Thirty-plus years of email has left a trail of mostly inaccessible digital debris. An unknown volume of records remains locked away in ‘personal’ and archived mailboxes. Often, the only way to find these records is via legal eDiscovery, but even that can be limited in terms of how back you can go.

Options for the preservation of legacy emails

The Council on Information and Library Resources (CLIR) published a detailed report in August 2018 titled ‘The Future of Email Archives: A Report from the Task Force on Technical Approaches to Email Archives‘.

The report noted (from page 58) three common approaches to the preservation of legacy emails:

  • Bit-Level Preservation
  • Migration (to MBOX, EML or even XML)
  • Emulation

In a follow up article, the Australian IDM magazine published an article in March 2020 by one of the CLIR report authors (Chris Prom). The article, titled ‘The Future of Past Email is PDF‘, suggested that PDF may be (or become) a more suitable long-term solution for preservation of legacy emails.

Preservation is one thing, what about access

There is little point in preserving important records if they cannot be accessed. The two must go together. In fact, preservation without the ability access a record is not a long different from destruction through negligence.

Assuming emails can be migrated to a long-term and accessible format, what then?

No-one (except possible well-funded archival institutions perhaps) is seriously likely to attempt to move or copy individual legacy emails to pre-defined and pre-existing containers or aggregations of other records. This would be like printing individual emails and storing them in the same paper file or box that other records on the same subject are stored.

Access to legacy emails in an digitally accessible, metadata-rich format like PDF provides a range of potential opportunities to ‘harvest’ and make use of the content, including through machine learning and artificial intelligence.

These options have been available for close to twenty years in the eDiscovery world, but to support specific legal requirements.

Search, discovery and retention/disposal tools available in the Microsoft 365 Compliance portal, along with the underlying Graph and AI tools (including SharePoint Syntex) provide the potential to manage legacy content, including emails.

The starting point is migrating all those old legacy emails to an accessible format.

Posted in Compliance, Electronic records, Exchange Online, Information Management, Microsoft Teams, Records management, Retention and disposal, Security

Using MS Teams without an Exchange Online mailbox

When people chat in Microsoft Teams (MS Teams), a ‘compliance’ copy of the chat is saved to either personal or (Microsoft 365) Group mailboxes. This copy is subject to retention policies, and can be found and exported via Content Search.

But what happens if there is no Exchange Online mailbox? It seems the chats become inaccessible which could be an issue from a recordkeeping and compliance point of view.

This post explains what happens, and why it may not be a good idea (from a compliance and recordkeeping point of view) not to disable the Exchange Online mailbox option as part of licence provisioning.

Licences and Exchange Online mailboxes

When an end-user is allocated a licence for Microsoft 365, a decision (sometimes incorporated into a script) is made about which of the purchased licences – and apps in those licences – will be assigned to that person.

E1, E3 and E5 licences include ‘Exchange Online’ as an option under ‘Apps’. This option is checked by default (along with many of the other options), but it can be disabled (as shown below).

If the checkbox option is disabled as part of the licence assigning process (not after), the end-user won’t have an Exchange mailbox and so won’t see the Outlook option when they log on to office.com portal. (Note – If they have an on-premise mailbox, that will continue to exist, nothing changes).

Having an Exchange Online mailbox is important if end-users are using MS Teams, because the ‘compliance’ copy of 1:1 chat messages in MS Teams are stored in a hidden folder (/Conversation History/Team Chat) in the Exchange Online mailbox of every participant in the chat. If the mailbox doesn’t exist, those copies aren’t made and so aren’t accessible and may be deleted.

If end-users chat with other end-users who don’t have an Exchange mailbox as shown in the example below, the same thing happen – no compliance copy is kept. The chat remains inaccessible (unless the Global Admins take over the account).

The exchange above, between Roger Bond and Charles, includes some specific key words. As we will see below, these chats cannot be found via a Content Search.

(On a related note, if the ability to create private channels is enabled and they create a private channel and chat there, the chats are also not saved because a compliance copy of private channel chats are stored in the mailboxes of the individual participants.)

Searching for chats when no mailbox exists

As we can see above, the word ‘mosquito’ was contained in the chat messages between Roger and Charles.

Content Searches are carried out via the Compliance portal and are more or less the same as eDiscovery searches in that they are created as cases.

From the Content Search option, a new search is created by clicking on ‘+New Search’, as shown below. The word ‘mosquito’ has been added as a keyword.

We then need to determine where the search will look. In this case the search will look through all the options shown below, including all mailboxes and Teams messages.

When the search was run, the results area shows the words ‘No results found’.

Clicking on ‘Status details’ in the search results, the following information is displayed – ‘0 items’ found. The ‘5 unindexed items’ is unrelated to this search and simply indicates that there are 5 unindexed items.

Double-checking the results

To confirm the results were accurate, another search was conducted where the end-user originally did not have a mailbox, and then was assigned one.

If the end-user didn’t have a mailbox but the other recipient/s of the message did, the Content Search found one copy of the chat message in the mailbox of the other participants. Only one item was found.

When the Exchange Online option was enabled for the end-user who previously did not have a mailbox (so they were now assigned a mailbox), a copy of the chat was found in the mailbox of both participants, as shown in the details below (‘2 items’).

Summary and implications

In summary:

  • If end users chat in the 1:1 area of MS Teams and don’t have an Exchange Online mailbox, no compliance copy of the chat will be saved, and so it will not be found via Content Search.
  • If any of the participants in the 1:1 chat have an Exchange Online mailbox, the chat will appear in the mailboxes of those participants.
  • If all participants in the 1:1 chat have an Exchange Online mailbox, the chat will be found in the mailbox of all participants.

Further to the above:

  • If end users can delete chats (via Teams policies) and don’t have a mailbox, no copy of the chat will exist.
  • If end-users with a mailbox can delete Teams chats, but a retention policy has been applied to the chats, the chats will be retained as per the retention policy (in a hidden folder).

And finally, if you allow private channels, end-users can create private channels in the Organisation Team. The chats in these private channels are usually stored in the personal mailboxes of participants (not the Group mailbox) – so these chats will also be inaccessible and cannot be found via Content Search.

The implications for the above are that, if you need to ensure that personal chat messages can be accessed (from Content Search), then the participants in the chat must have an Exchange Online mailbox.

Further, if you allow deletion of chats but need to be able to recover them for compliance purposes, a retention policy should be applied to Teams 1:1 chat.

Posted in Classification, Compliance, Exchange Online, Information Management, Microsoft Teams, Office 365, Office 365 Groups, Products and applications, Records management, Retention and disposal, SharePoint Online, Training and education

Planning for records retention in Office 365

Office 365 is sometimes referred to as an ‘ecosystem’. In theory this means that records could be stored anywhere across that ecosystem.

Unlike the ‘old’ on-premise world of standalone servers for each Microsoft application (Exchange, SharePoint, Skype) – and where specific retention policies could apply (including the Exchange Messaging Records Management MRM policy), the various elements that make up Office 365 are interconnected.

The most obvious example of this interconnectivity is Microsoft Teams which stores chat content in Exchange and provides access to content stored in both SharePoint (primarily the SharePoint site of the linked Office 365 Group) and OneDrive, and has links to other elements such as Planner.

Records continue to be created and kept in the various applications but retention policies are set centrally and can apply to any or all of the content across the ecosystem.

Managing records in Office 365, and applying retention rules to those records, requires an understanding of at least the key parts of the ecosystem – Exchange, Teams, SharePoint and OneDrive and how they interrelate, and from there establishing a plan for the implementation of retention.

What types of records are created in Office 365?

Records are defined as ‘evidence of business activity’ and are often associated with some form of metadata.

Evidence of business activity is an overarching term that can include:

  • Emails
  • Calendars
  • Documents and notebooks (in the sense of text on a page)
  • Plans, including both project plans and architectural plans and diagrams
  • Images/photographs and video
  • Chat and/or messages
  • Conversations (audio and/or video based)
  • Social media posts

All digital records contain some form of metadata, usually displayed as ‘Properties’.

Where are the records stored in Office 365?

Most records created organisations using Office 365 are likely to be created or stored in the following parts of the ecosystem:

  • Exchange/Outlook – for emails and calendars.
  • SharePoint and OneDrive – for documents and notebooks (in the sense of text on a page), plans, images/photographs and video.
  • Stream – for audio and video recordings.
  • MS Teams – for chat and/or messages, conversations (audio and/or video based). Note that 1:1 chats are stored in a hidden folder of the Exchange mailbox of the end-user/s participating in the chat, while Teams channel chat is stored in a hidden folder of the linked Office 365 Group mailbox.
  • Yammer – for (internal) social media posts.

It is also possible to import and archive certain external content such as Twitter tweets and Facebook content in Office 365.

The diagram below provides a overview of the main Office 365 applications and locations where records are created or stored. Under SharePoint, the term ‘Sites’ refers to all types of SharePoint sites, including those associated with Office 365 Groups. Libraries are shown separately because of the potential to apply a retention policy to a library – see below.

O365WheretheRecordsare

Note also that this diagram does not include network file shares (NFS) as the assumption is made that (a) NFS content will be migrated to SharePoint and the NFS made read only, and (b) all new content that would previously have been stored on the NFS is instead saved either to OneDrive for Business (for ‘personal’ or working documents) or SharePoint only.

Creating a plan to manage records retention across Office 365

In previous posts I have recommended that organisations implementing Office 365 have the following:

  • A basic architecture design model for SharePoint sites, including SharePoint sites linked with Office 365 Groups (and Teams in MS Teams).
  • A plan for creating and applying retention policies across the ecosystem.

Because SharePoint is the most likely location for records to be stored (aside from Exchange mailboxes and OneDrive accounts), there should be at least one retention policy for every SharePoint site (or group of sites), as well as policies for specific document libraries if the retention for the content in those libraries may be different from the retention on the overall site.

For example, a ‘Management’ site may contain a range of general content as well as specific content that needs to be retained for longer. 

  • The site can be covered by a single implicit retention policy of (say) 7 years. This policy will delete content in the background, based on date created or data modified. 
  • The document library where specific types of records with longer or different retention requirements are stored may have one or more explicit label-based policies applied to those libraries. This content will be retained while the rest of the site content is deleted via the first policy.

Structure of a retention plan for records in Office 365

A basic plan for creating and applying retention policies might look something like the following:

  • User mailboxes – one ‘general’ (implicit) retention policy for all mailboxes (say, 7 years after creation) and another more specific retention policy for specific mailboxes that require longer retention.
  • SharePoint sites – multiple (implicit) retention policies targeting one or more sites.
  • SharePoint libraries – multiple (explicit) label-based retention policies that are applied manually. These policies will usually a retention policy that is longer than any implicit retention policy as any implicit site policy will prevent the deletion of content before it reaches the end of that retention period.
  • Office 365 Groups (includes the associated mailbox and SharePoint site) – one ‘general’ (implicit) retention policy. See also below.
  • Teams channel chat – one ‘general’ (implicit) retention policy. Note that this content is stored in a special folder of the Office 365 Group mailbox.
  • 1:1 chat – one ‘general’ (implicit) retention policy. This content is stored in a special folder of the participant mailboxes.
  • OneDrive documents – one ‘general’ (implicit) retention policy for all ODfB accounts, plus the configuration of retention after the account is inactive.

At a high level, the retention policy plan might look something like the following – ‘implicit’ policies are shown in yellow, SharePoint document libraries may be subject to ‘explicit’, label-based policies. The ‘+7 years’ for OneDrive relates to inactive accounts, a setting set in the OneDrive Admin portal.

O365WheretheRecordsare2

Regarding Microsoft Office 365 Groups, Microsoft notes the following on this page about managing retention in Office 365:

To retain content for a Microsoft 365 group, you need to use the Microsoft 365 groups location. Even though an Microsoft 365 group has an Exchange mailbox, a retention policy that includes the entire Exchange location won’t include content in Microsoft 365 group mailboxes. A retention policy applied to an Microsoft 365 group includes both the group mailbox and site. A retention policy applied to an Microsoft 365 group protects the resources created by an Microsoft 365 group, which would include Microsoft Teams.

The actual plan should contain more detail and included as part of other recordkeeping documentation (perhaps stored on a ‘Records Management’ SharePoint site). The plan should include details about (a) where the policies have been applied and (b) the expected outcomes or actions for the policies, including automatic deletion or disposition review (for document libraries).

Keep in mind that, unless the organisation decides to acquire this option, there is no default backup for content in Office 365 – once a record had been deleted, it is gone forever and there may be no record of this beyond 90 days.

Posted in Electronic records, Exchange Online, Governance, Information Management, Microsoft Teams, Office 365, Office 365 Groups, Products and applications, Records management, Retention and disposal, SharePoint Online

Setting up SharePoint Online to manage records (as part of Office 365) – Part 1/3

This is the first of three posts that describe the main elements involved in setting up SharePoint Online to manage records.

This post focuses on the recordkeeping related elements in the Office 365 and Compliance admin portals:

  • Office 365 Admin – Licences, Roles and AD Groups (including Office 365 Groups)
  • Compliance Admin – Retention labels and policies (and some more options)

The second post focuses on SharePoint Online Admin centre configuration.

The third and last post focuses on SharePoint site collection provisioning and configuration to manage records

Office 365 admin center

O365AdminPortalUsersRolesGroups

The main elements that impact on the management of records in Office 365 are Users (for licences), Roles and Groups, as can be seen in the screenshot.

Users – licencing and applications

Organisations that acquire Office 365 will generally have the relevant licences required (a) to set up and administer SharePoint Online, and (b) for users to use it (and OneDrive for Business).

This post assumes that organisations will have at least an E3 licence which includes SharePoint for end users, visible as an app when they log on to https://office.com, along with all other applications included in the licence, for example Exchange/Outlook, OneDrive for Business, MS Teams and so on. End users with access to these items will also be able to download and use the equivalent mobile device apps.

Roles

The three key roles that impact on the management of records in SharePoint are as follows:

Global Admin (GA)

Global Admins:

  • Are responsible for managing the entire Office 365 environment. This includes creating new Groups (Security Groups, Distribution Lists and Office 365 Groups).
  • Are responsible for assigning key roles, including the SharePoint Administrator and Compliance Administrator (and other roles).
  • May have responsibility for, and/or the skills and knowledge required to set up and administer SharePoint Online and create new sites for the organisation.
  • May also be able to create and publish retention policies in the Compliance admin portal.

Note – Organisations that outsource the administration of Office 365 should always have at least one GA account to access the tenant if ever required. If they don’t have a log on, they should have or acquire a very good understanding of the access and privileges afforded to the outsourced company. 

SharePoint Administrator (SP Admin)

The SP Admin role will usually be a ‘system’ role that is responsible for managing the SharePoint environment, including OneDrive for Business. As noted above, a GA with the right skills can also manage the SharePoint environment. 

Generally speaking, SharePoint Administrators will focus on the technical and configuration aspects of SharePoint. They are not usually responsible for confirugint SharePoint to manage records, managing records, or creating and publishing retention policies. This role usually falls to either the GA or Compliance Administrator.

Compliance Administrator

The Compliance Admin role is responsible, among other things, for the creation and publishing of retention labels and policies in the Compliance Admin portal. A GA can perform this role (along with all other roles) if required.

Compliance Admins will usually be responsible for disposition reviews linked with retention labels, and be involved in eDiscovery cases.

The Compliance Admin can search and view the audit logs for all activity across Office 365 and can carry out broad content searches with the ability to export the content of those searches. As this role is relatively powerful, it should be limited to key senior individuals in the organisation.

Office 365 and Security Groups

Office 365 Groups are Azure/Exchange objects just like Security Groups and Distribution Lists. Accordingly, there should be controls around their creation, including naming conventions.

As every Office 365 Group has an associated SharePoint site, organisations should consider restricting the ability for end users to create Office 365 Groups, and only allowing Global Admins and members of a Security Group to do this. Neither SharePoint Admins or Compliance Admins would normally create AD Groups.

If the ability to create Office 365 Groups is not restricted, an Office 365 Group will be created with an associated SharePoint site whenever:

  • A new Team is created in MS Teams.
  • A new Group is created from Outlook.
  • A new Yammer Group/Community is created.

External sharing

The ability to share content externally from SharePoint and OneDrive for Business is controlled from the Office 365 Admin portal. This is a global setting that can be disabled by the Global Admins if required.

It is assumed, for the purpose of this post, that that setting is enabled to allow external sharing.

Note that enabling external sharing at the global level does not enable it globally for all SharePoint sites; sites must be individually modified to allow it.

Compliance Admin

The Compliance admin portal can be accessed by the GAs and also the Compliance Admins (and some other roles). It is where retention labels and policies are created (in line with the corporate file plan/BCS) and published, and disposition reviews are undertaken, so records managers need access.

Other options in this section that relate to the management of records include the audit logs, content search and eDiscovery.

Retention policies

Retention policies may be applied to all the key workloads in Office 365 where records are stored:

  • Exchange Online
  • SharePoint Online
  • OneDrive for Business
  • MS Teams
  • Office 365 Groups

Retention labels published as retention policies are visible to and can be applied by end-users. Generally these are more likely to be applied at the document library level rather than to individual records, or in mailboxes or OneDrive for Business.

Retention policies that are not based on labels may be applied to all, or parts of, the four workloads listed above. For example, they may be applied to all, or a sub-set of Exchange mailboxes or OneDrive for Business accounts, or SharePoint sites. Retention policies may also be applied to individual or team chats in MS Teams.

Organisations seeking to use retention policies in Office 365 should understand how these work, have a plan for their implementation, and keep track of what has been applied where.

  • Retention policies for all mailboxes or all ODfB accounts may replace previous on-premise backup options for those workloads. It is unlikely that end-users will (or will want to) apply retention labels published as policies to individual emails or folders in mailboxes or OneDrive.
  • SharePoint sites are likely to have either or a combination of explicit and implicit/invisible retention policies. Implicit, single period retention policies may be more suitable for entire smaller, short-lived SharePoint sites. Explicit retention policies may be more suitable for the diverse range of content on more complex and long-lasting sites. Some sites may be created and populated around the need to keep a particular type of record for a long period of time – for example, employee records.

Audit logs

The Office 365 audit logs are found in the Compliance admin portal. For an E3 licence, the content in the logs is stored for 90 days.

As audit logs are an important element in keeping records, organisations may need to consider ways to retain this content for a longer period.

Note – SharePoint document libraries record the name of anyone who edited a document (and also previous versions), but they don’t record the name of anyone who simply viewed it. SharePoint lists also include audit trails, making it possible to track changes in individual rows of a list.

Content searches and eDiscovery

The Compliance admin portal provides two similar options to search for content across Office 365. Both the Content Search and eDiscovery options provide the ability to establish a ‘case’ that can be run more than once.

The eDiscovery option provides the added ability to put content on Legal Hold. Advanced eDiscovery is available with a higher licence.

Next

Click on the links below to read the next two posts:

  • SharePoint Online Admin centre configuration.
  • SharePoint site collection provisioning and configuration to manage records.
Posted in Compliance, Electronic records, Exchange Online, Information Management, Microsoft Teams, Office 365 Groups, Products and applications, Records management, Retention and disposal, SharePoint Online

Understanding and applying retention policies to content in MS Teams

This post highlights the need to understand how retention works in MS Teams, why it may be related to how long you keep emails (including for backup purposes), and why you need to consider all the elements that make up an Office 365 Group when considering how – and how long – to retain content in MS Teams.

Overview of retention in MS Teams

If you are unfamiliar with how retention works with MS Teams, these two related sites provide very useful detail.

overview_of_security_and_compliance_in_microsoft_teams_image1
Image from the first link above – Security Compliance Overview

The quote below from the second link is relevant to this post:

‘Teams chats are stored in a hidden SubstrateHolds folder in the mailbox of each user in the chat, and Teams channel messages are stored in a hidden SubstratesHolds folder in the group mailbox for a team. Teams uses an Azure-powered chat service that also stores this data, and by default this service stores the data forever. With a Teams retention policy, when you delete data, the data is permanently deleted from both the Exchange mailboxes and the underlying chat service.’

and

‘Teams chats and channel messages aren’t affected by retention policies applied to user or group mailboxes in the Exchange email or Office 365 groups locations. Even though Teams chats and channel messages are stored in Exchange, they’re only affected by retention policies applied to the Teams locations.’

In summary:

  • One-to-one chat in MS Teams is stored in a hidden folder of the mailbox of each user in the chat. Documents shared in those chats are stored in the OneDrive for Business of the person who shared it.
  • Group chat in Team channels is stored in a hidden folder of the mailbox of the associated Office 365 Group – and also in an Azure chat service. Documents are stored in the Office 365 Group’s SharePoint site (other SharePoint site libraries may also be linked in a channel).

Another quote from the same post:

‘In many cases, organizations consider private chat data as more of a liability than channel messages, which are typically more project-related conversations.’

Teams content is kept in mailboxes, retention may be similar

Typically, in the on-premise past, organisations will have backed up their Exchange mailboxes (and possibly also enabled journaling, to capture emails), for disaster recovery, ‘archiving’ and investigations. Unless a decision is made to invest in cloud back-ups, Office 365 retention policies may also be applied to Exchange mailboxes, effectively replacing the need to back them up. Retention policies applied to Exchange mailboxes don’t affect the teams chat folder.

Organisations should probably apply the same retention period to both emails and Teams chats as they do to email mailbox backups now. That is, if mailboxes are typically kept for 7 – 10 years after the person leaves the organisation, then keep the Teams chats for the same period.

Note that, even if a poster deletes an item (if that option is enabled), it will still be retained if there is a retention policy.

Suggestions for retention in MS Teams

As there can be different retention requirements, depending on the subject matter, here are some suggestions for retention:

  • One-to-one chat is like email, you will never know everything that is being said or sent. So a single retention policy that mirrors email would be appropriate.
  • Teams chat is more likely to be about the subject of the Team, which is based on an Office 365 Group, its own mailbox, and has a SharePoint site. In this case, you could consider a retention policy applied to all Office 365 Groups or specific Groups – for example ‘Project Groups’, then ensure that the retention policy or policies cover all aspects of the Office 365 Group (mailbox, team chat, SharePoint).
  • If all the records relating to a particular subject matter (including email, chat and documents) must be retained for 25 years, then you need to understand all the options.

It underscores the need to plan carefully for retention management for all the key workloads in Office 365.