(The image above was part of collector’s album issued in 1930 by Echte Wagner, a German margarine company. Source – https://flashbak.com/wonderful-futuristic-visions-of-germany-by-artists-in-1930-381451/)
In the article, Tony describes where and how the chat component of MS Teams is stored and how this might affect eDiscovery.
He also makes the important point that, while it may be possible ‘… to backup Teams by copying the compliance records in an Exchange Online backup … you’ll never be able to restore those items into Teams.’ In other words, it is better to leave the data where it was created – in MS Teams. The post explains why this is the case.
This post draws on the article to describe the factors involving in managing the chat element of Teams as records. It notes that, while is is technically possible to export chat messages (in various ways), it may be much better from a recordkeeping point of view to leave them where they are and subject them to a retention policy.
Two key reasons for leaving chat messages in place are: (a) chat messages are dynamic and may not always be a static ‘thread’, and (b) the chat messages exported from Exchange may not contain the full content of the message.
What is a Teams chat?
A Teams chat consists of one or more electronic messages with at least two participants – a sender and a receiver.
There are two types of chat message in MS Chat:
- One-to-one/one-to-many ‘chat’ (top icon above).
- Channel-based Teams chat (second icon above). Teams chat is visible to all members of the Team. Within channel-based chats, a person may create a private channel which is visible only the person who created the private channel and any participants.
Messages created in both options could be regarded as records because they may contain evidence of business activity.
However, one-to-one chats have no logical subject or grouping. Only the chat messages in Team channel chat are connected through the context of the Team/channel.
Where and how are chat messages stored?
The following is a summary from Tony Redmond’s article.
Chat messages are stored directly in the backend Azure Cosmos DB (part of the so-called Microsoft 365 ‘substrate’). The version in the database is the complete version of the chat message.
The messages are then copied, less some content elements (for example: reactions, audio records, code snippets), to a hidden folder in either (a) end-user mailboxes for one-to-one chat and private channel chats, and (b) M365 Group mailboxes for channel chat.
Most export options, including the export option in Content Search and eDiscovery, draw their content from the mailbox version of the message. This has potential implications for the completeness of the chat message as a record.
Additionally, any export can only be a ‘point in time’ record unless there is absolute certainty that all chat on a given subject have ceased.
Implications for records managers
In addition to the concerns about a chat message (or exports of them) being complete, there are (at least) two other points relating to the management of chat messages as records in MS Teams:
- Knowing if chat messages on any given subject exist.
- Applying an appropriate retention policy.
Both of these points are discussed below.
The primary way to locate content on any given subject across Microsoft 365 is via the Content Search option in the Compliance portal. Access to the Content Search option is likely to be restricted. So, if records managers do not have access, they will need to ask the Global Administrators to conduct a search.
Content searches are very powerful. This Microsoft article, ‘Keyword queries and search conditions for Microsoft 365‘ provides details on how to search. The screenshot below shows an example of a very simple keyword queries with the option to add conditions.
Searches can be configured to find content in any or all of the following locations:
- Users, Groups, Teams
- Exchange email
- Office 365 group email
- Skype for Business
- Teams messages [the copy in the mailbox]
- SharePoint sites
- OneDrive accounts
- Office 365 group sites
- Teams sites
- Exchange public folders
Note that content search only works on the copies of the items in the Exchange mailboxes, not the backend Teams database. Accordingly, there is some potential for it to not find some content.
Both the mailbox content and the content discovered by the search can be exported. Teams chat messages can be exported as individual items or as a PST – but note that these message may exclude the elements as described in Tony’s article.
The problem with exporting the content either this way or via other export options (such as described in this post ‘How to export MS Teams chat to html (for backup)‘ (using the Microsoft Graph API) is that it creates a single ‘point in time’ copy; additional content could be added at any time and, if the chats were subject to a retention policy, they may already be deleted.
Managing chat messages ‘in place’ as records
As any export only creates a ‘point in time’ version, it makes more sense from a recordkeeping point of view to leave the chat messages where they are and apply one or more retention policies to ensure the records are preserved.
Ideally, organisations that may create or capture records on a given subject will have taken the time to establish a way for users to do this, including through the creation of a dedicated Microsoft 365 Group with an associated SharePoint site and Team in MS Teams.
For example, if there is a requirement to store all records relating to COVID-19, it would make sense (at the very least) to create a Microsoft 365 Group with that name; this will create: (a) a linked mailbox accessible by all members of the Group, (b) a SharePoint site with the same name, and (c) a Team in MS Teams. All of the content – emails, documents, chat, is linked via the same (subject) Group.
This model makes it easier to aggregate ‘like’ information and apply a single retention policy. It assumes there is (or will be) some degree of control over the creation of Teams (or very good communication to users) to prevent the creation of random Teams, Groups and SharePoint sites – AND to ensure that end-users chat about a given subject within a Team channel, not in one-to-one chat.
What retention period should be applied to chat messages?
The retention period applied to either one-to-one or Team channel messages will depend largely on the organisation’s business or regulatory requirements to keep records. There are two potential models.
The simplest model is to have a single retention policy for one-to-one chats, and a separate retention policy for all Teams channel chats.
As one-to-one chats are stored in the mailboxes of chat participants, it makes sense to retain the chat content for as long as the mailboxes. However, some organisations may seek to minimise the use of chat and have a much reduced retention period – even as little as a few days.
The creation and application of retention policies to Teams channel chat may require additional considerations. For example:
- As every Team is based on a Microsoft Group that has its own SharePoint site, it is probably a good idea to establish Teams based on subjects that logically map to a retention class. For example, if ‘customer correspondence’ needs to be kept for a minimum 5 years, and there is a Group/SharePoint site/Team for that subject, then all the content should have the same retention policy – although the Group mailbox and SharePoint site may have a policy applied to the Group, with a separate (but same retention period) applied to the Team.
- There may be a number of Teams that contain trivial content that does not need to be retained as records. These Teams could be subject to a specific implicit policy that deletes content after a given period – say 3 years.
In all cases, there is a requirement to plan for retention for records across all the Microsoft 365 workloads.
What happens to chat messages at the end of a retention period?
At the end of a Microsoft 365 retention policy period, both the mailbox version and the database version of the Teams chat message are deleted. To paraphrase Tony’s article, the Exchange Managed Folder Assistant removes expired records from mailboxes. Those deletions are synchronized back to Teams, which then removes the real messages from the backend database.
No record is kept of this deletion action except in the audit logs. Accordingly, if there is a requirement to keep a record of what was destroyed, this will need to be factored in to whatever retention policy is created.
The following are observations from using the ‘Move To’ function in a SharePoint Online library to move close to 18,500 files and folders over a weekend, from a ‘classic’ SharePoint subsite in a SharePoint Online site to a new and ‘free standing’ site in the same tenant. The moving was completed over a weekend to … Continue reading Observations on ‘Move To’ in SharePoint Online
Every SharePoint document library, including the Teams ‘Files’ tab, includes the ability to move or copy files somewhere else in the tenant. For some unknown reason, the arrow in the SharePoint library points to the right but in Teams it points to the left, and in Teams the ‘to’ is missing. What actually happens when … Continue reading What happens when files are moved in SharePoint?
Recordkeeping requires the capture and retention of event or process metadata about records; that is, data about who did what and when. This information is usually captured in audit logs or trails. In legacy EDRM systems, event metadata was captured with all other metadata about the record in the system’s database; the actual digital object … Continue reading Understanding why SharePoint viewers vanish when documents are moved