Posted in Classification, Compliance, Electronic records, Governance, Information Management, Microsoft Teams, Office 365 Groups, Products and applications, Records management, Retention and disposal, SharePoint Online

Managing MS Teams chat as records

(The image above was part of collector’s album issued in 1930 by Echte Wagner, a German margarine company. Source – https://flashbak.com/wonderful-futuristic-visions-of-germany-by-artists-in-1930-381451/)

On 19 May 2020, Tony Redmond published a very helpful article on the Office 365 for IT Pros website titled ‘Using Teams Compliance Data for eDiscovery‘.

In the article, Tony describes where and how the chat component of MS Teams is stored and how this might affect eDiscovery.

He also makes the important point that, while it may be possible ‘… to backup Teams by copying the compliance records in an Exchange Online backup … you’ll never be able to restore those items into Teams.’ In other words, it is better to leave the data where it was created – in MS Teams. The post explains why this is the case. 

This post draws on the article to describe the factors involving in managing the chat element of Teams as records. It notes that, while is is technically possible to export chat messages (in various ways), it may be much better from a recordkeeping point of view to leave them where they are and subject them to a retention policy.

Two key reasons for leaving chat messages in place are: (a) chat messages are dynamic and may not always be a static ‘thread’, and (b) the chat messages exported from Exchange may not contain the full content of the message. 

What is a Teams chat?

A Teams chat consists of one or more electronic messages with at least two participants – a sender and a receiver. 

msteamschatteams-1

There are two types of chat message in MS Chat:

  • One-to-one/one-to-many ‘chat’ (top icon above).
  • Channel-based Teams chat (second icon above). Teams chat is visible to all members of the Team. Within channel-based chats, a person may create a private channel which is visible only the person who created the private channel and any participants.

Messages created in both options could be regarded as records because they may contain evidence of business activity.

However, one-to-one chats have no logical subject or grouping. Only the chat messages in Team channel chat are connected through the context of the Team/channel. 

Where and how are chat messages stored?

The following is a summary from Tony Redmond’s article.

Chat messages are stored directly in the backend Azure Cosmos DB (part of the so-called Microsoft 365 ‘substrate’). The version in the database is the complete version of the chat message.

The messages are then copied, less some content elements (for example: reactions, audio records, code snippets), to a hidden folder in either (a) end-user mailboxes for one-to-one chat and private channel chats, and (b) M365 Group mailboxes for channel chat.

Most export options, including the export option in Content Search and eDiscovery, draw their content from the mailbox version of the message. This has potential implications for the completeness of the chat message as a record.

Additionally, any export can only be a ‘point in time’ record unless there is absolute certainty that all chat on a given subject have ceased. 

Implications for records managers

In addition to the concerns about a chat message (or exports of them) being complete, there are (at least) two other points relating to the management of chat messages as records in MS Teams:

  • Knowing if chat messages on any given subject exist. 
  • Applying an appropriate retention policy. 

Both of these points are discussed below. 

Finding content

The primary way to locate content on any given subject across Microsoft 365 is via the Content Search option in the Compliance portal. Access to the Content Search option is likely to be restricted. So, if records managers do not have access, they will need to ask the Global Administrators to conduct a search. 

Content searches are very powerful. This Microsoft article, ‘Keyword queries and search conditions for Microsoft 365‘ provides details on how to search. The screenshot below shows an example of a very simple keyword queries with the option to add conditions. 

ContentSearchQuery

Searches can be configured to find content in any or all of the following locations:

  • Users, Groups, Teams
    • Exchange email
    • Office 365 group email
    • Skype for Business
    • Teams messages [the copy in the mailbox]
    • To-Do
    • Sway
    • Forms
  • SharePoint
    • SharePoint sites
    • OneDrive accounts
    • Office 365 group sites
    • Teams sites
  • Exchange public folders

Note that content search only works on the copies of the items in the Exchange mailboxes, not the backend Teams database. Accordingly, there is some potential for it to not find some content.

Both the mailbox content and the content discovered by the search can be exported.  Teams chat messages can be exported as individual items or as a PST – but note that these message may exclude the elements as described in Tony’s article.

The problem with exporting the content either this way or via other export options (such as described in this post ‘How to export MS Teams chat to html (for backup)‘ (using the Microsoft Graph API) is that it creates a single ‘point in time’ copy; additional content could be added at any time and, if the chats were subject to a retention policy, they may already be deleted.

Managing chat messages ‘in place’ as records

As any export only creates a ‘point in time’ version, it makes more sense from a recordkeeping point of view to leave the chat messages where they are and apply one or more retention policies to ensure the records are preserved. 

Ideally, organisations that may create or capture records on a given subject will have taken the time to establish a way for users to do this, including through the creation of a dedicated Microsoft 365 Group with an associated SharePoint site and Team in MS Teams. 

For example, if there is a requirement to store all records relating to COVID-19, it would make sense (at the very least) to create a Microsoft 365 Group with that name; this will create: (a) a linked mailbox accessible by all members of the Group, (b) a SharePoint site with the same name, and (c) a Team in MS Teams. All of the content – emails, documents, chat, is linked via the same (subject) Group. 

This model makes it easier to aggregate ‘like’ information and apply a single retention policy. It assumes there is (or will be) some degree of control over the creation of Teams (or very good communication to users) to prevent the creation of random Teams, Groups and SharePoint sites – AND to ensure that end-users chat about a given subject within a Team channel, not in one-to-one chat. 

What retention period should be applied to chat messages?

The retention period applied to either one-to-one or Team channel messages will depend largely on the organisation’s business or regulatory requirements to keep records. There are two potential models. 

The simplest model is to have a single retention policy for one-to-one chats, and a separate retention policy for all Teams channel chats.

As one-to-one chats are stored in the mailboxes of chat participants, it makes sense to retain the chat content for as long as the mailboxes. However, some organisations may seek to minimise the use of chat and have a much reduced retention period – even as little as a few days. 

The creation and application of retention policies to Teams channel chat may require additional considerations. For example:

  • As every Team is based on a Microsoft Group that has its own SharePoint site, it is probably a good idea to establish Teams based on subjects that logically map to a retention class. For example, if ‘customer correspondence’ needs to be kept for a minimum 5 years, and there is a Group/SharePoint site/Team for that subject, then all the content should have the same retention policy – although the Group mailbox and SharePoint site may have a policy applied to the Group, with a separate (but same retention period) applied to the Team. 
  • There may be a number of Teams that contain trivial content that does not need to be retained as records. These Teams could be subject to a specific implicit policy that deletes content after a given period – say 3 years. 

In all cases, there is a requirement to plan for retention for records across all the Microsoft 365 workloads. 

What happens to chat messages at the end of a retention period?

At the end of a Microsoft 365 retention policy period, both the mailbox version and the database version of the Teams chat message are deleted. To paraphrase Tony’s article, the Exchange Managed Folder Assistant removes expired records from mailboxes. Those deletions are synchronized back to Teams, which then removes the real messages from the backend database.

No record is kept of this deletion action except in the audit logs. Accordingly, if there is a requirement to keep a record of what was destroyed, this will need to be factored in to whatever retention policy is created. 

 

Using MS Teams without an Exchange Online mailbox

When people chat in Microsoft Teams (MS Teams), a ‘compliance’ copy of the chat is saved to either personal or (Microsoft 365) Group mailboxes. This copy is subject to retention policies, and can be found and exported via Content Search. But what happens if there is no Exchange Online mailbox? It seems the chats become […]

A modern way to manage the retention of digital records

In his April 2007 article titled ‘Useful Void: The Art of Forgetting in the Age of Ubiquitous Computing’ (Harvard University RWP07-022), Viktor Mayer-Schönberger noted that the default human behaviour for millenia was to forget. Only information that needed to be kept would be retained. He noted that the digital world had changed the default to […]

Author:

I am an experienced information management professional based in Melbourne, Australia. I have had close to 40 years of practical working knowledge across the full spectrum of information, records and content management issues, and direct and practical experience with contemporary and emerging business and information and enterprise content management systems. My product knowledge includes SharePoint 2010/2013/Online and OneDrive (SharePoint Administrator), Office 365 (including as a Global Administrator), Yammer, Sway, TRIM Context (R6.2 & 7.1), ECM Documentum, Alfresco Share; and other online systems. www.andrewwarland.com.au

12 thoughts on “Managing MS Teams chat as records

  1. This post has been really useful in helping me understand chat storage and retention – thank you! I wonder if you can help with a query I still have: If I apply a retention policy to chats for a select group of users to delete chats after a certain amount of time but no policy at all to any other users, what would happen at the end of the retention period to a chat between a user from the group with the policy applied and a user without the policy? Would the user without the policy no longer be able to see the messages from the other participant?

    1. Thanks for the very interesting question Sian. Just to be clear, I understand your question to relate to 1:1 chat, not channel-based chat – but where the 1:1 chat involves several people. I will test this and report back but my understanding is that the retention policy applied to one person will delete the copies of the chat messages in their personal Exchange mailbox which in turn deletes the copy from the Teams database. What I don’t know is whether it will also delete the messages for the other participants OR make the message invisible to the person to whom the policy applied OR make the message still available via Teams. I’ll check this and come back with a comment.

      1. Thanks so much for looking into this. Yes, I do mean 1:1 chat rather than channel-based chat – but I mean chats that only involve 2 people as well as chats that might involve more people. (To be a bit more specific, I’m thinking of an education setting where generally we might want to delete staff chats after a short period of time, but wouldn’t want to delete student chats or interactions between students and staff.)

      2. Thanks for clarifying the question Sian. It’s an interesting point. If you have a targeted short-term policy for staff chats, that will delete all their chats. However, if you have a second longer-term policy for students (and staff chats with students) would would have to exclude those staff from that policy. What I’m not sure is if the short-term policy will delete the chat from the staff chat with students. I think yes, leaving the staff/student chat with only one side of the picture. I should be able to confirm this today or tomorrow.

  2. Hi Andrew – I’m sorry to bug you, but I just wondered if you’d been able to get an answer from your test of this?

    1. Sorry I didn’t follow up with this Sian. I set up a policy to delete chat messages from one person only and it seems to have also deleted the chat from the other side also, which is not what I expected to see. Just to check, I have just now set up a different, longer policy that applies to the other side. In summary: (a) sender A has a 2 day retention, (b) sender B (as part of everyone except sender A) has a 30 day retention. I’ll come back to you by Friday your time with the outcome.

    2. Hi Sian, here is the result of the tests. I created two retention policies for MS Teams chats (not channel chats): (a) a 30 day policy for everyone except Person A; (b) a 2 day policy for only Person A. Then I exchanged chats between Person A and another person. After the 2-day retention policy expired, the chat messages from or two person A were deleted for everyone (no longer visible to either side). However, as the chat messages were subject to a longer retention policy, they were still accessible via Content Search. So it seems that any retention policy will remove both sides of the chat but if the chats are subject to a longer policy, the chats will still be retained for that longer period. Hope this answers the question. It basically means that any retention policy will remove the chats from both sides.

      1. Hi Andrew, thank you so much for testing that for me. Unfortunately it’s not the result I was hoping for! This means (if I’m understanding correctly) that a conversation between a member of staff (retention, say, 6 months) and a student (no retention policy) would disappear for both people at the end of the retention period. A student would no longer be able to refer back to the conversation (even if we put a permanent retention policy on student chats, since the students couldn’t use Content Search). And such an interaction between staff and student would probably need to be retained for much longer in any case. So I think what I’m wishing for is the ability to set multiple retention policies for chat based on purpose, not user. We will either have to not set a short retention policy at all for staff who interact with students or have some strict guidance around when and how to use chat.
        Anyway, I’m just musing out loud there. Thank you very much for your help!

      2. Thanks Sian, yes I was surprised at the outcome also, but it kind of makes sense, I guess since the message is deleted. It may be worth posting this on the Tech Community blog – if you don’t have access, I might post it anyway to see if others have some ideas about it. Let me know if you want me to do this.

      3. I haven’t posted to the Tech Community blog before, so if you would like to do that, please do go ahead. I am considering posting to one of the RM mailing lists to see if anyone has any advice about chat retention in light of this, particularly in a university setting.

      4. Hi Sian, the only reply I have received so far is ‘thank you for sharing this information’ (from a Microsoft person). So I think what I’ve said is correct.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s