One of the first things to understand, when managing records in Microsoft 365, is where the records are stored.
Generally speaking, most records will be stored in either (a) Exchange mailboxes (which also store the ‘compliance copies’ of Teams chats) or (b) SharePoint sites (including sites linked with MS Teams). Some emails may be copied to SharePoint and some records may also end up in OneDrive accounts.
Records should provide evidence of business activities and may also be associated with metadata. But while records may be evidence of such activities, compliance requirements may differ. For example:
Some records must be retained permanently and transferred to archival institutions as a public record, while others may be of temporary or transient value.
The evidentiary value of records may depend on the context and content of the record. For example, records detailing a high value contract decision versus the minutes of a low-level team meeting.
With the above differences in mind, the following is a suggested decision tree for the management of records, in a controlled Microsoft 365 environment. This decision tree will not apply in organisations where end-users can create their own Teams and SharePoint sites.
The decision tree below starts with the simple question to the end-user or business area wanting a new Team or SharePoint site:
The answer to this question should help to identify what type of SharePoint site, or Team (with a SharePoint site), is required to manage the records. Note that ALL of the boxes should be considered BEFORE a site (or Team) is created.
Note: the box outline colours represent the four different types of site that are likely to be created.
How will the content be accessed?
It is important to understand how end-users will create, capture, manage and access the content that is stored in SharePoint.
Microsoft have made it clear that they expect ‘most’ end-users will access SharePoint via Teams.
They may also sync the SharePoint document library to File Explorer. If that option is not removed, then it is important to ensure that end-users know how to access the Recycle Bin and version history by going to the Files tab in Teams and clicking on ‘Open in SharePoint’.
If end-users are likely to mostly use the Teams Files tab or the synced version in File Explorer, it may be useful to suggest that they save the SharePoint site as a favourite in their browser.
It is important not to lose site of the Microsoft 365 Group-based option which gives the Group an email account that can be accessed via Outlook (the SharePoint site for the Group can also be accessed directly in Outlook Online).
What type of SharePoint site will be required?
Once the two points above are clear, it is then possible to understand what type of site will need to be created, and how the records will (or will need to) be managed. For example, if Content Types and/or additional metadata is required.
As a general rule, the SharePoint sites linked with Teams are more likely to remain simple, folder-based libraries because, while additional Content Types and metadata can be used, end-users may not react favourably to additional document saving requirements, especially if they access the library via File Explorer.
Organisations with more complex recordkeeping requirements might consider establishing SharePoint sites dedicated for that purpose and accessed via the browser. These may and may not be linked with Microsoft 365 Groups.
The following shows how the new site will be created (if it’s not scripted). Microsoft recently announced that it will add the ‘Created From’ column to the list of active sites in the SharePoint admin portal.
Different site types are likely to have different security controls. Team/Microsoft 365 Group-based sites use the Group Owners and Members to restrict access to the SharePoint site (the Owners and Members Group are added (respectively) to the Owners and Members permission group of the SharePoint sites), whereas a SharePoint site not linked with an M365 Group is more likely to use Security Groups.
Communication sites are likely to have very few Members and ‘Everyone except external users’ added to the Visitors permission group.
Finally, a decision needs to be made about what type of retention policy will be applied to the SharePoint site or the content stored in it. My suggestion is to apply a broad ‘safety net’ retention policy to all SharePoint sites and then use label-based policies as necessary.
As much as possible, the content of sites should ‘map’ to business function and the retention classes in that business function. For example, try to avoid storing legal records with HR records and property records on the same site. Retention management will get complicated.
Label-based policies might have no disposal outcome (‘Do nothing’) but they ensure that the records cannot be destroyed and are labelled accordingly. For example, records that are of permanent value could be labelled with a ‘Do nothing’ retention label in dedicated document libraries where edit rights have been removed, thereby locking them from disposal.
Once the site has been created, it should be provisioned with three key elements shown below.
Records managers should be assigned to a Security Group that is added to the Site Collection Admin area of every SharePoint site. This will allow records managers to manage the content of those sites.
Document IDs should always be enabled and configured (remembering they have a 12 character prefix limit)
The SharePoint Viewers feature should be enabled on every site to provide visibility of who viewed the content.
The decision tree should help to guide the create of new sites and Teams (which have a SharePoint site) for the storage and management of records.
One of the most confusing aspects of Teams and SharePoint in Microsoft 365 is the relationship between permission groups used to control access to both of these resources. This is especially the case as every Team in MS Teams has an associated SharePoint site (the ‘Files’ tab).
This post explains how permission groups work between MS Teams, Microsoft 365 Groups and SharePoint.
SharePoint permission groups
Before discussing how Teams permissions relate to SharePoint, here is a brief reminder of how SharePoint permissions work.
SharePoint has always had three default permission groups, prefixed by the URL name of the site, as shown in the screenshot below (the name of the site always prefixes the words Owners, Members and Visitors).
People (including in a Group, see below) added to the Owners permission group have full access (full control) to all parts of the site and are usually responsible for managing the SharePoint site. There would normally be two or three site owners.
People (including in a Group, see below) added to the Members permission group have add/edit (contribute) rights.
People added to the Visitors permission group have read-only (view) rights.
These permissions are set at the site level and inherited on everything in the site, unless that inheritance is broken and unique permission are applied. Additional permission groups can be created as necessary but most SharePoint sites only use the default Owners, Members and Visitors groups.
Microsoft 365 Groups
Microsoft 365 Groups were introduced in 2017 and control access to resources, like Security Groups.
However, unlike Security Groups, which usually provide access to individual resources (such as a single SharePoint site, or Line of Business (LOB) system), Microsoft 365 Groups control access to multiple linked Microsoft 365 resources.
Microsoft 365 groups, distribution lists, mail-enabled security groups, and security groups (collectively referred to as Active Directory (AD) groups, are all created in ‘Groups’ area of the Microsoft 365 Admin portal.
When a new group is created, the following options appear.
As noted above, Microsoft 365 groups are recommended. It is important to understand the relationship between Microsoft 365 groups, Teams and SharePoint.
When a new Microsoft 365 group is created (from the dialogue above), it creates:
At least one Owner must be specified. The Owner/s are responsible for managing the Members group.
An Exchange mailbox with the same email @ name as the Microsoft 365 group. The mailbox is visible in Outlook to the members of the Group.
A SharePoint site with the same URL name as the Microsoft 365 group.
By default (unless the checkbox is unchecked), a new Team is also created in MS Teams.
When a new Team is created from MS Teams, or a new SharePoint Team site is created, it creates:
A Microsoft 365 Group with an Exchange mailbox and a SharePoint site (‘Files’ tab).
The name of the Team becomes the name of the Group and the SharePoint site.
The mailbox is not visible in Outlook and is only used for calendaring and for the storage of Teams chats (in a hidden folder).
Importantly, when a new Microsoft 365 group or Team is created (which creates a Microsoft 365 group), the Group Owners: (a) are the same as the Team Owners and (b) are added to the SharePoint Owners permission group, as explained below. .
Group/Team Owners and Members
In other words, the Microsoft 365 group owners (group) is added to the SharePoint site owners permission group – a ‘group within a group’.
That is, the Microsoft 365 group controls access to the Team and the SharePoint site as shown in the diagram below. Security Groups may also be added to the Microsoft 365 Group site, but this does not provide access to the Team.
This ‘group within a group’ model is visible from the ‘Site Permissions’ section of the gear/cog icon as shown below (the name of the Microsoft 365 Group/Team/SharePoint site is ‘SharePoint Admin’). The SharePoint Admin Group Owners (group) is in the SharePoint site owners group, and the SharePoint Admin Group Members (group) is in the Site members group.
If a mouse hovers over the Group ‘icon’ (in the above example, GO or GM), it is possible to view the members of the Group and, for Owners, to modify that list. Confusingly, the ‘GM’ in the SharePoint site permissions group becomes ‘SG’ in the drop down list.
You can also see the ‘group within group’ model from the back-end ‘Advanced permissions’ section of the SharePoint site, but you cannot manage the Microsoft 365 Group members here.
Implementing the model
As with Security Groups, the members of Microsoft 365 Groups will usually be a logical group of people who require access to something, in this case access to the SharePoint site or the Team (for chat, files, or other resources).
The main thing to remember is that membership of the (backend) Microsoft 365 Group provides access to BOTH the Team and the Team’s SharePoint site (the ‘Files’ tab in a Team).
Every Team in MS Teams will usually consist of the members of a logical group with a common interest – a business unit, project team, or with some other work relationship, for example, the members of a committee. The Team Owners are responsible for managing the Team Members.
The Team Owners are the SharePoint site owners and are responsible for managing the site if they decide to access it directly. The Team Members are the SharePoint site members and have the ability to add or edit content, usually via the ‘Files’ tab in Teams.
Note: Security Groups with the same members as Microsoft 365 Groups (and Teams) may already exist. There is no need to add a Security Group if it has the same members as a Microsoft 365 Group.
As noted earlier, a Group/Team does not have visitors with read-only rights. Every Member of the Team has add/edit access to both the Team and its associated SharePoint site.
If there is a requirement to give specific other people either add/edit or read-only access to the SharePoint site, that outcome is achieved by adding people by name, or a Security Group, to either the SharePoint Members or Visitors group.
If there is a requirement to give everyone in the organisation either add/edit rights, or read only access, to the SharePoint site, that outcome is achieved by adding ‘Everyone except external users’ to either the SharePoint Members or Visitors group.
External guests may also be added to the Team and the Team’s SharePoint site.
During 2020, many organisations rolled out Microsoft Teams to support the need for employees to work from home (WFH) without paying much attention to the way Teams were named.
A reminder that when a new Team is created, it creates a Microsoft 365 (M365) Group. Every M365 Group has a SharePoint site (visible from the ‘Files’ tab in the Team channels) and an Exchange mailbox (used for calendaring and to store the ‘compliance copy’ of chat messages).
The name given to the Team becomes the display name for the M365 Group and the SharePoint site. For example, ‘Testing multi word Team name’.
The same name, less any spaces between words, becomes the URL name and the email address. For example, ‘/sites/TestingmultiwordTeamname’ and ‘TestingmultiwordTeamname@tenantname.onmicrosoft.com’.
What happens if you want to change the name of the Team, M365 Group or SharePoint site? And what are the potential implications of changing names?
Changing the name of the Team or Group
To change the name of the Team, click on the three dot menu to the right of the name, then ‘Edit Team’ and change the name in the dialog box that appears.
The new Team name will appear immediately. The name of the M365 Group will change soon after.
The name of the M365 Group (and its email address) can also be changed by admins via the Groups section of the Microsoft 365 admin portal. The name of the Team will change soon after.
The display name on the SharePoint site may take a little longer to change.
If you need to change the SharePoint URL name, go to the SharePoint Admin portal, click on the site name and, in the General tab area, click on ‘Edit’ under the URL name.
As long as you can access this section and the new URL name is available, it can be changed:
Keep in mind that, if you change the site URL name, the Team (initially at least) may throw an error:
But if you click ‘Open in SharePoint’, it will re-connect the site to the Team/Group and become visible again.
Implications of changing names
Generally there are no implications in changing the display name of a Team or a SharePoint site as described above.
However, ideally, there should be some correlation between the name of the Team/Group, the display name of the associated SharePoint site, and the URL name. It is not uncommon to see Teams or SharePoint site display names that bear little resemblance to the site URL name.
The main implication of changing a site URL name is that it may break any links, either shared or embedded in documents. For example, the example below is the URL of a link with the URL name highlighted in bold. If the URL is changed, the link will no longer work:
SharePoint is a core foundational element in Microsoft 365. It is primarily used for the storage of digital objects (including pages) in document libraries and rows and columns of data in lists. It is ubiquitous and almost impossible to remove from a Microsoft 365 licence because it ‘powers’ so many different things.
While the idea that anyone can easily create a SharePoint site seems a good idea in some ways, from a recordkeeping of view this starts to look like network file shares all over again.
Microsoft’s response to the default ‘free for all’ ability to create SharePoint sites is to use the so-called ‘records management’ functionality (via the more expensive E5 licence) to auto-classify content and auto-apply retention labels. The problem is that those (more expensive options) provide limited functionality, including inadequate metadata details to make decisions on disposal, and similarly inadequate metadata (for records subject to disposition review labels only) as ‘proof of disposition’.
So, records managers are more often than not left with a network file share-like sprawl of uncontrolled content.
Unfortunately, the ability to create a new SharePoint site is fairly easy, almost as easy as creating a folder on a … network file share.
The following is a list of the main ways a person can create a SharePoint site. Have I missed any?
This option also allows the administrator to provision new SharePoint sites.
2. Via the SharePoint Admin portal (+ Create)
This option allows the creation of three main types of sites: modern team sites (Team site), communication sites, and non-Microsoft 365 Group-linked sites (Other options).
3. By creating a Microsoft 365 Group
Microsoft 365 Groups are created in the Microsoft 365 Admin portal, in the Groups section, Add a group > Microsoft 365. This is also where Security Groups and Distribution Lists (both collectively known as ‘AD Groups’) are created.
Every new Microsoft 365 Group creates both a SharePoint site and an Exchange mailbox that is visible in the Outlook application (under ‘Groups’) of everyone who is an Owner or a Member of the Group.
The new Group creation process allows the Group email address to be created (it really should be the same as the Group name), the Group to be made public or private, and a new Team to be created.
Because the Microsoft 365 Group name becomes the SharePoint site (URL) name, it is a good idea to consider naming conventions.
4. By an end-user creating a new Team in MS Teams
Unless the creation of Microsoft 365 Groups is not restricted, an end-user can create a new SharePoint site (possibly without realising it) by creating a new Team in MS Teams. There is nothing in the creation process to indicate that (a) they will create a SharePoint site or a Microsoft 365 Group, or (b) that they will be the Owner of the Team, Group and SharePoint site – and therefore have responsibility for managing the Team/Group membership.
Every new Team creates a Microsoft 365 Group which always has a SharePoint site and an Exchange Online mailbox that is not visible in Outlook.
5. By creating a Private Channel in MS Teams
If the option is not disabled in the MS Teams admin portal under Teams > Teams Policies, end users will be able to create private channel in a Teams channel. Every private channel creates a new SharePoint site with a name that is an extension of the ‘parent’ Team site name.
For example, if the parent site name is ‘Finance’ and the private channel is named ‘Invoice chat’, the new SharePoint site will be ‘Finance-Invoicechat’. These new site is not connected with the ‘parent’ site and is not visible in the list of Active Sites from the SharePoint admin portal (and so the SharePoint Admin won’t know it exists). It is only visible in the list of Sites under the Resources section of the Microsoft 365 Admin portal.
A private channel does not create a new Microsoft 365 Group. A ‘compliance copy’ of the chats in the private channel are stored in the Exchange Online mailboxes of individual participants in the chat.
6. By the Teams Admin creating a new Team
The MS Teams admin area includes the ability for the Teams admin to go to Manage Teams, click +Add and create a new Team.
As with the end-user creation process, a new Team creates a Microsoft 365 Group that has an Exchange mailbox and a SharePoint site.
7. From the end-user SharePoint portal (+ Create site)
This process creates a Microsoft 365 Group that has a SharePoint site and an Exchange mailbox. It also creates a new Team with the same name.
It is recommended that the ability for end-users to create new sites this way is disabled, at least initially. This is done from the SharePoint admin portal under Settings > Site Creation.
8. From OneDrive for Business as a ‘shared library’
This option is relatively new. When the end-user opens their OneDrive for Business, they will see ‘Create shared library’ directly under a list of sites they have access to under a heading ‘Shared libraries’ (they are actually SharePoint sites; when you click on the site name, it (confusingly) displays the document libraries as … folders.
9. When a new Plan is created in Planner
If end-users open the Planner app, they will see ‘New Plan’ on the top left. This opens a dialogue to create a New Plan or add one to an existing Microsoft 365 Group. The process of creating a new Plan creates a new Microsoft 365 Group with a SharePoint site.
10. When a new Yammer community is created
End users with access to Yammer can click on ‘Create a Community’ from Yammer.
To quote from the Microsoft 365 documentation ‘Join and create a community in Yammer‘: ‘When a new Office 365 connected Yammer community is created, it gets a new SharePoint site, SharePoint document library, OneNote notebook, plan in Microsoft Planner, and shows up in the Global Address Book.’
Why have Microsoft allowed this?
It’s a smarter way to manage access.
Some years back, Microsoft moved away from the idea of having Security Groups that give access to individual IT resources, to having individual Microsoft 365 Groups that provide access to multiple IT resources, in this case resources across Microsoft 365. One Microsoft 365 Group controls access to a SharePoint site, an Exchange mailbox, a Team, a Plan, and a Yammer Community. Security Groups don’t have that sort of functionality.
The trade off is that you get all of these options with a Microsoft 365 Group, whether you like it or not.
But, some of the decisions don’t seem to make sense.
Why allow end-users to create a private channel in Teams when they can simply use the 1:1 chat area?
Why allow the creation of a so-called ‘Shared Library’ from OneDrive, limited to and controlled by the person who created it, when a SharePoint site provides that functionality.
Why does an end-user need an Exchange mailbox (for the Microsoft 365 Group) when they create a new site from the ‘Create site’ option in SharePoint?
And why does a new Plan create a SharePoint site? For what purpose?
Perhaps there is a reason for it. It’s just not clear.
When people chat in Microsoft Teams (MS Teams), a ‘compliance’ copy of the chat is saved to either personal or (Microsoft 365) Group mailboxes. This copy is subject to retention policies, and can be found and exported via Content Search.
But what happens if there is no Exchange Online mailbox? It seems the chats become inaccessible which could be an issue from a recordkeeping and compliance point of view.
This post explains what happens, and why it may not be a good idea (from a compliance and recordkeeping point of view) not to disable the Exchange Online mailbox option as part of licence provisioning.
Licences and Exchange Online mailboxes
When an end-user is allocated a licence for Microsoft 365, a decision (sometimes incorporated into a script) is made about which of the purchased licences – and apps in those licences – will be assigned to that person.
E1, E3 and E5 licences include ‘Exchange Online’ as an option under ‘Apps’. This option is checked by default (along with many of the other options), but it can be disabled (as shown below).
If the checkbox option is disabled as part of the licence assigning process (not after), the end-user won’t have an Exchange mailbox and so won’t see the Outlook option when they log on to office.com portal. (Note – If they have an on-premise mailbox, that will continue to exist, nothing changes).
Having an Exchange Online mailbox is important if end-users are using MS Teams, because the ‘compliance’ copy of 1:1 chat messages in MS Teams are stored in a hidden folder (/Conversation History/Team Chat) in the Exchange Online mailbox of every participant in the chat. If the mailbox doesn’t exist, those copies aren’t made and so aren’t accessible and may be deleted.
If end-users chat with other end-users who don’t have an Exchange mailbox as shown in the example below, the same thing happen – no compliance copy is kept. The chat remains inaccessible (unless the Global Admins take over the account).
The exchange above, between Roger Bond and Charles, includes some specific key words. As we will see below, these chats cannot be found via a Content Search.
(On a related note, if the ability to create private channels is enabled and they create a private channel and chat there, the chats are also not saved because a compliance copy of private channel chats are stored in the mailboxes of the individual participants.)
Searching for chats when no mailbox exists
As we can see above, the word ‘mosquito’ was contained in the chat messages between Roger and Charles.
Content Searches are carried out via the Compliance portal and are more or less the same as eDiscovery searches in that they are created as cases.
From the Content Search option, a new search is created by clicking on ‘+New Search’, as shown below. The word ‘mosquito’ has been added as a keyword.
We then need to determine where the search will look. In this case the search will look through all the options shown below, including all mailboxes and Teams messages.
When the search was run, the results area shows the words ‘No results found’.
Clicking on ‘Status details’ in the search results, the following information is displayed – ‘0 items’ found. The ‘5 unindexed items’ is unrelated to this search and simply indicates that there are 5 unindexed items.
Double-checking the results
To confirm the results were accurate, another search was conducted where the end-user originally did not have a mailbox, and then was assigned one.
If the end-user didn’t have a mailbox but the other recipient/s of the message did, the Content Search found one copy of the chat message in the mailbox of the other participants. Only one item was found.
When the Exchange Online option was enabled for the end-user who previously did not have a mailbox (so they were now assigned a mailbox), a copy of the chat was found in the mailbox of both participants, as shown in the details below (‘2 items’).
Summary and implications
If end users chat in the 1:1 area of MS Teams and don’t have an Exchange Online mailbox, no compliance copy of the chat will be saved, and so it will not be found via Content Search.
If any of the participants in the 1:1 chat have an Exchange Online mailbox, the chat will appear in the mailboxes of those participants.
If all participants in the 1:1 chat have an Exchange Online mailbox, the chat will be found in the mailbox of all participants.
Further to the above:
If end users can delete chats (via Teams policies) and don’t have a mailbox, no copy of the chat will exist.
If end-users with a mailbox can delete Teams chats, but a retention policy has been applied to the chats, the chats will be retained as per the retention policy (in a hidden folder).
And finally, if you allow private channels, end-users can create private channels in the Organisation Team. The chats in these private channels are usually stored in the personal mailboxes of participants (not the Group mailbox) – so these chats will also be inaccessible and cannot be found via Content Search.
The implications for the above are that, if you need to ensure that personal chat messages can be accessed (from Content Search), then the participants in the chat must have an Exchange Online mailbox.
Further, if you allow deletion of chats but need to be able to recover them for compliance purposes, a retention policy should be applied to Teams 1:1 chat.
At the 2020 Microsoft Ignite conference, Jeff Teper presented a diagram titled ‘Microsoft 365’. The diagram showed only four icons: Teams, Outlook, Office and Edge.
The implication of this diagram was that, for most end-users, Teams is now (or will become) their primary portal into Microsoft 365. As stated by Jeff Teper, SharePoint is a foundation platform, the out of sight content engine. Edge’s ability to serve up search results from Microsoft 365 further reduces the need to go to SharePoint.
So, what are the implications for managing records?
SharePoint as a recordkeeping system
For a long time, records have been created, captured and stored in recordkeeping systems.
In the paper world, the recordkeeping system consisted of paper records stored in files and boxes and detailed in registers. With the introduction of computers in the 1980s, registers were transferred to databases, making it a bit easier to find records. In the late 1990s, recordkeeping databases were linked with (separate) file stores and became electronic document and records management (EDRM) systems that continued to manage paper records (the so-called ‘hybrid’ systems).
For almost a decade (since SharePoint 2010 was introduced), SharePoint has contended with files shares and EDRM systems as an alternative recordkeeping system, providing almost all the same core functionality.
The ability to create a record in a single location, then share and co-author it from that location, has completely removed the requirement to copy a record to a separate recordkeeping system.
And then came Teams
Someone at Microsoft had incredible foresight to see the potential for a new user interface that would replace products like Lync and Skype for chat and conferencing, and would also provide access to files stored in SharePoint.
SharePoint has been a core part of the Microsoft productivity offerings for a very long time and people have built careers around developing functionality on the SharePoint platform to appeal to end-users, the intranet being the most common case in point, with customised team sites close behind.
The arrival of Microsoft 365 Groups and then Teams in 2017 was perhaps not widely noticed. One could argue that end by the beginning of 2020, it was still largely unnoticed.
And then came a pandemic and working from home. Teams – which may have been largely ignored or overlooked until then – was already ready to take its place next to Outlook, Office and Edge as a primary end-user interface.
New Teams were created, sometimes with abandon (and were sometimes just as quickly abandoned).
Both 1:1 (or 1:many) chats and channel chats took off. Files were created and shared via OneDrive for Business (‘Files’ in the 1:1 chat area), or via the back-end SharePoint sites (‘Files’ in the channel chat area).
There was (and maybe still is) a belief that files were being saved to Teams but not SharePoint. ‘We are storing everything in Teams’ was not an uncommon expression, sometimes followed by ‘but we’re not using SharePoint or OneDrive’.
The year 2020 saw a huge increase in the volume of records stored in SharePoint sites linked with Teams, as well as a completely new set of records – chats (‘compliance’ copies of which are stored in Exchange mailboxes).
The diagram below provides an overview of the relationship between Teams, Microsoft 365 Groups, Exchange mailboxes, SharePoint and OneDrive for Business.
What about SharePoint?
As the diagram above shows, SharePoint has not disappeared. Many organisations will continue to use, and ask end-users to access, SharePoint sites directly to store and manage records.
But accessing SharePoint from SharePoint may become less necessary over time. At Ignite 2020, the ability to pin a ‘home site’ (such as an intranet) to Teams was demonstrated. Even the intranet may end up in Teams.
As Jeff Teper said, SharePoint is a foundation platform, one that does not get in the way of collaboration and productivity but powers it.
Implications for records managers
Records managers, who were likely already on a steep learning curve regarding SharePoint, need to continue to improve their knowledge of the SharePoint platform. On a positive note, SharePoint Online is a much easier application to learn and manage, compared with its earlier on-premise predecessors.
In organisations that have been using SharePoint for a while and/or have allowed the free-creation of Teams in MS Teams, there will some requirement for retrospective analysis, review, and cleaning up.
In all organisations, there will be a requirement to establish some form of governance and oversight of records (files and chats) that have been created, including for the purpose of retention and disposal/disposition.
Where MS Teams has been implemented with little thought given to naming conventions, SharePoint site provisioning, or access controls, records managers should been given access to and review the list of all SharePoint sites that have been created, including from MS Teams. This will provide an initial idea of the volume of content and activity on each site, and what action needs to be taken on things like inactive Teams.
Ideally, records managers should be added to the Site Collection Administrators (SCA) group of every SharePoint site, including MS Teams-based sites. This action will give records managers access to the content on every site and to help advise on the management of records in those sites (including Team-based sites).
The best way to do this is to add records managers to a Security Group and then add that Group to the SCA group of every site. This access could be deferred for sites that contain very sensitive information, although typically records managers would have access to all records, including if they had an EDRMS. And, access is always recorded in audit logs or the local site ‘viewers’ (where enabled) and ‘last modified by’ information.
Access to the chat content of Teams (including 1:1 chats) will not normally be required; some understanding of the content could be inferred from the name of the Team or the SharePoint content. If necessary, Global Admins or a Compliance Admin can run a Content Search across Teams to find chat content, and/or export that content by an individual person or subject.
Records managers will also need to advise on the appropriate retention policy or policies that need to be created and then applied to:
The chat content in 1:1 chats.
The chat content in the various Teams.
SharePoint sites linked with Teams.
OneDrive for Business accounts. An additional consideration is how long the content of inactive ODfB acccounts should be retained via the ‘Storage’ policy (default is 30 days then permanent deletion).
SharePoint sites not linked with MS Teams. This includes whole sites as well as library-based retention policies.
Office 365 Groups (mailbox/SharePoint site). If linked with a Team, a second retention policy is required for the Team chat content retention (second dot point above). For example, one policy ‘GroupABC’ and a second policy ‘GroupABCTeamChat’.
As many of the above retention policies replace the need for backups, records managers need to discuss the options with their IT colleagues.
Forward looking implications
Ideally, there should be some form of governance around the creation of new Teams in MS Teams. These governance arrangements might include:
The necessary access for records managers. For example, Site Collection Administrator on every site, and/or a customised Compliance Admin role to create and access retention policies.
Controls around the creation of new Teams, including naming conventions. If not controlled, what processes will ensure that records are properly managed.
Retention implications. For example, can the new site and/or the channel chat content be covered by another retention policy – e.g., ‘All Teams with assessed low-level working content should be kept for 5 years’.
Simple best practice guidance for all new users, including on how to share and co-author.
Retention policies for all Microsoft 365 content, not just SharePoint.
Reviews of the content of OneDrive for Business accounts of departed end-users, especially for people in senior or decision making positions. It is relatively common practice for end-users to delete (and download) this content before they leave their jobs.
Monitoring and oversight of content, including access to reporting dashboards.
So, is Microsoft 365 just Teams, Outlook and Office (in Edge)?
For many, or not most information based end-users, MS Teams is likely to become the primary interface to Microsoft 365 collaboration team spaces including SharePoint and OneDrive. Just like Outlook, Teams will probably be left open all day.
In theory, the volume of low-value emails, and emails with attachments, should reduce over time.
The developing role of records managers
In this new world, the role of records managers will change from being the curators of records copied to and stored in a separate ‘records and document management’ system, to being records compliance analysts or perhaps, corporate knowledge and information managers and content analysts.
They will learn what the Graph can do, and help to guide AI tools including machine learning and machine teaching, Project Cortex and SharePoint Syntex. They will be responsible for monitoring content across the Microsoft 365 platform, creating and applying retention policies and managing the outcome of those policies, working more interactively with the Graph, and with a range of data.
In organisations that have a requirement to transfer records to archival institutions, the new knowledge and information managers will have a key role in ensuring that this data is suitable for transfer.
They might even have oversight of old paper records gathering dust until they can be destroyed.
The international standard for records management, ISO 15489-1:2016 (‘Information and documentation – Records management – Part 1: Concepts and Principles’), defines records as ‘information created, received, and maintained as evidence and as an asset by an organization or person, in pursuit of legal obligations or in the transaction of business’.
Among other things, the standard notes that records systems may exist in a variety of forms, not necessary as or in a single or dedicated application. It also underlines the importance of appraisal; that is, the recurrent analysis of business context, business activity, processes and risk for the purpose of determining what records to make and keep and how to manage them over time – especially given the complexity of contemporary recordkeeping.
In terms of risks, the standard states that risk management is required to develop strategies for managing records and the management of records as a risk management strategy in itself.
Unlike traditional electronic document and records management (EDRM) systems that are used to store copies of records created and stored in other applications (‘exception management’), the Microsoft 365 environment is a single system in which records are a sub-set of the entire content (‘exception identification’).
This post discusses how records can be collated, grouped and aggregated in Microsoft 365 to meet requirements for management records. It emphases the point made in the international standard that the risk to records should be understood and minimised.
Records and context
Records are usually created or captured in some form of context – for example a business activity or project. This in turn provides the basis for collating, grouping or aggregating those records according to that context – commonly, a ‘subject’ or ‘topic’.
Records may be a subset of a broader subject (or series). They may be relevant or relate to more than one context or subject.
Digital records that may have no obvious context when they are first created or capture (for example a casual email about an ‘unusual virus outbreak’ in November 2019) may form part of a specific context only when their value is recognised (‘global pandemic’).
Grouping digital records
Grouping records in the digital world has up until now usually involved copying a digital record, created or captured in one system (such as email or a network file share), to a digital ‘file’ in another system such as an electronic document and records management (EDRM) system. The digital ‘file’ in those systems is a virtual representation; the records are actually stored in a file share, linked by metadata in the form of a file number.
The grouping of digital records as exceptions had (and continues to have) several flaws:
It assumed that all types of digital records could be stored in a digital ‘file’ from where they could be faithfully and reliably rendered (and not just stored as zipped versions of exported content from the originating system).
It relied on the willingness of end-users (often after training) and/or a technical third-party system, to copy a record to the system. This ‘exception management’ meant that some records were not copied to the EDRMS.
It was a ‘point in time’ capture. The original digital record remained in the system where it was created or captured, and might also be attached to emails and from there saved to multiple other locations.
There was no way of knowing if all the records in the file were all the records relating to the subject.
Where are the records created or captured in Microsoft 365
Most business records in Microsoft 365 will be created or captured in Outlook/Exchange mailboxes, SharePoint site libraries or MS Teams (which stores chat in Exchange mailboxes and documents in SharePoint or OneDrive). (For the purpose of this post, OneDrive is seen as a personal working space that should not be used to store business records.)
Regardless of whether they are created or captured in Exchange or SharePoint (including via Teams), all of the content – records and non records – created or captured in Microsoft 365 is stored in the Azure substrate. This effectively means that records in Microsoft 365 are a sub-set of all the other content stored in the Azure substrate.
Consequently, the management of records in Microsoft 365 involves exception identification. That is, identifying records and ensuring they are managed appropriately as much as possible where they are captured or created – and placing other controls over all the other content as necessary.
Everything created and stored in Microsoft 365 – including all the very rich metadata associated with every digital record – is subject to the Graph. The Graph identifies relationships and ‘signals’ not only between digital content but between people (agents) and business activities.
The Graph powers Delve and Discovery and the soon-to-be-released Project Cortex, presenting information (they have access to) to end-users that can sometimes be unsettling for people used to working in relative privacy. See below for further discussion about Project Cortex.
Additionally, as all the content in Microsoft 365 is stored in the Azure back-end, most of it can be searched and (where necessary) exported through the Content Search option in the Compliance portal, a capability that supports eDiscovery. This capability means that even when records are not ‘manually’ identified as records, there is a better chance they will be found.
How are records aggregated in Microsoft 365
There are three main ways that records are, or can be, aggregated in Microsoft 365: Exchange mailboxes, SharePoint site libraries, and Microsoft Groups that have a mailbox and a SharePoint site and can be linked to (or created from) a Team in MS Teams.
Exchange aggregates email records by:
Personal mailboxes, accessible only the ‘owner’ (end-user).
Shared mailboxes, accessible to those who have access.
Microsoft 365 Group mailboxes, accessible to the members of the Group (including anyone added to the Group).
Although a mailbox is a form of aggregation, there is no way to relate or link emails stored there with other related records stored in SharePoint unless they are copied to a SharePoint document library, as can be seen in the example below. This is recommended if an organisation wants to keep emails together with other records.
Emails copied to a SharePoint document library are a ‘point in time’ copy; there may be additional replies to the email, forming a thread that isn’t captured.
The alternatives to copying emails to SharePoint are:
Leave all emails in mailboxes and use Content Search to find and export them to SharePoint as a PST.
Creating a Microsoft 365 Group with an associated mailbox and SharePoint site, so that the records are retained in the context of the Group.
In any case, all mailboxes should be subject to a minimum retention period to ensure that any email that might be a record is preserved for that period. Certain mailboxes (for example, senior or key staff members) may be kept for longer periods and then exported for permanent storage.
SharePoint document libraries are logical aggregations for the storage of records, including emails copied from Exchange mailboxes.
Ideally, individual libraries that are used for the storage of records should map to a business activity and/or records retention class; this mapping should be reflected in the library name.
NOTE: Individual document libraries should not be used to store records relating to multiple subjects or mapping to more than one retention class or policy.
Document libraries may be assigned as much metadata as required, and content stored in them can be defined through the use of metadata and/or content types.
Microsoft 365 Groups (including Teams in MS Teams)
Microsoft 365 Groups provide a way to group and manage records, including MS Teams channel chats, in the context of the Group.
Every Group includes a mailbox (visible in Outlook) and a SharePoint site, and can be linked to new Team in MS Teams. Teams channel chats are stored in a hidden folder in the Group mailbox. Any documents and records are stored in the ‘Files’ tab of the channel, which surfaces the default ‘Documents’ library in the connected SharePoint site.
If the creation of Teams is allowed from the MS Teams application, every new Team creates a Microsoft Group (with the same name) and a SharePoint site (with the same name), however the mailbox (with the hidden folder for channel chats) is not visible from Outlook.
(The exception here are private channels; if these are allowed: (a) the chat content is stored in the Exchange mailbox of the each participant, and (b) a new SharePoint site is created for the ‘Files’.
The relationship between the content created by the Group is most obviously visible from the ‘Activity’ web part of the SharePoint site of the Group as can be seen in the screenshot below. This shows (right to left), an original incoming email from Outlook in the Group’s mailbox, the copy saved to the SharePoint document library, and the Word document reply. The specific context of the record (= the ‘file’) – ‘Correspondence 2020’ – is defined by the document library.
What about records in 1:1 Teams chat
As with OneDrive, Teams 1:1 chat should not be used to create or capture records, but may be used as a ‘working’ space.
However, ‘should’ and ‘reality’ can be different things. There are two ways to address this:
Explictly, through communication to end-users. Make it clear that Teams 1:1 chat and OneDrive are NOT to be used to create or capture records. Applying short-term retention policies to this content may assist with reducing (or increasing) this risk.
Implicitly, through monitoring and retention policies. Apply longer-term retention policies to the content and use Content Search/eDiscovery to look for content that may be records. Additionally, review the content of the OneDrive of departed staff and ensure that any records are kept.
Implications for managing records
The implications for collating, grouping and aggregating records in Microsoft 365 are as follows.
SharePoint document libraries will continue to be the primary aggregation for managing corporate records, including emails copied from Outlook.
Organisations should establish an architecture model for SharePoint sites that are used to manage records. The model may include a mix of the following: (a) sites mapped to business functions with libraries mapped to business activities and retention classes, (b) entire sites used to create and capture records relating to a single activity, where the entire site is mapped to a retention class, and (c) MS Groups (and Teams) with an associated SharePoint site, where the Group (mailbox/SharePoint site) is subject to a single retention class (and the Team channel chat also).
More effort, in terms of site/library set up, metadata, access controls, retention and end-of-retention process is likely to be required for the management of high-level, high-risk and permanent records.
Personal mailboxes in Exchange will continue to exist as a form of aggregation, and consideration should be given to having different retention policies for different ‘types’ of mailbox, to ensure that any email that could be a record is not deleted too quickly.
Addendum – Other options that collate, group and aggregate content in Microsoft 365
As noted earlier, all of the content created or captured in Microsoft 365 is stored in the backend Azure substrate. Consequently, it is possible to search across all or part of that content to find related information and, where required, export it to a different location.
The global Content Search is accessed from the Compliance portal and access requires elevated privileges – Global Admin or Compliance Admin.
Searches are created as cases and are based on keywords, conditions (such as ‘Sender’ for emails), and locations – all or specific. When a new content search is created or run, the Global Admins are alerted, providing a form of oversight in addition to audit logs.
While content searches find content is related to the search parameters, and legal holds can then be applied to that content, they do not create any form of aggregation in a recordkeeping sense.
The Graph, Delve, Discovery
Microsoft describe the Graph as being ‘the gateway to data and intelligence in Microsoft 365 [that can be used via the Microsoft Graph API] to access the tremendous amount of data in Microsoft 365, Windows 10, and Enterprise Mobility + Security’ and ‘… build apps that support scenarios spanning across productivity, collaboration, education, people and workplace intelligence, and much more. (Source ‘Overview of Microsoft Graph‘)
The Graph is commonly represented in diagrams similar to the one below.
Most end-users will encounter the Graph through either Delve or the Discover option in both the office.com portal and their OneDrive for Business accounts.
It is not uncommon for end-users to express surprise at the content (that they have access to) that is presented. Commonly this will show documents that a colleague is working on, or connections between people. Disabling Delve does not fix permissions; if a person has access to a document that appears in Delve, they will be able to search for it and find it that way.
Over time, the Graph can also provide other information based on the relationships or ‘signals’ it finds between all the different content in Microsoft 365.
While the Graph can present groups of records that have some relationship to the end-user, it does not aggregate those records or maintain a single consistent view. However, the Graph powers the new Project Cortex that does do something similar.
Project Cortex was announced by Microsoft in April 2019. To quote the announcement, Project Cortex:
Uses advanced AI to deliver insights and expertise in the apps you use every day, to harness collective knowledge and to empower people and teams to learn, upskill and innovate faster.
Uses AI to reason over content across teams and systems, recognizing content types, extracting important information, and automatically organizing content into shared topics like projects, products, processes and customers. Cortex then creates a knowledge network based on relationships among topics, content, and people.
From a recordkeeping aggregation point of view, a core functionality of Project Cortex is its ability to create ‘topic cards’ based on the rich metadata that makes up all the content in Microsoft 365. Again to quote the announcement:
Project Cortex securely collects content that is created and shared every day in Microsoft 365—including files, conversations, recorded meetings and video—and it categorizes the content based on its type, and tags it with extracted metadata.
AI then applies advanced topic mining logic—whether its content contained in Microsoft 365 or connected from external systems—to identify topics and relate content to those topics.
Topics can reflect any knowledge that’s important, including customers, products, projects, policies and procedures. Technically, AI is creating knowledge entities, a new object class, in the Microsoft Graph. The relationships between those topics—those knowledge entities—and the experiences that connect this knowledge with people creates your knowledge network.
Topic cards – or ‘knowledge entities’ – are a form of AI-generated aggregation.
However, topic cards will only present information that an end-user has access to and so the nirvana of presenting emails or Teams 1:1 chats in these cards as a form of aggregation for recordkeeping purposes is not likely to be realised through Project Cortex.
In the article, Tony describes where and how the chat component of MS Teams is stored and how this might affect eDiscovery.
He also makes the important point that, while it may be possible ‘… to backup Teams by copying the compliance records in an Exchange Online backup … you’ll never be able to restore those items into Teams.’ In other words, it is better to leave the data where it was created – in MS Teams. The post explains why this is the case.
This post draws on the article to describe the factors involving in managing the chat element of Teams as records. It notes that, while is is technically possible to export chat messages (in various ways), it may be much better from a recordkeeping point of view to leave them where they are and subject them to a retention policy.
Two key reasons for leaving chat messages in place are: (a) chat messages are dynamic and may not always be a static ‘thread’, and (b) the chat messages exported from Exchange may not contain the full content of the message.
What is a Teams chat?
A Teams chat consists of one or more electronic messages with at least two participants – a sender and a receiver.
There are two types of chat message in MS Chat:
One-to-one/one-to-many ‘chat’ (top icon above).
Channel-based Teams chat (second icon above). Teams chat is visible to all members of the Team. Within channel-based chats, a person may create a private channel which is visible only the person who created the private channel and any participants.
Messages created in both options could be regarded as records because they may contain evidence of business activity.
However, one-to-one chats have no logical subject or grouping. Only the chat messages in Team channel chat are connected through the context of the Team/channel.
Where and how are chat messages stored?
The following is a summary from Tony Redmond’s article.
Chat messages are stored directly in the backend Azure Cosmos DB (part of the so-called Microsoft 365 ‘substrate’). The version in the database is the complete version of the chat message.
The messages are then copied, less some content elements (for example: reactions, audio records, code snippets), to a hidden folder in either (a) end-user mailboxes for one-to-one chat and private channel chats, and (b) M365 Group mailboxes for channel chat.
Most export options, including the export option in Content Search and eDiscovery, draw their content from the mailbox version of the message. This has potential implications for the completeness of the chat message as a record.
Additionally, any export can only be a ‘point in time’ record unless there is absolute certainty that all chat on a given subject have ceased.
Implications for records managers
In addition to the concerns about a chat message (or exports of them) being complete, there are (at least) two other points relating to the management of chat messages as records in MS Teams:
Knowing if chat messages on any given subject exist.
Applying an appropriate retention policy.
Both of these points are discussed below.
The primary way to locate content on any given subject across Microsoft 365 is via the Content Search option in the Compliance portal. Access to the Content Search option is likely to be restricted. So, if records managers do not have access, they will need to ask the Global Administrators to conduct a search.
Searches can be configured to find content in any or all of the following locations:
Users, Groups, Teams
Office 365 group email
Skype for Business
Teams messages [the copy in the mailbox]
Office 365 group sites
Exchange public folders
Note that content search only works on the copies of the items in the Exchange mailboxes, not the backend Teams database. Accordingly, there is some potential for it to not find some content.
Both the mailbox content and the content discovered by the search can be exported. Teams chat messages can be exported as individual items or as a PST – but note that these message may exclude the elements as described in Tony’s article.
The problem with exporting the content either this way or via other export options (such as described in this post ‘How to export MS Teams chat to html (for backup)‘ (using the Microsoft Graph API) is that it creates a single ‘point in time’ copy; additional content could be added at any time and, if the chats were subject to a retention policy, they may already be deleted.
Managing chat messages ‘in place’ as records
As any export only creates a ‘point in time’ version, it makes more sense from a recordkeeping point of view to leave the chat messages where they are and apply one or more retention policies to ensure the records are preserved.
Ideally, organisations that may create or capture records on a given subject will have taken the time to establish a way for users to do this, including through the creation of a dedicated Microsoft 365 Group with an associated SharePoint site and Team in MS Teams.
For example, if there is a requirement to store all records relating to COVID-19, it would make sense (at the very least) to create a Microsoft 365 Group with that name; this will create: (a) a linked mailbox accessible by all members of the Group, (b) a SharePoint site with the same name, and (c) a Team in MS Teams. All of the content – emails, documents, chat, is linked via the same (subject) Group.
This model makes it easier to aggregate ‘like’ information and apply a single retention policy. It assumes there is (or will be) some degree of control over the creation of Teams (or very good communication to users) to prevent the creation of random Teams, Groups and SharePoint sites – AND to ensure that end-users chat about a given subject within a Team channel, not in one-to-one chat.
What retention period should be applied to chat messages?
The retention period applied to either one-to-one or Team channel messages will depend largely on the organisation’s business or regulatory requirements to keep records. There are two potential models.
The simplest model is to have a single retention policy for one-to-one chats, and a separate retention policy for all Teams channel chats.
As one-to-one chats are stored in the mailboxes of chat participants, it makes sense to retain the chat content for as long as the mailboxes. However, some organisations may seek to minimise the use of chat and have a much reduced retention period – even as little as a few days.
The creation and application of retention policies to Teams channel chat may require additional considerations. For example:
As every Team is based on a Microsoft Group that has its own SharePoint site, it is probably a good idea to establish Teams based on subjects that logically map to a retention class. For example, if ‘customer correspondence’ needs to be kept for a minimum 5 years, and there is a Group/SharePoint site/Team for that subject, then all the content should have the same retention policy – although the Group mailbox and SharePoint site may have a policy applied to the Group, with a separate (but same retention period) applied to the Team.
There may be a number of Teams that contain trivial content that does not need to be retained as records. These Teams could be subject to a specific implicit policy that deletes content after a given period – say 3 years.
In all cases, there is a requirement to plan for retention for records across all the Microsoft 365 workloads.
What happens to chat messages at the end of a retention period?
At the end of a Microsoft 365 retention policy period, both the mailbox version and the database version of the Teams chat message are deleted. To paraphrase Tony’s article, the Exchange Managed Folder Assistant removes expired records from mailboxes. Those deletions are synchronized back to Teams, which then removes the real messages from the backend database.
No record is kept of this deletion action except in the audit logs. Accordingly, if there is a requirement to keep a record of what was destroyed, this will need to be factored in to whatever retention policy is created.
The ability to manage records ‘in place’ in SharePoint has existed since around 2013. But this is not the same thing as leaving records where they were created or captured and managing them there – ‘in place’. This post explains the difference between the two ‘in place’ options. In brief: The Microsoft ‘in place’ model […]
SharePoint Online document libraries include the option to move content to a different location. But what happens to the metadata when this content is moved? This post explains what happens to the metadata when different types of content stored in a SharePoint Online document library is moved (not copied) using the ‘Move to‘ option to […]
A common question asked by many organisations is whether Microsoft 365 (M365) retention policies – labels in particular – can be applied to folders in SharePoint document libraries so the content in those folders will have the same label. The quick answer is yes, but it is a manual process and – for all its […]
Office 365 is sometimes referred to as an ‘ecosystem’. In theory this means that records could be stored anywhere across that ecosystem.
Unlike the ‘old’ on-premise world of standalone servers for each Microsoft application (Exchange, SharePoint, Skype) – and where specific retention policies could apply (including the Exchange Messaging Records Management MRM policy), the various elements that make up Office 365 are interconnected.
The most obvious example of this interconnectivity is Microsoft Teams which stores chat content in Exchange and provides access to content stored in both SharePoint (primarily the SharePoint site of the linked Office 365 Group) and OneDrive, and has links to other elements such as Planner.
Records continue to be created and kept in the various applications but retention policies are set centrally and can apply to any or all of the content across the ecosystem.
Managing records in Office 365, and applying retention rules to those records, requires an understanding of at least the key parts of the ecosystem – Exchange, Teams, SharePoint and OneDrive and how they interrelate, and from there establishing a plan for the implementation of retention.
What types of records are created in Office 365?
Records are defined as ‘evidence of business activity’ and are often associated with some form of metadata.
Evidence of business activity is an overarching term that can include:
Documents and notebooks (in the sense of text on a page)
Plans, including both project plans and architectural plans and diagrams
Images/photographs and video
Chat and/or messages
Conversations (audio and/or video based)
Social media posts
All digital records contain some form of metadata, usually displayed as ‘Properties’.
Where are the records stored in Office 365?
Most records created organisations using Office 365 are likely to be created or stored in the following parts of the ecosystem:
Exchange/Outlook – for emails and calendars.
SharePoint and OneDrive – for documents and notebooks (in the sense of text on a page), plans, images/photographs and video.
Stream – for audio and video recordings.
MS Teams – for chat and/or messages, conversations (audio and/or video based). Note that 1:1 chats are stored in a hidden folder of the Exchange mailbox of the end-user/s participating in the chat, while Teams channel chat is stored in a hidden folder of the linked Office 365 Group mailbox.
Yammer – for (internal) social media posts.
It is also possible to import and archive certain external content such as Twitter tweets and Facebook content in Office 365.
The diagram below provides a overview of the main Office 365 applications and locations where records are created or stored. Under SharePoint, the term ‘Sites’ refers to all types of SharePoint sites, including those associated with Office 365 Groups. Libraries are shown separately because of the potential to apply a retention policy to a library – see below.
Note also that this diagram does not include network file shares (NFS) as the assumption is made that (a) NFS content will be migrated to SharePoint and the NFS made read only, and (b) all new content that would previously have been stored on the NFS is instead saved either to OneDrive for Business (for ‘personal’ or working documents) or SharePoint only.
Creating a plan to manage records retention across Office 365
In previous posts I have recommended that organisations implementing Office 365 have the following:
A basic architecture design model for SharePoint sites, including SharePoint sites linked with Office 365 Groups (and Teams in MS Teams).
A plan for creating and applying retention policies across the ecosystem.
Because SharePoint is the most likely location for records to be stored (aside from Exchange mailboxes and OneDrive accounts), there should be at least one retention policy for every SharePoint site (or group of sites), as well as policies for specific document libraries if the retention for the content in those libraries may be different from the retention on the overall site.
For example, a ‘Management’ site may contain a range of general content as well as specific content that needs to be retained for longer.
The site can be covered by a single implicit retention policy of (say) 7 years. This policy will delete content in the background, based on date created or data modified.
The document library where specific types of records with longer or different retention requirements are stored may have one or more explicit label-based policies applied to those libraries. This content will be retained while the rest of the site content is deleted via the first policy.
Structure of a retention plan for records in Office 365
A basic plan for creating and applying retention policies might look something like the following:
User mailboxes – one ‘general’ (implicit) retention policy for all mailboxes (say, 7 years after creation) and another more specific retention policy for specific mailboxes that require longer retention.
SharePoint sites – multiple (implicit) retention policies targeting one or more sites.
SharePoint libraries – multiple (explicit) label-based retention policies that are applied manually. These policies will usually a retention policy that is longer than any implicit retention policy as any implicit site policy will prevent the deletion of content before it reaches the end of that retention period.
Office 365 Groups (includes the associated mailbox and SharePoint site) – one ‘general’ (implicit) retention policy. See also below.
Teams channel chat – one ‘general’ (implicit) retention policy. Note that this content is stored in a special folder of the Office 365 Group mailbox.
1:1 chat – one ‘general’ (implicit) retention policy. This content is stored in a special folder of the participant mailboxes.
OneDrive documents – one ‘general’ (implicit) retention policy for all ODfB accounts, plus the configuration of retention after the account is inactive.
At a high level, the retention policy plan might look something like the following – ‘implicit’ policies are shown in yellow, SharePoint document libraries may be subject to ‘explicit’, label-based policies. The ‘+7 years’ for OneDrive relates to inactive accounts, a setting set in the OneDrive Admin portal.
To retain content for a Microsoft 365 group, you need to use the Microsoft 365 groups location. Even though an Microsoft 365 group has an Exchange mailbox, a retention policy that includes the entire Exchange location won’t include content in Microsoft 365 group mailboxes. A retention policy applied to an Microsoft 365 group includes both the group mailbox and site. A retention policy applied to an Microsoft 365 group protects the resources created by an Microsoft 365 group, which would include Microsoft Teams.
The actual plan should contain more detail and included as part of other recordkeeping documentation (perhaps stored on a ‘Records Management’ SharePoint site). The plan should include details about (a) where the policies have been applied and (b) the expected outcomes or actions for the policies, including automatic deletion or disposition review (for document libraries).
Keep in mind that, unless the organisation decides to acquire this option, there is no default backup for content in Office 365 – once a record had been deleted, it is gone forever and there may be no record of this beyond 90 days.
On 27 March 2020 I asked, via Twitter, whether organisations that rolled out MS Teams will wonder in the future who created all the random (and randomly-named) SharePoint sites.
The reason for this question was because many organisations, scrambling to establish ways for staff to work from home, decided to make use of MS Teams in their (often newly implemented) Office 365 suite of apps.
I have seen multiple organisations since late 2019 ask ‘who created all those SharePoint sites?’ when they reviewed the list. The current COVID-19 work-from-home situation will only make this situation ‘worse’ and, without effective oversight or controls, result in the creation of multiple uncontrolled SharePoint sites.
Unlike other products like Zoom, Whatsapp, Facetime and Skype, however, MS Teams is not a standalone product, but a core element in the Microsoft Office 365 ecosystem.
The key point is this – every Team in MS Teams has a linked SharePoint site (and an Exchange mailbox, where all the chat content is stored). You can’t disable these options.
What happens if you create a Team in MS Teams?
The good thing about the one-to-one chat element of MS Teams is that it’s relatively intuitive and easy to use, including on the mobile app. You only need to tell users it’s like Skype or Whatsapp, but for internal user only, and most pick it up quickly.
The Teams part of MS Teams is not quite as intuitive, but early adopters generally understand the basic concepts – that a Team has members, and you can have multiple chat channels for each Team.
Once end-users understand how a Team works (and this can take some time because one-to-one chat can include multiple people), they might notice this option at the bottom left of the app:
Creating a new team sounds like a great idea, so end-users may try:
My guess is that end-users are more likely to want to ‘build a team from scratch’ as shown below, because the second option doesn’t really make sense.
There is a good chance they will want the Team to be ‘Private’, although may not fully understand what this means. A Public Team sounds like a Yammer Group (or Community).
So far, so good, the end-user can give the Team any name they like:
At the bottom of the naming screen is the option to ‘Create’. The end-user is then invited to add members to their new Team. This seems a fairly obvious step, and they can add whoever they want. New members are by default ‘Members’ but they can be changed to ‘Owners’ if necessary. There is no control over this process.
The new team now appears on the left-hand menu of MS Teams:
The new team opens at the default ‘General’ channel.
On the main part of the Team, the following options are offered:
Along the top, ‘Posts’, ‘Files’, ‘Wiki’ and a + to add more applications. (Hint – the ‘Files’ option points to the SharePoint site that has been created behind the scenes).
Across the middle, three options to ‘Add more people’, ‘Create more channels’ and ‘Open the FAQ’
At the bottom, the option to ‘Start a new conversation’ with various other options including the ‘Meet now’ video option.
The end-user can now get on with chatting, sharing files, and adding apps to do other things.
But what else has happened?
As noted above, the ‘Files’ tab in the General channel gives a clue to the existence of the connected SharePoint site. End-users may not care terribly much about this, for them it provides the option to create, upload, share and collaborate on files.
A new Office 365 Group is created
But before we get to the SharePoint site, it’s important to understand the one-to-one relationship between a Team in MS Teams and an Office 365 Group. If you do not know what an Office 365 Group is, please read this Microsoft guidance on Office 365 Groups.
In very simple terms:
Every new Team in MS Teams creates a new Office 365 Group.
The Owner of the Office 365 Group is the Owner of the team; the members of the Group are the Members of the team, as added by the person who created the Team.
The new Office 365 Group appears in the list of Groups in the Office 365 Admin portal, as shown below. Access to this part of the Admin portal is normally restricted to Global Admins (who would normally be responsible for creating other types of AD Groups, such as Security Groups and Distribution Lists.
A new Exchange mailbox has been created
Note that the process has also created an Exchange mailbox with a Group email address. The new Exchange mailbox will now appear in the Outlook client of everyone in the Team – something they are unlikely to notice.
As noted above, all the chat messages in the Team are stored in a hidden folder in the Exchange mailbox for the Team.
A new SharePoint site has been created
If we go across to the SharePoint Admin portal, which is normally restricted to Global Admins and SharePoint Admins, we can see that a new SharePoint site has been created, and is owned by the ‘Group owners’.
The SharePoint Admin has had no involvement in the creation, naming, or structure of this new site. And, just to add another factor, the SharePoint Admin cannot access the site – see below.
The Team owner may not realise it, but they now have a SharePoint site. The new site’s ‘Documents’ library appears in the ‘Files’ tab as shown below.
And, just to add a confusing element, the site includes the invitation (at the bottom left) to create a new Team!
As noted above the SharePoint Admin can ‘see’ that this site exists in the list of sites but cannot actually access it. The Global Admin, on the other hand, can access it.
So the person responsible for managing SharePoint across the organisation cannot access the SharePoint site, which is not a good thing from an information governance point of view.
The reason they cannot access the site is because they were not added to the Site Collection Admin Group when the site was created. And, just to make it a bit more confusing, the ‘Users and Permissions’ section of Site Settings, where the ‘Site collection administrators’ section is found (see screenshot below), does not appear in Office 365 Group-based SharePoint sites.
So, how does the SharePoint Admin get access to this site to configure and manage it? There are two ways:
The Global Admin can go to /_layouts/15/mngsiteadmin.aspx (after the site name URL) and add them (or a Security Group with them in it) there.
The SharePoint Admin can click on the site details in the SharePoint admin portal and add him/herself as an Owner. This puts them in the Site Collection Admin section along with the Group Owner.
This post began with a simple question – if organisations allow end-users to create Teams to work from home, how will they manage all the SharePoint sites that are created through the process described above?
There is no one answer to this question but it’s worth understanding exactly what happens – and what else is created (including Planner) – when a Team is created. Organisations seem to go one of two ways:
Let end users create Teams and deal with the consequences later, including attempts at auto-classification and retention policy application across the various elements of the new Office 365 Group – mailbox, SharePoint site, Team chat. This is the Microsoft default and the preference of many organisations that are don’t have compliance issues or can accept the risks of uncontrolled information stores.
Control the creation of Teams, but make any controlled process as easy as possible for end-users to keep them working quickly, and manage the content in mailboxes, SharePoint and Teams proactively. While not the preferred option, it will help with the management of corporate information down the track.