Knowledge Management in Office 365

A few articles in the past few weeks, and some internal discussions, prompted some thinking around how Office 365 can support knowledge management (KM) – however that may be defined.

What is Knowledge Management?

According to many knowledge management sources online, knowledge management appeared around 1990, and paralleled the rise of document management. Both appear to have arisen as computers appeared (from the mid 1980s) and digital ways of capturing and managing information took hold, and records management was still primarily focused on the management of paper records.

An early (1994) definition for the term ‘knowledge management’ suggested that it was ‘… the process of capturing, distributing, and effectively using knowledge’ (Davenport, 1994. Koenig, 2012)

Bryant Duhon expanded on this somewhat imprecise definition in his 1998 article ‘It’s All in our Heads’ (my emphasis):

‘Knowledge management is a discipline that promotes an integrated approach to identifying, capturing, evaluating, retrieving, and sharing all of an enterprise’s information assets. These assets may include databases, documents, policies, procedures, and previously un-captured expertise and experience in individual workers.’ (Duhon, 1998)

A key element was capturing the knowledge acquired by individuals.

Koenig (2012) noted that ‘Perhaps the most central thrust in KM is to capture and make available, so it can be used by others in the organization, the information and knowledge that is in people’s heads as it were, and that has never been explicitly set down.’

Explicit/implicit versus tacit knowledge

Generally speaking, there is a difference between explicit and implicit knowledge, the information that is recorded, and ‘the information and knowledge that is in people’s heads’ (and walks out doors when people leave).

The latter is defined generally as tacit knowledge. That is, information that is ‘understood or implied, without being stated’, from the Latin tacitus, the past participle of tacere ‘be silent’. (https://en.oxforddictionaries.com/definition/tacit)

I have worked with the issue of how to access and capture the knowledge in the heads of departing employees since around 1984, when I was first made aware that the departure of some very senior and/or long-term staff meant that we would lose access to the information they knew, gained not only from learned knowledge but also in many cases from many decades of personal experience.

At the time it was not my responsibility to worry about it, but I saw attempts to conduct interviews and document procedures and processes with departing (or already departed) employees.

This pre-digital era activity stuck in my head – was interviewing the departed employees the only way to get this information out of their heads?

(As a side note I learned that it was important to interview and talk to my ageing parents and their siblings about their memories and experiences before those memories were lost forever).

Enter the computer age

I consider myself lucky to have been witness over a generation to the change in working practices from paper to digital.

The start of the digital era from the mid 1980s and ubiquitous access to computers on desktops, person to person emails, network file shares and personal folders created another related dilemma – even if the information was created (or captured) by a user, how could it be accessed?

Users were encouraged to put this information in repositories – mostly document management systems – but the fact that email and information on file shares were stored in different servers meant that unless users would actively move emails to a document management system, that information remained hidden away.

What was needed was a way for users to create and store information – emails, documents – wherever they wanted to put it, and for that information to be accessible, restricted only by relevant security controls.

The only systems that seemed to really do this effectively were eDiscovery tools. Perhaps this was not surprising, as the survival (and financial viability) of a company might depend on the ability to find the information that was required.

The rise of smart phones and ubiquitous, always-on, digital communication within the past 10 years has only added to the types of knowledge available and the methods used to capture it.

In my opinion, traditional recordkeeping practices have not kept up and often remain rooted in the idea that knowledge can be stored in a single location or container. How does one capture instant messages sent via encrypted messaging services in a records container?

Microsoft Graph

Microsoft introduced the Microsoft Graph in 2015. The image below demonstrates how the Graph connects content created and stored through the Office 365 (and connected) environment/s.

The image above should resonate with most people who work in an office. We send emails, create documents or data, set tasks, make appointments, attend and record meetings, have digital conversations, send messages, connect with colleagues, maintaining personal profiles.

The Microsoft Graph collects and analyses this information and presents it to users based on their context. According to Microsoft:

‘Microsoft Graph is made up of resources connected by relationships. For example, a user can be connected to a group through a member of relationship, and to another user through a manager relationship. (The Graph) can traverse these relationships to access these connected resources and perform actions on them through the API. You can also get valuable insights and intelligence about the data from Microsoft Graph. For example, you can get the popular files trending around a particular user, or get the most relevant people around a user.’

(Source for image and text: https://developer.microsoft.com/en-us/graph/docs)

According to Tony Redmond, Microsoft Graph’s REST-based APIs provide ‘… a common access approach to all manner of Office 365 data from Exchange and SharePoint to Teams and Planner’. The Graph Explorer, a newly introduced user interface, extends the ability to access information, wherever it lives. (https://developer.microsoft.com/en-us/graph/graph-explorer)

How does a person access this knowledge?

In my opinion, two key points about tacit knowledge are that:

• It can be captured easily, just as other digital applications capture information about us, including by what we click on or search for.
• It can be accessed without a person necessarily having to search for it.

Most of us by now are familiar with the way Facebook, LinkedIn, eBay, Amazon and so on capture information about our interests and present suggestions for what we might like to do next. It does this by understanding our context

Organisational knowledge management should be the same. Users should go about their business using the various digital applications available to them and other users should be able to see that information or knowledge because they have an interest in the same subject matter, or need to know it to do their work.

Users should be presented with information (subject to any security restrictions) because it relates to their work context or interests. They should not have to go looking for knowledge (although that is an option, just as finding a friend in Facebook is an option), knowledge should come to them.

How does Office 365 do this?

Most Office 365 enterprise or business users will have one or two ways to access this information:

• Delve (may require a higher licence such as E3 for enterprise clients)
• The One Drive for Business ‘Discover’ option.

The ‘Discover’ option allows a user to explore further, to see what others are working on. The response I get to Discover is both positive and slightly startled – the latter because it will be possible to know what others are actually doing.

Why is this important?

The ability to access and ‘harness’ collective knowledge in this way is essential to modern day workplaces.

To quote Microsoft:

‘As the pace of work accelerates, it’s more important than ever that you tap into the collective knowledge of your organisation to find answers, inform decision making, re-purpose successes and learn from lessons of the past’. (Moneypenny, 2017)

Serendipitous discovery

In his 2007 book ‘Everything Is Miscellaneous: The Power of the New Digital Disorder’, David Weinberger spoke about three types of order:

• The first order is the order of physical things, like how books are lined up on shelves in a library.
• The second order is the catalogue order. A catalogue typically refers to a physical order; it is still physical, but one can make several catalogs of the same physical order. Weinberger’s prime example is the card catalog of libraries.
• The third order of order is the digital order, where there is no limit to the number of possible orderings. The digital order frees itself from physical reality, and in it, everything can be connected and related to everything else: Everything is miscellaneous.

The phrase ‘herding cats’ always comes to mind in relation to digital information. It resists order or compartmentalisation.

Further, your order is not my order, my way of browsing or searching may not correspond with your logic for storing or describing it (especially on network file shares!).

The internet pioneered serendipitous discovery. It is now completely taken for granted when, as noted above, we are are offered suggested friends in Facebook, jobs in LinkedIn, purchases on eBay and so on. We are presented this information because the application has collected information about what we clicked on, what jobs we do (or did), who our friends are, and what we like to search for.

The idea that our work environment can do the same thing and present information automatically based on our context (information finds us) is sometimes surprising for people used to the second order of things.

 

Davenport, Thomas H. (1994), Saving IT’s Soul: Human Centered Information Management.  Harvard Business Review,  March-April, 72 (2)pp. 119-131. Duhon, Bryant (1998), It’s All in our Heads. Inform, September, 12 (8). Quoted in Koenig (2012).

Duhon, Bryant (1998), It’s All in our Heads. Inform, September, 12 (8), pp. 8-13.

Koenig, Michael (4 May 2012), What is KM? Knowledge Management Explained, http://www.kmworld.com/Articles/Editorial/What-Is-…/What-is-KM-Knowledge-Management-Explained-82405.aspx, accessed 21 July 2017

Naomi Moneypenny (17 May 2017), Harnessing Collective Knowledge with SharePoint and Yammer, https://techcommunity.microsoft.com/t5/SharePoint-Blog/Harnessing-Collective-Knowledge-with-SharePoint-and-Yammer/ba-p/70164, accessed 21 July 2017

Redmond, Tony (20 July 2017), Exploring Office 365 with the Graph Explorer, https://www.petri.com/exploring-office-365-graph-explorer, accessed 21 July 2017

Weinberger, David, (2007) ‘Everything Is Miscellaneous: The Power of the New Digital Disorder’

Migrating to SharePoint Online – Early Learning with Modern and Communication sites

We have had a ‘controlled’ on-premise SharePoint environment since early 2012, starting with SharePoint 2010 and moving to SharePoint 2013 two and a half years ago.

‘Controlled’ in this sense means that users cannot create their own sites or sub-sites and site owners are responsible for managing their sites, including creating libraries and lists and managing page content.

Governance model

Our governance model, originally based on a Microsoft governance model, provided a good balance between (a) the need for excessive IT control and effort (there’s only two of us managing the whole environment), and (b) the potential for a feral environment when site creation gets out of hand.

An early decision was made to use multiple web applications for teams, projects, publishing sites, the intranet, and ‘apps’ (a handful of ‘purpose-built’ sites).

Another key governance decision made in 2012 was to keep the environment as much as possible ‘out of the box’ (OOTB) and avoid customization. By doing this we aimed to ensure that upgrades would be relatively straightforward. This didn’t prevent site owners from being fairly creative with their sites, especially site pages.

Preparing for SharePoint Online

If you are planning to move to Office 365 and SharePoint Online (SPO), you should understand how existing sites will migrate to the new platform, especially with the release of new ‘modern’ SharePoint sites and more recently ‘communication’ sites.

One of the first considerations is the architecture of the new SPO sites. These use only name-based paths – ‘/sites’ or ‘/teams’. If you have (like we did) multiple web applications or complex hierarchies of sites, you will need to consider how these will map to the new architecture.

For example:

• Sites in multiple web applications will need to be mapped to either /teams or /sites. For example, one of our web applications was /projects; these will be migrated to /teams and all new project sites will be Office 365 Group based, with a ‘PRJ’ prefix.
• Sites in complex hierarchies can, potentially, continue in SPO, but the SPO model is more suited to multiple, separate sites at the same level. A hierarchy or organisational structure may change and this could cause problems for moving content between sites. Having said that, all SharePoint sites site under the top level https://(organisation name).sharepoint.com ‘root’ site, followed by either /sites or /teams – e.g., https://(organisation name).sharepoint.com/sites/example.

Migrating site content

Most SharePoint site content consists of a combination of pages, libraries and lists, and the data stored in each.

Each has a new counterpart in SPO and you need to understood these in advance of migrating. Note however that Microsoft have continued the ‘classic’ look in SPO so that the pages look the same (for the time being); libraries and lists on the other hand are converted immediately to the new ‘modern’ style on migration.

Libraries and Lists

The most visible change to libraries and lists is the removal of the familiar ribbon menu and its replacement with a much simpler and user-friendly version, one that is almost identical with the new ‘ribbon’ that appears in OneDrive for Business.

The main library ribbon is as follows:

The ribbon changes when a document is selected, in this case a Word document:

The new ‘ribbon’ was designed to make it as easy as possible for users to add, edit and access content, including on mobile devices, focusing on the primary actions users need to perform:

• Add new content (including creating a new Office document from within the library, or a new folder or link)
• Edit content (including by using Office Online applications)
• Move and copy content
• Share content

The ribbon is minimalistic and expands with additional options with a document is selected. The following options are accessed by clicking the three-dot ‘ellipsis’ to the far right of the ribbon menu, or clicking on the ellipsis to the right of the document name:

• Copy to
• Rename
• Version history
• Alert me
• Manage by Alerts
• Check Out/In

‘Flow’ is a new option in both libraries and lists, replacing the older style library or list workflows (and possibly some simple SharePoint Designer workflows).

The primary consideration when moving to modern libraries and lists is change management. On a positive note, users who found the old ribbon menu just a bit too complex should find the new ribbon simple to use.

Library Settings and List Settings still remain and have the same look and feel; this option is now accessed from the gear/cog icon.

A new (or rather slightly modified) option for SPO users on the ribbon is the ability to synchronise (‘sync’) the SPO library selected with File Explorer. This option allows users to access SPO content from the familiar File Explorer view, although various library options such as check out/in are not available; the documents in File Explorer are copies.

• Note: Migrating to SPO provides the opportunity to ‘clean up’ libraries and lists, especially libraries without content.

Site pages

Perhaps one of the most challenging changes for SharePoint administrators and site owners or users will be the introduction of new ‘modern’ pages. This may be a challenge for organisations that have implemented or allowed site page customizations.

SharePoint Administrators need to make themselves familiar with the structure and layout of modern site pages well in advance of any planned migration, especially to understand how existing pages will migrate.

The main changes to site pages are the absence of the ribbon and completely new web parts. Instead of a ribbon, each new web part includes various editing options, outlined below.

The introduction of ‘communication’ sites in late June 2017 added to both the site type potential as well as the options for constructing a page. All of these changes make the new site pages mobile friendly.

Another key point to consider, in terms of site design, is whether sub-sites are really required.

New site page web parts

The new web parts are visible when any modern page is placed in edit mode; when you click on the page you will see the + option that allows you to add the required web part. This replaces the ‘App Part’ and ‘Web Part’ options under the SP2013 ribbon ‘INSERT’ option.

The new web parts are presented in three groups.

The first section offers the following web parts.

• Text. Allows formatted text to be insert in a defined area on the page. Similar in a way to the FORMAT TEXT options on the ribbon menu in SP2013, and also presenting text in a Content Editor Web Part. However, it only includes rich text (headings, formatting, but no tables or images).
• Image. Allows an image to be placed on the page, similar to SP2013 INSERT – Picture. No text can be added, and so if you need to place text and images together, you may end up with multiple text boxes with an image above or below.
• Document. Displays the first page of a document within a defined area. This may used as alternative to a table.
• Link. Allows a direct link to be provided to any other content. Similar to INSERT – Link in SP2013.
• Embed. Almost the same as the ‘Embed Code’ option in SP2013 INSERT ribbon menu, but note there are some limitations.
• Highlighted Content. Allows different types of content from the site or other locations to be displayed on the page. The content can be filtered and sorted, and various layout options are available. Type options are: Documents, Pages, News, Videos, Images, Events, Issues, Tasks, Links, Contacts, or All. As at the date of writing this post, the option to display the content from a List is still not available – but see below.

The next section offers various page layout options, similar to the Text Layout option under FORMAT TEXT.

• One column
• Two columns
• Three columns
• One-third left column
• One-third right column

The last section offers the following web parts.

• Bing maps. Displays a Bing map.
• Document library (preview). Presents an editable list view of documents.
• Events. Displays items created in the events list.
• Hero. Provides a way to highlight and link to content using two different designs: ‘topic’, which presents 1 – 5 tiles; ‘showcase’ which presents 1 – 5 layers. The tiles or layers both include the ability to add a photograph and a link to other content.
• Image gallery. Displays photographs from an image library.
• List (preview). Presents an editable list view of a list.
• News. Displays news that is created as news pages.
• Office 365 Video. To be deprecated in favour of Stream (see below). Presents a link to a video.
• People. Shows people from Active Directory.
• Power BI (preview).
• Quick chart. Displays a chart.
• Quick links. Displays links to other content.
• Site activity. Presents a tiled list of content that has been created recently on the site.
• Stream (preview). This will replace the option under SP2013 INSERT – Video.
• Yammer feed. Displays a Yammer group feed.

For more details on the new page options, see:
https://techcommunity.microsoft.com/t5/SharePoint-Blog/Reach-your-audience-via-SharePoint-communication-sites-in-Office/ba-p/70079

Considerations using the new modern pages

Aside from the overall page layout using the new web parts in modern pages, the key issues we have identified so far with migrating old site pages have been the following, none of which are possible in the OOTB modern site pages without (possibly) using the SharePoint Framework (see below):

• Content presented in tables, including images.
• Images with links, including image maps.
• Multicoloured text.
• Images embedded next to text.

If you have allowed extensive page editing or customisations, you may need to consider how to move away from this model.

Why are the page options now limited?

In a word – consistency, but also flexibility using the new SharePoint Framework (SPFx). Site Owners (and others) may have been able to create a range of page content in SP2013 or SP2010. Without central control, this could result in a range of user experiences which may in turn affect user take up. Consistency across SharePoint sites provides users with a familiar navigation model.

The need to access SharePoint on mobile devices also likely drove the requirement for consistency of content.

What are the other options?

The new SharePoint Framework (SPFx) offers the ability to create your own custom SharePoint web parts.

However, rather than use SPFx to re-create the web parts or options that no longer exist, it may be worth considering whether these ways of presenting information are still valid – for example, presenting information in a table on a page was a popular option, but was it the best way to present that content?

Office 365 – SharePoint Communication Sites

Microsoft released the new ‘Communication Sites’ into the SharePoint environment for First Release customers in late July 2017. The release of these new and eagerly anticipated site types underlined the need for a good SharePoint architecture, especially when moving from on-premise to online in Office 365.

What are Communication Sites?

To quote Microsoft, Commmunication Sites ‘… are perfect for internal cross-company campaigns, weekly and monthly reports or status updates, product launches, events and more.’ (Source: https://blogs.office.com/en-us/2017/06/27/sharepoint-communication-sites-begin-rollout-to-office-365-customers/)

But what are they and how do they fit into your SharePoint architecture? What the relationship between Communication Sites and other sites using the publishing features of SharePoint?

Communication sites are, essentially, a new type of online-only site with three different top-level site page designs:

• Topic. Use when you have ‘a lot of information to share, such as news, events and other content’.
• Showcase. Use when you want ‘to feature a product, team or event using photos or images’.
• Blank. Build your own.

Depending on the architecture of your current SharePoint environment, topic-based SharPoint sites have the ability to replace the top-level site of a publishing-based intranet site. The default layout of topic-based sites makes use of the ‘hero’ web part that presents information in several ’tiles’ on the screen as well as other web parts such as ‘news’, ‘events’, ‘documents’ and ‘contacts’. Multiple columns can be displayed on the page and various other options are possible, including by using the SharePoint Framework.

Showcase-based sites, on the other hand, allow you to promote and showcase parts of the organisation, events or products. The default layout also uses the hero web part that allows content to be displayed in one to five layers.

The blank design allows you to create your own site structure.

To quote Microsoft on the link above (which includes lots of screenshots), ‘When you create a page on a communication site, you can embed documents and video, and dynamically pull in real-time data from across Office 365, including documents from SharePoint, Power BI reports, Microsoft Stream videos and Yammer discussions. The resulting page is a rich and dynamic communication’.

How do you create Communication Sites?

Communication sites are created in the same (new) way as Office 365 Group-based sites, by clicking on the ‘Create Site’ option in the SharePoint portal (https://(your company).sharepoint.com/_layouts/15/sharepoint.aspx).

Clicking this option presents two options as shown above: (a) team sites and (b) communication sites. Only authorised users who can create O365 Groups can create a Group-based team site or a Communication site.

Creating a new Communication site using this option does not create an O365 Group, unlike a Group-based team site.

Note: The path for both new Group-based and Communication sites is set in the SharePoint Admin portal. In our experience most Group-based sites need to be created in the /teams name path, while Communication sites should be created in the /sites name path. It can take a little while (we found up to 20 minutes) for the changed option to appear in the SharePoint portal, ‘create site’ option.

When the ‘Communication Site’ option is selected, the authorised user must (a) select which design (topic, showcase, or blank) and (b) give the site a name (which becomes the URL address). We found it was very easy for a use not to select the correct site design because it appears on the left, whereas all the other options including the name appear on the right of the site creation process. The new site is created quickly after ‘Finish’ is selected – in a matter of minutes.

Note: The new sign designs are only available at the top level of the site. New sub-sites are standard sub-sites which, depending on your set up, are probably going to be ‘classic’ site pages with modern libraries and lists. The site pages can of course be easily swapped over for a new modern page, but these pages do not include (or do not seem to inherit) the same design options as on the top level topic and showcase based sites. There may be an architecture or design reason for this – see below.

Using Communication Sites

As noted above, Communication sites have two primary potential uses:

• Replacement for top level intranet sites that are usually built on sites with publishing features enabled
• New ‘showcase’ sites, that may also already exist as publishing sites

The meaning of ‘intranet’ in this context may vary, but in our context the intranet is a standard top-level site, with multiple sub-sites, with publishing features enabled and common organisation-wide centralised information such as news, organisational structure and information, and policies and forms. It may also include extensive customisation. Other types of ‘intranet’ might include:

• The top level in a hierarchy of team and publishing sites, all known as the ‘intranet’.
• Any other SharePoint site that is known as the ‘intranet’. This might include team sites.

Considerations when using Communication Sites

As noted above, the ‘topic’ and ‘showcase’ design elements of Communication sites are restricted to the top level site only. However, many ‘intranet’ sites include at least one level of sub-site. Therefore, careful consideration needs to be given to the architecture of the proposed ‘intranet’ if a decision is made to use Commmunication sites instead of traditional publishing sites for this purpose.

Communication sites include the following default elements:

• Top level site page, using the ‘hero’ web part that provides links to other information.
• Site pages (includes the top level page and any news pages)
• News (pages)
• Events (calendar)
• Documents (library)

Other apps that can be added to these sites include:

• Custom list
• Site mailbox

Organisations may also make use of the SharePoint Framework to add other types of content on the pages.

Clearly, this may limit the potential to use a Communication site to completely replace an existing multi-sub-site intranet.

The lesson that may be drawn from this is that Communication sites using the ‘topic’ design are not intended to be a complete replacement for a multi-sub-site intranet. The inference is that replacement intranets may actually be made up of multiple different sites.

A possible structure (based on a typical intranet site) might be made up of the following elements:

• Organisation ‘home site’ using the ‘topic’ design. This would typically be the first ‘go-to’ place for users to learn more about how the organisation works, the latest news, and policies and forms. It may also include multiple links to other applications or content. ‘Hero’ web part links may point to content within the site, or to other Communication sites (topic or showcase).
• A dedicated sub-site for policies and forms.
• News pages
• Multiple ‘showcase’ design sites for each organisational area or event, to promote their work, instead of using sub-sites from the main site to do this.
• Multiple sites under the ‘/teams’ (includes Group-based sites) and ‘/sites’ name paths.

How do you find anything?

A possible concern to separating elements of existing SharePoint sites into completely separate sites is finding the content; if the information forms part of the same site, it should be possible to find it relatively easily.

The simple answer to this is that the ‘Search’ option in SharePoint Online no longer points to the same site by default, and instead searches across all SharePoint content, regardless of its location.

Conclusions

Organisations that continue to host their SharePoint sites in on-premise servers will need to consider and plan how to migrate their sites, including their intranet, into the new SharePoint Online environment, with the following options:

• Team, publishing and other ‘traditional’ site types created via the SharePoint Admin portal, under the ‘/sites’ or ‘/teams’ paths.
• Office 365-Group based sites, created from the SharePoint Portal, which also creates a Group and all associated elements. Alternatively, O365 Groups created in the ‘Groups’ section of the Office 365 Admin portal, that create O365-linked SharePoint sites. The latter option is preferred to maintain naming conventions and restrict uncontrolled growth and inconsistent naming of both Groups and SharePoint sites.
• Communication sites, created from the SharePoint portal.

Traditional, multi-level intranets will almost certainly need to be discarded in favour of multi-site based intranet content, unless the organisation is prepared to use standard sub-site (modern) page layouts to present information to users.

Organisations that continue to want to have complex intranet sites may need to explore the SharePoint Framework and engage third-party vendors who can support this model.

Whichever option is selected, an important element not to lose sight of is the ability to access (and if necessary, add to or edit) content via a mobile device. The more complex the site, the harder it will be (without considerable extra cost) to present it on a mobile device.

 

 

 

 

Applying (new) Retention Policies to Office 365 Content

From time to time I’m asked about the way records retention policies ‘work’ in SharePoint. A common criticism has been that SharePoint’s retention model is based on applying retention policies to individual records (e.g., documents in a library or individual emails) rather than to aggregations of records, the most obvious of which is a document library.

The idea of storing and managing related records together in a single aggregation derives from the management of paper records – in files, boxes, and series. This model (of aggregations containing all records relating to a given subject) was largely replicated in electronic document management systems (EDMS – many of which were used to register paper files and boxes previously) when they appeared or were modified to manage digital records in the late 1990s.

In fact, many EDM systems did not actually manage records in an aggregation; the actual digital records were stored in a secure network file stored, and presented in the EDMS user interface though a common ‘file number’ (or similar) ID.

In any case, the ability to store all digital records on the same subject together in the one system (e.g., EDMS) was always hampered by the fact that (a) email and documents were created by different systems, (b) stored in different locations (servers), and (c) use of network file shares continued more or less unabated.

The increasing complexity and types of digital records underlines the difficulty of ever storing, let alone managing or applying retention and disposal actions, to them in a single aggregation.

Until recently, Microsoft’s retention and disposal options reflected the fact that applications used to create digital records stored them in different locations (servers) – Exchange and SharePoint. Retention policies targeted individual records stored in those applications, rather than aggregations.

In March 2017, Microsoft introduced a new, single central way to create and apply retention and disposal policies to most Office 365 content, wherever it was stored – Exchange, SharePoint, OneDrive for Business, Office 365 Groups, and Skype for Business.

This post:

• Summarizes the existing ‘out of the box’ retention and disposal options in SharePoint, but not Exchange (see my earlier post on this subject).
• Discusses issues with existing retention and disposal options in SharePoint.
• Describes how the new centrally-managed retention policies and labels can be applied to most content in Office 365.
• Discusses why applying retention policies to individual records rather than aggregations may be a better option in the digital world.

Records managers working in organisations that use Office 365 to manage records should familiarize themselves with the way these new retention policies work.

Note: The details in this post are based on the Australian recordkeeping context, which may be different from your specific location.

SharePoint out of the box (OOTB) retention and disposal options

Until recently, the only available OOTB options to apply retention and disposal actions to SharePoint were to:

• Apply an information management policy to an entire site via the Site Collection Settings. This option is suitable for short-lived sites such as project or closed, archived sites, but less suitable for long-lived team sites which might have a range of different content.
• Create a retention policy using the information management policy settings in Content Types. This option applies the policy to individual records. Content Types also include the ability to ‘transfer’ (actually copy) records after a defined period to another location, such as a Records Center.
• Use a folder-based information management policy. This option requires the default Content Type-based policy on a document library to be changed via Library Settings – Information Management Policy Settings, to Library and Folders.

Another option was to adopt a form of ‘retention in place’ and regard each library as a logical aggregation of records, the equivalent of a ‘file’, and manage retention and disposal manually or using PowerShell scripts to identify libraries for potential disposal based on the last modified date of the records. Some vendors have developed a similar model to manage retention policies on libraries using a central ‘console’.

Applying retention and disposal actions to individual records

Both the Content Type and folder-based options noted above apply the retention policy to individual records in the library, not the library (aggregation/container) as a whole.

That is, disposal was based on a time period after which each individual record was created, modified, or declared a record. The logic behind this model appears to be that a document library may store multiple record types each with different retention requirements. This may not be true for all document libraries, but it usually is for many.

Applying automated disposal actions on individual records (rather than an aggregation of records) is probably counter-intuitive for most records managers. The main concerns, from a recordkeeping (and possibly also archival) point of view are the absence of (a) a documented review and approval process before the records are destroyed, and (b) a metadata record of what was destroyed. That is, the records simple disappear from the document library, removing records that may would be relevant to the context of the original aggregation. This, of course, assumes that all records relating to the subject were stored in a single aggregation which, as noted above, may not always be the case.

Global Retention Policies and Labels in Office 365

In March 2017, Microsoft introduced two new ‘global’ retention options – retention policies and labels – to Office 365. The two options allow organisations to apply centrally set and apply retention policies to the same type of record, in whatever form and wherever they are stored – emails in Exchange, documents and lists in SharePoint, conversations (in Office 365 Groups and Skype)..

Examples of ‘types’ of information could include:

• Corporate records that must be kept for the life of the company.
• Financial records that need to be kept for 7 years.
• ‘Working records’ that could be deleted after a minimum period of time.
• Personnel records or staff files that had to be kept indefinitely.

As Tony Redmond noted in this recent article, these new retention policies build on the type of retention policies first released in Exchange 2010 using folder, system, personal and default tags. The article suggests that organisations that have applied Exchange retention policies may need to consider the impact of these new types of policies. In particular, the ability to move email to archive mailboxes is lost, replaced with a retention policy.

How Retention Policies work

Retention policies in Office 365 are created by authorized users (ideally, records managers) in the Retention section of the Security and Compliance Center.

Creating a new retention policy

Each policy has the following options: Name, Settings, Locations and Preservation Lock.

Name

The name of the retention policy should reflect the class name or number in the records retention schedules so that it can be easily identified and applied to content wherever it can be applied in Office 365 (see below for ‘Locations’).

Settings

The two Settings options are based on two questions:

• Do you want to retain the content? 
  • If ‘Yes, I want to retain it’ is selected, the choices are either ‘Forever’ or a configurable ‘n days/months/years’ (e.g. 7 years). The administrator must then decide if, once it reaches that point, the record should be deleted or not. If ‘Yes’ is selected, the content will be deleted from where it is currently stored as described in the next two points.
  • >>For SharePoint content there are two options when the retention period expires. (1) If the record has not been modified or deleted it will be deleted from the original library where it was stored, and then remain in the two-stage Recycle Bin for up to 90 days. (2) If the content has been modified or deleted, it is transferred to the hidden Preservation Hold library that is created when the retention policy is applied to a SharePoint site and deleted from that library. In this case, the administrator has only 7 days to recover the content before it is deleted permanently.
  • >>For Exchange content there are also two options. (1) If the item is modified or permanently deleted by the user during the retention period, the item is copied (if modified) or moved (if deleted) to the Recoverable Items folder. The retention policy process identifies and deletes items whose retention period has expired within 14 to 30 (configurable) days of the end of the retention period.  (2) If the item is not modified or deleted during the retention period, the same process runs on all folders in the mailbox and identifies items whose retention period has expired. These items are also permanently deleted within 14 to 30 days of the end of the retention period. (Note: If a user leaves the organization, and their mailbox is included in a retention policy, the mailbox becomes an inactive mailbox. ‘The contents of an inactive mailbox are still subject to any retention policy that was placed on the mailbox before it was made inactive.)
  • If ‘No’ is selected, the content will be left in place and must be manually deleted at some point.
• No, just delete the content that’s older than … The options are to delete: (a) after ‘n days/months/years’, and (b) based on when it was created or modified.

The (subtle) difference between these two options is that the first option (Yes) ensures that records are not permanently deleted before the end of the retention period, while the second option (No) just deletes records permanently at the end of the retention period.

Advanced retention settings are also available these allow the administrator to create a search query with specific words phrases, or link the policy with the same sensitive information options found under DLP policies, e.g., financial, medical and health, privacy, and custom.

Locations

The Locations section sets where the policy will be applied. By default this is all locations across Office 365, including content in Exchange, SharePoint, OneDrive, Office 365 Groups and Skype for Business.

• Office 365 has a limit of 10 organisation-wide policies and entire-location policies combined per tenant. Therefore, careful consideration should be given to what specific types of record need a global policy, especially given that not all types of records will be found globally across the organisation.

The alternative option is to apply the policy only to specific locations or users. In most cases this is likely to be Exchange and SharePoint where the majority of key records are created and stored.

• A retention policy that includes or excludes over 1,000 specific users can contain no more than 1,000 mailboxes and 100 sites. A tenant can contain no more than 1,000 such retention policies. According to Microsoft ‘… you can get over these limits by applying either an org-wide policy or a policy that applies to entire locations’.

Retention policies applied to a SharePoint site or OneDrive account result in the creation of a hidden Preservation Hold library as noted above.

Retention policies applied to Exchange user mailboxes apply the policy to the mailbox. For public folders, the retention policy is applied at the folder level.

Preservation Lock

Finally, the administrator has the option to apply a Preservation Lock, which prevents anyone from changing or deleting the policy after it is turned on. This option should only be applied in specific circumstances as it cannot be turned off or made less restricted (by anyone, including the administrator) after it has been applied. .

Review and save

Finally, the new retention policy should be reviewed, may be saved for later, or published.

Labels

A separate option for managing retention and disposal is to use (retention) labels, which should not be confused with security labels. This option is designed to replace the following:

• Exchange Online retention tags and retention policies, also known as messaging records management (MRM).
• In SharePoint Online and OneDrive for Business: (a) in-place records management, (b) the Records Center, and (c) information management policies.

Labels are used to manage retention policies for specific types of content across the Office 365 environment. Labels can be applied automatically to content if it matches certain conditions or keywords (E5 licence only), or manually by users to emails, documents, or Office 365 Group conversations.

See below for the relationship and priority between retention policies and labels.

Who can create labels

Labels are created by individuals (ideally records managers or similar) assigned to a compliance role in the Security and Compliance Admin portal in Office 365.

Creating Labels

Labels are created in the Security and Compliance Admin Portal under ‘Classifications’. Labels may also be created without having an associated retention policy; that is, a label can be created and applied to content as no more than a visual ‘tag’. A policy can be added to it at a later stage.

If the ‘Retention’ option is enabled for labels (on/off switch), a new section appears titled ‘When users apply this label to content’. This section is where the retention policy is defined with two options:

• Retain the content. The choices are either ‘Forever’ or ‘n days/months/years’ (e.g., 7 years). The administrator must decide if, once it reaches that point, the labelled record should be deleted or not. The ‘Yes’ and ‘No’ options are the same as for retention policies, described above.
  • If ‘Yes’ is selected, the record will be deleted from where it is stored. Administrators have 93 days to recover records that have not been edited or deleted, or 7 days to records that have been edited or deleted (and moved to the Preservation Hold library).
  • If ‘No’ is selected, the content will be left in place and must be manually deleted.
• Don’t retain the content. The choices are to delete (a) after ‘n days/months/years’, and (b) based on when the record was created, modified, or labelled.

If the first option (‘Retain the content’) above is selected a check box option allows the administrator to use the label to classify content as a record. If the content is classified as a record, users are unable to change or delete the content or change or remove the label. They may still, however, edit the metadata.

The final step in the process is to review the settings. Once created, the administrator is returned to the main Labels screen which displays the label that has been created, allowing the administrator to then publish it.

Label limitations when used on a SharePoint document library

There are some limitations to applying a default label to a SharePoint document library:

• It applies the label to all records except those that already have a label and those contained in document sets.
• If the default label is removed, it removes the label from all records except those that have a label and those contained in document sets.
• Labels cannot be applied to folders in SharePoint or OneDrive (but can be applied to folders in Exchange).
• If the record is moved to a different library that has a different default label, it will inherit that label. Conversely, if it is moved to a library with no label, the existing label will be removed.

Note: When labels are published to an Office 365 group, the labels appear in both the group site and group mailbox in Outlook on the web. The experience of applying a label to content is identical to that shown above for email and documents.

What about legal holds?

eDiscovery in Office 365 is based around the creation of ‘cases’ in a SharePoint eDiscovery site. Cases are generally established in response to litigation (or potential litigation) and can be used to search across a range of sources. Once found, the information that forms part of the case can then be placed on hold, overriding any retention policy. However, once the hold is released, retention policies on records continue.

For more information on this subject, see:

https://support.office.com/en-gb/article/Add-content-to-a-case-and-place-sources-on-hold-in-the-eDiscovery-Center-54d70de9-1ec2-4325-84f3-aeb588554479?ui=en-US&rs=en-GB&ad=GB

What’s the relationship between retention policies and labels?

Retention policies and labels do the same thing but the former is more likely to be set centrally, while the latter is set by the end user. This means that a record could have more than one retention policy applied to it.

According to Microsoft’s documentation (link below), records will be retained until the end of the longest retention period applied to it, regardless of whether that policy was based on the retention policy or the label.

Are retention policies and labels better than previous retention options?

One of the primary benefits of the new retention policy regime in Office 365 is that it enables organisations to apply retention policies centrally rather than do this separately for each application (e.g., Exchange, SharePoint) as was the case until recently. It also allows end users to apply retention policies via labels.

Retention and disposal continues to be based on the individual record, or type of record (as defined by the policy or label), not logical aggregations or containers of records such as a document library.

As noted above, the concept of an aggregation that contains all the records on a given subject is ill-suited to the digital world. The reality is that records may be created using different applications (e.g., email in Exchange, document, list item or page in SharePoint, conversation in Groups, discussions in Skype etc) and stored in multiple application locations (e.g. in Exchange folders, SharePoint libraries, etc).

The dilemma for many records managers using Office 365 is how to store or manage records together in context, including based on the organisation’s File Plan or Business Classification Scheme (BCS) terms. The need to keep records together has been the driver behind the integration of EDRM systems with email applications, allowing email to be ‘captured’ in the EDRM along with other types of documents. This has rarely been successful in practice and, in most cases, emails are duplicated and remain stored in the email server.

The new Office 365 retention policies, including those applied as labels to specific types of content, may well be the answer to this dilemma. Rather than try to capture all types of records (e.g, document email, list item, conversation) in a single aggregation or container, Office 365 allows the option for them to be stored wherever the user prefers, with the same retention policy applied.

If necessary, all records with the same label can then be found using a content search in the ‘Search and Investigation’ section of Office 365.

In my view, there are still some shortcomings in basing retention policies on individual record types:

• Individual documents, rather than logical aggregations of documents, will be continue to be subject to disposal actions.
• Records that may provide context to other records (including those stored in different locations) may be destroyed.
• Appraisal options may be limited and appropriate review and approval steps before disposal may not be possible.
• Disposal actions may be automatic and unrecoverable.
• There may be no record kept, including the metadata, of the individual records that were destroyed.
• It is not known how courts might view the automatic disposal of records without prior review and approval.

Final thoughts

The new Office 365 records retention policy and label options centralise the management of retention and disposal for most types of records across Office 365, reducing complexity.

Retention and disposal continues to be based on individual records rather than aggregations, but this may be better suited to the digital world in which aggregations of records may not always be achievable.

Records managers working in organisations using Office 365 need to understand and provide guidance to IT on how records retention schedules can be applied as retention policies, and how they can be directly involved in decisions regarding the new options.

For more information: –

https://support.office.com/en-us/article/Overview-of-retention-policies-5e377752-700d-4870-9b6d-12bfc12d2423

https://support.office.com/en-us/article/Overview-of-labels-af398293-c69d-465e-a249-d74561552d30

 

Office 365 & SharePoint Online – Data Loss Prevention (DLP)

Summary

Office 365 includes a range of information security and protection capabilities. This post focusses on the configuration and implementation of Data Loss Prevention (‘DLP’) capabilities in SharePoint Online and OneDrive for Business (ODfB).

Note: Microsoft have advised that the Office 365 DLP framework will apply to both Exchange and SharePoint/ODfB in the near future. For Exchange settings see https://technet.microsoft.com/en-us/library/jj200706(v=exchg.150).aspx and https://technet.microsoft.com/en-us/library/jj150527(v=exchg.150).aspx for more information.

Purpose of DLP

The purpose of DLP is to protect specific and definable types of sensitive company or agency information by preventing (or monitoring) its deliberate or inadvertent exfiltration from the organisation.

Examples of exfiltration methods where DLP can be used include:

• Attachments to emails.
• Uploads to web-based systems.

Examples of the types of sensitive information that can be protected with DLP include:

• Financial data. For example, bank account numbers, tax file numbers, credit/debit card numbers.
• Personal and sensitive information (PSI). For example, driver’s licence numbers, tax file numbers, passport numbers.
• Medical and Health records. For example, medical account numbers.

The requirement to protect sensitive information is the subject of legislation in a number of countries.

Enabling Data Loss Prevention in Office 365

DLP in Office 365 enabled through policies that are set in the Security and Compliance Admin Centre of the Office 365 Admin Portal, under ‘Threat Management’ > ‘Data Loss Prevention’.

DLP policies are set by the Office 365 Global Administrator, as well as the Compliance Administrator and/or the Security Administrator if these roles have been configured in the Security and Compliance Admin Centre.

To create a DLP policy, the Administrator clicks on the + icon in the Data Loss Prevention screen. This opens a new window with the following options displayed.

A custom policy is one that is defined by the organisation. It would normally be for content that contains specific values.

The options ‘Financial regulations’, ‘Medical and health regulations’, and ‘Privacy regulations’ include default Microsoft-provided policies. Each of these default policies includes a description, coverage (e.g., what information is protected), and where the information is to be protected (e.g., in SharePoint Online, OneDrive for Business, and Exchange Online).

Enabling and modifying default policies

After selecting a default policy, the authorised user must then identify the services that may store the information that need to be protected – SharePoint Online, OneDrive.

Note: The option to choose Exchange Online is (as of 13 March 2017) still unselectable.

The next option allows the Administrator to customise the rule that has been chosen. If a default policy has been selected in the previous dialog, options for that policy will display; these may include ‘count sensitivity’ (i.e., how many times the sensitive content is identified. Low count means high sensitivity to sensitive content.

The Administrator may add a new rule or edit one of the default options.

The Administrator may modify the conditions, actions and what happens when there is an incident for each of the default policies – see below for further details.

Defining custom DLP policies

If a custom policy is required, the Administrator clicks on ‘Custom Policy’ from the ‘Data Loss Prevention’ opening dialog screen, and then ‘Next’ at the bottom of the screen. The Administrator must define which services are to be protected (same as for default policies, above).

The next screen allows the Administrator to create a new policy, via the + icon.

In the new window that opens, the Administrator can then must define the new DLP rule through Conditions, Actions, Incident reports and General.

Conditions, Actions, Incident reports, General options

For either default or custom policies, the Administrator must set the following rules:

• Conditions – what will cause the policy to run?
• Actions – what will happen when the policy runs?
• Incident reports – how is reporting managed?
• General – any other points.

Conditions

For default policies, conditions are pre-defined and are based on (a) the type of content (e.g., credit card numbers, bank account numbers) and (b) whether the content is shared internally or externally.

These pre-defined conditions may be removed or edited, and new conditions may be added. Editing options include the number of times the sensitive content is found (‘Min count’, ‘Max count’), and both maximum and minimum percentage-based ‘confidence levels’.

For custom policies, the Administrator must define which conditions are to be met:

• If you choose ‘Content contains sensitive information’, you must define the information through a + option. This brings up all the default choices provided by Microsoft.
• If you choose ‘Content is shared with’, it allows you define if the information is shared with people inside or outside the organisation.
• If you choose ‘Document properties contain any of these values’, you must define the values that would be found in a document. Note that, if this option is selected, the property must be configured in the SharePoint Online search settings.

Actions

For default policies, the actions to be taken are pre-defined and are based on sending a notification.

For custom policies, the Administrator must first decide whether the action will be to (a) block the content or (b) send a notification.

If ‘Block the content’ is selected the user will be unable to send an email or access the shared content.

If ‘Send a notification’ is selected it offers the same options as for custom policies. Note the ability to customise the email notification.

Incident Reports

When ‘Incident Reports’ is selected for both custom and default policies, the following options are available. Incident reports should be sent to the Administrator/s.

General

Default policies are pre-named but the name can be modified. This is also where the policy can be disabled.

Custom policies must be named and a decision made whether to enable it, test it, or turn it off. As noted below it is possible to test the policy first, to collect data.

DLP Reporting

Reporting from the DLP policies is accessed from the Security and Compliance Centre > Reports > Dashboard.

Applying information security and protection capabilities in Office 365 & SharePoint Online

Office 365 includes a range of information security and protection capabilities. These capabilities are first set in Azure and then applied across the Office 365 environment, including in Exchange and SharePoint Online. This post focuses on the application of these capabilities and settings to SharePoint Online.

Enterprise E3 and E4 plans include the ability to protect information in Office 365 (Microsoft Exchange Online, Microsoft SharePoint Online, and Microsoft OneDrive for Business). If you don’t have one of those plans you will need a subscription to Microsoft Azure Rights Management.

Enabling Information Protection in Azure

The following steps must be carried out the first time Information Protection is enabled on Azure:

• Log on to Azure (as a Global Administrator).
• On the hub menu, click New. From the MARKETPLACE list, select Security + Identity.
• In the Security + Identity section, in the FEATURED APPS list, select Azure Information Protection.
• In the Azure Information Protection section, click Create.

This creates the Azure Information Protection section so that the next time you sign in to the portal, you can select the service from the hub ‘More Services’ list.

Default Azure Information Protection policies

There are four default levels in Azure Information Protection:

• Public
• Internal
• Confidential
• Secret

Once set, these levels can be applied as labels to information content. Sub-labels and new labels may also be created, as necessary via the ‘+ Add a new label’ option.

The configuration settings are shown below:

Each of these label/level settings may:

• Be enabled or disabled
• Be colour-coded
• Include visual markings (the ‘Marking’ column)
• Include conditions
• Include additional protection settings.

Each includes a suggested colour and recommended tip, which are are accessed via the three dot menu to the right of each label.

Markings

When selected, this option will place a label watermark text on any document when the label is selected.

Conditions

Conditions may be applied, for example, if credit card numbers are detected in the text. It allows the organisation to define how conditions apply, how often (Occurrences), and whether the label would be applied automatically or is just a recommended option.

Global Policy Settings

In addition to the settings per level, there are three global policy settings:

• All documents and emails must have a label (applied automatically or by users): Off/On
  • When set to On, all saved documents and sent emails must have a label applied. The labeling might be manually assigned by a user, automatically as a result of a condition, or be assigned by default (by setting the Select the default label option).
• Select the default label:
  • This option allows the organisation the default label to be be assigned to documents and emails that do not have a label.
  • Note: A label with sub-labels cannot be set as the default.
• Users must provide justification to set a lower classification label, remove a label, or remove protection: Off/On [Not applicable to sub-labels]
  • This option allows you to request user justification to set a lower classification level, remove a label, or remove protection. The action and their justification reason is logged in their local Windows event log: Application > Microsoft Azure Information Protection.

Custom Site

A custom site may be set up for the Azure Information Protection client ‘Tell me more’ web page.

Unique ‘Scoped’ Policies

In addition to the default policies listed above, a unique policy may be created. These are called Scoped Policies.

Enabling (and Disabling) Azure Information Protection

The steps above are used to set up the labels. They must then be enabled to provide protection. The steps below also allow protection to be removed.

From the Azure Information Protection section, click on the label to be set, then click on Protect. This action opens the Permission settings section.

Select Azure RMS and ‘Select template’, and then click the drop down box and select the default label template. This will probably show as, e.g., ‘(Your Company Name) – Confidential’.

Click ‘Done’ to enable this label and repeat for the others.

Note: If a new template is created after the Label section is opened, you will need to close this section and return to step 2 (to select the label to change), so that the newly created template is retrieved from Azure.

Removing Protection

Users must have the appropriate permissions to remove Rights Management protection to apply a label that has this option. This option requires them to have the Export (for Office documents) or Full Control usage right, or be the Rights Management owner (automatically grants the Full Control usage right), or be a super user for Azure Rights Management. The default rights management templates do not include the usage rights that lets users remove protection.

If users do not have permissions to remove Rights Management protection and select this label with the Remove Protection option, they see the following message: Azure Information Protection cannot apply this label. If this problem persists, contact your administrator.

Additional notes

If a departmental template is selected, or if onboarding controls have been configured:

• Users who are outside the configured scope of the template or who are excluded from applying Azure Rights Management protection will still see the label but cannot apply it. If they select the label, they see the following message: Azure Information Protection cannot apply this label. If this problem persists, contact your administrator.
• All templates are always shown, even if a scoped policy only is configured. For example, a scoped policy for the Marketing group; the Azure RMS templates that can be selected will not be restricted to templates that are scoped to the Marketing group – it is possible to select a departmental template that selected users cannot use. It is a good idea (to help troubleshoot issues later on) to name departmental templates to match the labels in the scoped policy.

Once these settings are made, they need to be published (via the ‘Publish’ option) to become active.

Enabling Information Protection in Office 365

Activating Information Protection in the Office 365 Admin Portal

Once they have been configured and published, it is then necessary to enable the required settings in the Office 365 Admin Portal (Settings > Services & add-ins > Microsoft Azure Information Protection).

To do this, log on to the Office 365 Admin Portal (as a Global Administrator) then click on ‘Services & add-ins’ under Settings. Click ‘Activate’ to activate the service.

Activating Information Protection for Exchange and SharePoint Online

Once the service is activated for Office 365, it can then be activated in the Exchange and SharePoint Admin Centres. In SharePoint Online this is done via the Admin Center section ‘Settings’ and ‘Information Rights Management (IRM)’.

Configuring SharePoint and SharePoint Libraries for IRM

As at 12 March 2017, it is only  possible to link Azure Information Protection classification policies with SharePoint Online if a new site is created via the SharePoint end user portal, as it appears as an option when enabled. Sites created via the SharePoint Admin Portal do not (yet) include the option to apply a protection classification.

If the creation of sites via the SharePoint end user portal is enabled, users with appropriate permissions (e.g., Owners with Full Control) can apply Information Rights Management to SharePoint libraries in their sites.

IRM is enabled on each individual library or list where the settings will be applied via Library Settings > Information Rights Management, under Permissions and Management.

Check the box to ‘Restrict permissions on this library on download’. Only one policy can be set per library.

Assigning Information Protection labels to Office documents

[NOTE: for clients that have installed versions of Office, the Azure Information Protection client needs to be installed on the desktop. See this site for more information: https://docs.microsoft.com/en-us/information-protection/get-started/infoprotect-tutorial-step3%5D

When labels are configured and enabled, they can then be be automatically assigned to a document or email. Or, you can prompt users to select the label that you recommend:

• Automatic classification applies to Word, Excel, and PowerPoint when files are saved, and apply to Outlook when emails are sent. It is not possible to use automatic classification for files that were previously manually labeled.
• Recommended classification applies to Word, Excel, and PowerPoint when files are saved.

Applying the policies to Exchange and office

The site below describes how to apply these policies to Exchange and Office applications. These are not discussed further here.

https://github.com/Microsoft/Azure-RMSDocs/blob/master/Azure-RMSDocs/deploy-use/configure-applications.md

Managing Project Records in Office 365

The introduction of Office 365 Groups brings a new way of working with and managing project records, including emails, documents and other types of records. But controls need to be in place to prevent uncontrolled growth.

A typical project team is likely to create two main types of record – documents and emails. More often than not in the digital world these are kept separate and unconnected to with the main project records, unless the user saves emails to where the documents are stored, or all documents remain attached to emails.

The introduction of Office 365 Groups brings an innovative way to work in projects and keep all project-related records together.

What are Office 365 Groups?

Groups are similar in some respects to a both (a) Distribution List (DL) in that they allow a group of people with a common interest to communicate with each – albeit on a point-to-point basis without new users being able to access earlier emails, and (b) a (public or private) Yammer group in that they allow the members of the group to discuss issues together ‘out loud’ instead of in one-to-one emails.

In addition to ‘conversations’ that take place in Groups, Groups also have an associated SharePoint site, a shared calendar, a plan in Planner, and a notebook in OneNote. These options are visible from the Group view in Outlook:

A (private) Group can be linked directly to a Team (in Microsoft Teams), allowing further types of exchange, including in multiple channels.

Office 365 Groups allow all types of project records – emails, conversations, documents, plans, chats, notes – to be accessed in the one place linked by the unique name given to the Group when it was created. External guests may also be invited a Group.

But, to be clear, this does not mean that these records are all stored in the one location; the records remain in Exchange, SharePoint, OneNote, Planner, or Teams. What connects them together is the unique name or identifier.

Creating Groups

The default settings in Office 365 allow Office 365 Groups (and SharePoint sites and Teams) to be created by anyone in the organisation. The danger in allowing these default settings is uncontrolled growth; when a Group or Team is created it also creates an associated SharePoint site (that is not yet visible in the SharePoint Admin portal).

To minimise uncontrolled growth, it is recommended that these default options be disabled, and that the creation of Office 365 Groups, SharePoint sites and Teams, be limited to the Office 365 Administrators, based on requests from users.

Groups should, ideally, be assigned a prefix to distinguish them from each other and from DLs and Security Groups (SGs) that are also used in Outlook. It will be interesting to see to what extent DLs are replaced over time by Office 365 Groups, as the latter are more functionally useful.

A suggest prefix for name of a project Group could be ‘PRJ’ as shown below. The same name is then used on the SharePoint site, in Planner, in OneNote and, if the Group is private, on the associated Team in Microsoft Teams making the connection between them clear.

Note:

• It is not possible to associate a public Group with a new Team; if a new Team is created with the same name as a public Group, it will create a Group with the same name).
• Creating a new Modern Team Site from the ‘New Site’ option (if enabled) on the user’s SharePoint portal also creates a Group. If controls do not exist (and the options are not disabled), users will quickly start to create multiple SharePoint sites that have associated Groups, and things could get out of hand very quickly).

Managing Project Records More Effectively

Office 365 Groups, and their associated elements – SharePoint, Planner, Teams etc – allow project records to be accessed from a single point – Outlook (on a browser or mobile device app).

Each of these elements can also be accessed from both iOS and Android apps, allowing all members of the team to communicate and share information more effectively.

Instead of sending project documents attached to emails, documents can be sent as links in email, conversations and team chats. Documents can also be proactively and jointly edited by multiple people at the same time, including using both apps-based and online versions of Office applications.

These options, via Office 365 Groups, should improve the way project records are managed.

How Office 365 challenges traditional records management practices

If your organisation is using SharePoint on-premise now, or just starting out with Office 365, it is important to understand how the Office 365 ecosystem will challenge traditional ways of managing records practices while at the same time delivering a transformational all-digital experience for end users.

SharePoint On-Premises

When configured well, SharePoint on-premises (e.g. versions up to SharePoint 2016) allowed organisations to manage unstructured (i.e., document-based) content through a hierarchy of site collections – sites/sub-sites – document libraries – (folders/document sets) – documents.

In on-premise SharePoint environments, document libraries could be used to store and manage records, thereby becoming the logical containers or aggregations of records, similar to ‘files’ in traditional EDRM systems.

The Office 365 ecosystem

Office 365 changes and challenges the on-premises model of SharePoint by adding new ways of working to standard SharePoint team and publishing sites. These new ways of working include:

• Office 365 Groups, each of which has a dedicated SharePoint site
• OneDrive for Business, a personal version of SharePoint
• Yammer
• Skype for Business
• Delve
• Planner
• Sway

Why is this important? 

SharePoint has been clearly positioned as Microsoft’s online document management engine. SharePoint, not network file shares, is the document management future. And so, by extension, it becomes the future location for the management of digital records for any organisation that subscribes to Office 365.

From both the business and end-user points of view, SharePoint provides easy-to-use and more efficient content management and collaboration capabilities allowing users to access and use a range of content anywhere, anytime, on any device. Coupled with collaboration options such as Office 365 Groups, Yammer, and Skype for Business, information is now available across a number of different applications within the same single ecosystem.

From a records management point of view, this new way of working challenges the idea that information can be stored in the context of a single function, activity or transaction that created it. Instead, it supports the concept that digital information cannot truly be assigned to a single function or context; its context may also depend on the context of the person seeking to access it.

That is, how one person stores information is not necessarily how others may expect to find or use it. Think of the parallels with eBay, Facebook, LinkedIn and similar products – algorithms present information to you, often in a ‘feed’, based on what the application knows about you, not how other people store that information.

‘Modern’ Team Sites

The most striking change with ‘modern’ team sites in SharePoint Online (compared with SharePoint 2013 and earlier) is the disappearance of the ribbon menu and the simplification of the user-experience to be more or less identical with OneDrive for Business.

When any library is selected (and before a document is selected), the user is presented with the common options: New (Folder, Word, Excel, PowerPoint, OneNote, Link), Upload, Quick Edit, and Sync.

When a document is selected, the user is presented with a context-specific menu offering again commonly used options: Open, Share, Get a link, Download, Delete, Pin to top, Move to, Copy to, Rename, Version History, Alert me, and Check out.

The familiar Library Settings, previously located on the ribbon menu, are now found via the Office 365 settings ‘cog’.

Microsoft have also changed the look of SharePoint Online sites and provided a new ‘SharePoint’ landing page to help users access all the sites they are following, and also present suggestions for sites to follow. In other words, the system understands the user’s context and presents content suggestions, the same way Facebook users are invited to befriend people.

From a records management point of view, little has changed with document libraries in team sites. SharePoint Online continues to offer all the same features as before:

• Almost unlimited metadata options allowing multiple metadata-based views to be set up
• Unique, persistent document IDs
• Folders and document sets (although the latter are even harder to set up than they were)
• Versioning (and more efficient storage of versions)
• Popularity trends and per-document views
• Detailed audit trails
• Access/permission controls
• Legal compliance/retention and disposal
• Powerful search
• Full integration with Office but now allowing users to save directly by default to SharePoint and OneDrive by default.
• Hyperlinkable documents
• Easy sharing

While it is still possible in SharePoint Online to manage records out the box, the other elements that make up the Office 365 ecosystem provide a much broader and complex environment for the storage and management of records. SharePoint Online is just one component of this environment.

Office 365 Groups

Office 365 Groups provide a way for a group of people within the organisation – as well as external users – to discuss and share information.

• They are similar to Active Directory (AD) Distribution Groups in the sense that they are a pre-defined organisational group designed to receive information.
• They are different in that, instead of being just the recipient of information, users (and people who join the group at a later date) can see all discussions that have been sent to all members and access any Group documents.

Office 365 Groups are made up of two main content elements: ‘Conversations’ email-based threads and ‘Files’.

• Conversation threads are based on simple email exchanges presented in Outlook – currently it is not possible to create folders in the group.
• The Files option in Office 365 Groups is a SharePoint site that allows the group to store, share and collaborate on any unstructured content.

Groups also include a calendar and a group Notebook (which opens OneNote Online in the Group SharePoint site).

Office 365 Groups content is stored either within the context of the Group’s email-based conversations or in unstructured content stored in an associated SharePoint site.

Office 365 Groups SharePoint sites are visible in the user’s list of SharePoint sites, making it easy to get back to the Group’s site or its conversations.

OneDrive for Business

OneDrive for Business is built on the SharePoint engine. The consumer version of OneDrive has been around for a few years and is a direct competitor to the likes of Google Drive, iDrive, DropBox, Box and so on.

OneDrive for Business, the online replacement for ‘personal’ network drives, allows users to store, synchronise and share ‘personal’ work information through an interface that in Office 365 is now almost identical with modern team sites (less the Library Settings).

As with personal drives on network drives, content stored by users on OneDrive for Business is inaccessible unless shared with others. Organisations have only 30 days by default to do something about the user’s OneDrive for Business content when they cease to be an employee, before the content is deleted.

Options to manage the otherwise hidden content of a departed user’s OneDrive for Business account include allowing the user’s manager to review and if necessary move or delete it, allowing an authorised person in IT to review it, and/or backing it up to other storage so it is not deleted.

Yammer

While the long-term future of Yammer is unclear in the face of Office 365 Groups, Yammer may still exist and capture information and records for a time to come.

Skype for Business

In addition to Yammer and the conversation options provided through Office 365 Groups, Skype for Business provides yet another option to discuss and share information including via voice and/or video calls.

Delve

All the options described above provide a function-rich environment to store and manage unstructured content and collaborate with other people both within and external to the organisation. But how to make sense of all this information?

Depending on licensing, Delve provides a way to find content that may be relevant to the user.

Delve suggests a range of content that may be of interest (based on the user’s profile, connections and content created or accessed), and provides an analysis of the user’s activity as recorded in Outlook, the calendar and other actions.

Challenges with managing records in Office 365

While Office 365 provides a transformative digital experience for end users, managing the records created and stored in various parts of Office 365 presents new challenges for records managers.

For example, there is far less ability to control the way content is stored or described in specific, pre-defined and/or metadata-driven aggregations and contexts. Users are likely to use whatever application is the most appropriate or convenient. For example, they may use OneDrive for Business to create and store large volumes of content, hidden away from corporate view. They may even share content from this application, including with external users.

The default settings in SharePoint, if not disabled, provide end-users with considerable latitude to create new SharePoint sites and Office 365 Groups, in addition to their personal OneDrive for Business sites, to store, manage and share rich digital content including with external users. In reality, these settings probably need to be disabled to prevent uncontrolled growth in the environment.

Even if records managers (as Site Collection Administrators) have oversight and control of the creation of SharePoint Online team sites, some questions arise:

• How will they extend this control to SharePoint sites created to support Office 365 Groups, or the conversations that take place within those groups?
• What about content stored in and shared from OneDrive for Business?
• How will it be possible in the future to bring together all information about a given function/activity for disposal or disposition actions, especially if it’s not all stored in the one aggregation?

Good SharePoint (and Office 365) governance requires a good balance of control. Too much control and users will be put off using and benefiting from the ecosystem. Too little and the ecosystem may become uncontrollable but possibly very ‘lively’ in terms of content profusion.

Ideally, users should feel that they have the ability to manage their information within a lightly controlled environment – for example, SharePoint site owners cannot create new Sites (to prevent the massive proliferation of sites) but they can create document libraries (thereby reducing IT administrative controls).

Can analytics help with managing records?

Analytics via the Office Graph may provide a way to bring together information and records in context, a context (or contexts) which may be unforeseen by the person who created the content in the first instance. For example, a user may store information in a document library, unaware of its relevance or similarity to others in the organisation. Analytics may be able to connect the two, or the different people doing similar things.

At this stage, Analytics does not seem to provide the ability to bring together all information about a given subject. The model, instead, appears to be about presenting or making information accessible in any context at any time to users depending on their context at the time.

eDiscovery?

eDiscovery, a feature available from SharePoint 2013, has the potential ability to bring together all information about a given subject from across the Office 365 ecosystem. However, the primary purpose of eDiscovery is to support legal processes, not records management.

New ways of thinking are necessary

Records managers need to think differently about how they will approach the management of all types of digital records and other content (conversations, discussions, photographs, videos, Sway presentations) created and stored by users across the complex ecosystems that is Office 365.

It will no longer be possible to assume that all records relating to a given function/activity pair, subject, or context can or will be stored in the same aggregation of records. Instead, records managers need to find other ways to manage digital content, including to manage disposition activities.

Artificial Intelligence (AI) may provide the clue to this. Microsoft CEO Satya Nadella made this very clear in a keynote presentation to the Microsoft Ignite conference on 26 September where he noted that AI would be able to: “… to reason over large amounts of data and convert that into intelligence”. He also noted Microsoft’s ambition is to create an intelligent assistant that “… can take text input, can take speech input, that knows your deeply. It knows your context, your family, your work. It knows about the world.”

Nadella also noted that: ” The most profound shift is in the fact that the data underneath the applications of Office 365 is exposed in a graph structure. And in a trusted, private-preserving way, we can reason over this data and create intelligence. That’s really the profound shift in Office 365.” (Source: https://techcrunch.com/2016/09/26/microsoft-ceo-satya-nadella-on-how-ai-will-transform-his-company/)

(Note, the last two paragraphs were added on 29 September to include comments made by Satya Nadella about Microsoft’s AI ambitions).

 

 

Information Security in SharePoint Online

Until now, the security of information stored in SharePoint on-premise implementations was largely based on access control groups that gave or restricted access to the content on the site. Access to the content, and ability to do anything with it (e.g., edit, read) depending on what group you belonged to. The main five access control groups are:

• SharePoint Administrator/s: Access to everything.
• Site Collection Administrator: (Usually) access to everything, but this can be disabled.
• Site Owners: ‘Full Control’ access to everything (except for the Site Collection Administration elements in Site Settings).
• Site Members: ‘Contribute’ or add/edit access.
• Site Visitors: Read only.

Other groups such as Designer and Reader existed for specific purposes.

At any point from the top level Site Collection downwards through all the content, these inherited permissions could be stopped and unique permissions – including for both individuals and new access groups – could be created and applied to control access to content.

Audit logs supplemented access controls by providing details of who did (including changing security permissions) or accessed what, and when. While the SharePoint Administrator and Site Collection Administrator’s names are not visible to Site Owners, Members or Visitors, they appear in the audit logs if any activity is recorded. System account activity is also recorded in the logs.

New Security Controls in SharePoint Online

SharePoint Online brings a range of new options to protect the security of information, in addition to access controls. These options, some of which are included with SharePoint 2013 an onwards, are:

• Information security classifications
• Data Loss Prevention (DLP)
• Audited sharing
• Information Rights Management (IRM)
• Shredded storage (new from SP 2013)

Two of these options can be seen in the following Microsoft diagram:

Source: ‘Monitoring and protecting sensitive data in Office 365’ https://msdn.microsoft.com/en-us/library/mt718319.aspx

Information Security Classifications

According to a number of online sources, from at least March 2011, Microsoft has classified its own information into three categories: High Business Impact (HBI), Moderate Business Impact (MBI), and Low Business Impact (LBI).

• High Business Impact (HBI): Authentication / authorization credentials (i.e., usernames and passwords, private cryptography keys, PIN’s, and hardware or software tokens), and highly sensitive personally identifiable information (PII) including government provided credentials (i.e. passport, social security, or driver’s license numbers), financial data such as credit card information, credit reports, or personal income statements, and medical information such as records and biometric identifiers.
• Moderate Business Impact (MBI): Includes all personally identifiable information (PII) that is not classified as HBI such as: Information that can be used to contact an individual such as name, address, e-mail address, fax number, phone number, IP address, etc; Information regarding an individual’s race, ethnic origin, political opinions, religious beliefs, trade union membership, physical or mental health, sexual orientation, commission or alleged commission of offenses and court proceedings.
• Low Business Impact (LBI): Includes all other information that does not fall into the HBI or MBI categories.

Source: ‘Microsoft Vendor Data Privacy – Part 1’ (March 2011) https://www.auditwest.com/microsoft-vendor-data-privacy/

Microsoft released code (via Github) to apply these classifications to SharePoint on-premise deployments in 2014.

Source: https://github.com/OfficeDev/PnP/tree/master/Solutions/Governance.TimerJobs

In 2016 Microsoft released a Technical Case Study highlighting how it migrated all its SharePoint content to SharePoint Online – and how information classification formed part of that process.

Source: ‘SharePoint to the Cloud – Learn how Microsoft ran its own migration’ (Case Study – 2016)  https://msdn.microsoft.com/en-us/library/mt668814.aspx

In May 2016, Microsoft announced that this form of classification would be added to new SharePoint Online site collections during 2016.

The application of security classifications to SharePoint Online sites has two elements:

• Security and compliance policies, set by the SharePoint Administrator via either the ‘Security policies’ or ‘Data management’ section of the Office 365 Security & Compliance Center. [As of 23 May 2016 the only policies are ‘Device management’ and ‘Data Loss Prevention’. While the DLP policies appear to allow the inclusion of security classifications, it is expected that Microsoft will add more options to support the application of security classifications during 2016. See below for more information on DLP.]
• A new drop-down, three choice (LBI, MBI, HBI) option in the ‘Start a new site’ dialogue box under the question ‘How sensitive is your data?’ The choice of classification invokes the relevant security and compliance policies.

Microsoft provides examples of the types of information that would be covered by each of these at this interactive site: https://www.microsoft.com/security/data/

The application of these policies will enable organisations to control what happens to information stored in sites assigned these classifications. Among other things, this can prevent users from sending (or trying to send) MBI or HBI classified information to people not allowed to receive or view it, including through DLP policies discussed in the next section.

Data Loss Prevention (DLP)

Data Loss Prevention policies allow organisations to:

• Identify sensitive information across both SharePoint Online and OneDrive for Business sites (and in Exchange, through the same settings).
• Prevent the accidental sharing of sensitive information, including information classified MBI or HBI.
• Monitor and protect sensitive information in the desktop versions of Word, Excel and Powerpoint 2016.
• Help users learn how to stay compliant by providing DLP tips.
• View reporting on compliance with policies.

 

DLP Conditions

DLP works by giving Site Administrators the ability to create and apply DLP policies in the Security & Compliance Center for SharePoint (which includes OneDrive for Business; there is a separate Center for Exchange). In the Center, the Administrator navigates from ‘Security policies’ to ‘Data loss prevention’.

The DLP policy area includes a range of ‘ready-to-use’, financial, medical and privacy templates for a number of countries including the US, UK and Australia. Examples of pre-defined Australian sensitive information types include: bank account numbers, driver’s licence numbers, medical account numbers, passport numbers, and tax file numbers.

You may also create a custom DLP policy.

Sources: https://technet.microsoft.com/en-us/library/ms.o365.cc.newpolicyfromtemplate.aspx  https://support.office.com/en-gb/article/Send-notifications-and-show-policy-tips-for-DLP-policies-87496bc5-9601-4473-8021-cb05c71369c1

DLP Actions

Specific actions must be set for every DLP policy; that is, what happens if the policy conditions are met. The default actions are:

• Block access to content (for everyone except its owner, the person who last modified the content, and the owner of the site where the content is stored AND send a notification by email.
• Suggest a Policy Tip to users. Options are (a) Use the default Policy Tip or (b) Customise the Policy Tip.
• Allow override options. There is one main checkable option (‘Allow people who receive this notification to override the actions in this rule’) and two sub options:
  • A business justification is required to override this rule, and
  • A false positive can override this rule.

In addition to these actions, where the DLP policy identifies sensitive content in a document stored in SharePoint Online or OneDrive for Business it displays a small warning ‘stop’ sign icon on the document icon. Hovering over the item displays information about the DLP policy and options to resolve it.

DLP Incident Reports

Incident reports are designed to alert a compliance officer to details of events triggered by the DLP conditions, and provide reporting on those events.

Sources:

https://technet.microsoft.com/en-US/library/ms.o365.cc.DLPLandingPage.aspx

Audited Sharing

Information sharing is a common activity in SharePoint and in SharePoint 2016 and SharePoint Online it is actively encouraged through a new Share option.

In addition to other existing audit options, sharing activity can now be audited in SharePoint Online. The audit logs for Office 365 (which must be enabled) are accessed through the Office 365 Admin Center > Security & Compliance Center > Search & investigation > Audit log search.

Source: https://support.office.com/en-us/article/Use-sharing-auditing-in-the-Office-365-audit-log-50bbf89f-7870-4c2a-ae14-42635e0cfc01?ui=en-US&rs=en-US&ad=US]

Information Rights Management (IRM)

Microsoft’s Information Rights Management capability provides an additional layer of protection for a number of document types at the list and library level in SharePoint Online sites.

Supported document types include PDF, the 97-2003 file formats for Word, Excel and PowerPoint (e.g., Office documents without the ‘x’ at the end of the file extension – ‘word.doc’, the Office Open XML formats for Word, Excel, and PowerPoint (e.g. with the ‘x’ at the end – ‘word.docx’), the XML Paper Specification (XPS) format.

According to Microsoft, IRM:

‘… enables you to limit the actions that users can take on files that have been downloaded from lists or libraries. IRM encrypts the downloaded files and limits the set of users and programs that are allowed to decrypt these files. IRM can also limit the rights of the users who are allowed to read files, so that they cannot take actions such as print copies of the files or copy text from them.’

IRM is enabled via the Office 365 Admin Center > Admin > SharePoint > Settings > Information Rights Management > ‘Use the IRM service specific in your configuration’ and then ‘Refresh IRM Settings’.

Image source: ‘Apply IRM to a List or Library’ https://support.office.com/en-us/article/Apply-Information-Rights-Management-to-a-list-or-library-3bdb5c4e-94fc-4741-b02f-4e7cc3c54aa1

 

When IRM is activated on a library, any file that is downloaded is encrypted so that only authorised people can view them. Again, according to Microsoft:

‘Each rights-managed file also contains an issuance license that imposes restrictions on the people who view the file. Typical restrictions include making a file read-only, disabling the copying of text, preventing people from saving a local copy, and preventing people from printing the file. Client programs that can read IRM-supported file types use the issuance license within the rights-managed file to enforce these restrictions. This is how a rights-managed file retains its protection even after it is downloaded.’

Source:

https://support.office.com/en-us/article/Set-up-Information-Rights-Management-IRM-in-SharePoint-admin-center-239ce6eb-4e81-42db-bf86-a01362fed65c

Shredded storage

Shredded storage, as the name suggests, describes the way documents are stored in SharePoint, starting from SharePoint 2013. Instead of storing a document as a single blob, documents are stored in multiple blobs.

This is a more efficient – and possibly more secure – way to manage documents when they are updated by only updating the element/s that were changed. According to a Microsoft presentation on 4 May 2016:

‘… every file stored in SharePoint is broken down into multiple chunks that are individually encrypted. And, the keys are stored separately to keep the data safe. In the future, we would like to give you the ability to manage and bring your own encryption keys that are used to encrypt your data stored in SharePoint. If you want, you can revoke our access to the keys. And we will not be able to access your data in the service’.

Source:

https://blogs.technet.microsoft.com/wbaer/2012/11/12/introduction-to-shredded-storage-in-sharepoint-2013-rtm-update/

Other Information Security related options

The Microsoft website ‘Monitoring and protecting sensitive data in Office 365’ provides further information about other Information Security options in Office 365, including reporting options to support auditing of activity in the tenant.

Source: https://msdn.microsoft.com/en-us/library/mt718319.aspx