Posted in Digital preservation, Disasters, Electronic records, Governance, Information Management, OneDrive for Business, Records management, Retention and disposal

Managing the retention of content stored in OneDrive for Business accounts

The methods available to manage the retention of content stored by end-users in their Office 365 OneDrive for Business (ODfB) accounts are not always well understood.

Organisations may initially default (in their thinking) to backing up the content because that’s what was always done in the past. A change of thinking may be required.

This post:

  • Explains some of the key differences between ‘home drives’ and ODfB accounts.
  • Highlights the need for organisations to understand their business requirements for retention of ‘personal’ content, and not assume traditional backup methods are the only option.
  • Also highlights the need for organisations to understand the potential risks (and potentially unnecessary additional costs) associated with backing up Office 365 content.
  • Describes two simple options for the retention of content stored in ODfB accounts.
  • Suggests that organisations can probably use a combination of a single Office 365 retention policy and a change to the storage retention period for inactive accounts, instead of backups to achieve the same outcome.

What are ‘home drives’?

In many organisations, home drives are usually a dedicated area on a network file share designed to allow end-users to store ‘working’ documents and ‘personal’ content.

Using the network file shares for home drives ensures that the content stored in them is backed up as part of standard disaster recovery processes while the user is still active (for disaster recovery and to recover deleted items) and still accessible (as an ‘archive’) after they leave the organisation.

In some organisations, home drives may instead be an area on the user’s computer (C drive). Any content stored on local computers is not backed up.

Generally speaking, home drives – whether in the NFS or on the user’s computer, are not accessible once the end-user leaves the office. This has given rise to the fairly regular use of USB storage devices or uncontrolled, internet-based, file storage systems such as DropBox.

How is ODfB different from home drives?

In organisations that implement Office 365, ODfB is the replacement for ‘home’ or ‘personal’ drives.

Although they offer similar functionality for end-users (in terms of the ability to access the content from File Explorer), ODfB accounts are fundamentally different in several ways.

  • The content can be accessed on almost any device. No VPN is required.
  • With Windows 10 devices, the content is synced to and can be accessed via File Explorer. This makes ODfB an almost identical replacement for existing home drives in terms of look and feel, and functionality (plus even more functionality, such as the ability to share directly).
  • There is no accessible back up – Microsoft is entirely responsible for disaster recovery. If organisations want to back up ODfB accounts from Office 365, they will need to acquire a third-party product. The ability to establish retention for the content (last two dot points below) may make the need for back up redundant.
  • There is a 90 day Recycle Bin accessible via the browser-based interface. This allows end-users to restore the content they deleted themselves within that time-frame.
  • Organisations can set a storage retention period that will apply once the end-user leaves and their account is deactivated.
  • Organisations can also set a retention policy that will prevent the deletion of content while the user remains active.

Both the last two options are the subject of this post.

Access to and retention of home drives vs ODfB accounts

In many organisations, the content stored by end-users in their home drives is considered to be ‘private’ to them, despite the system being owned by the organisation.

While they can be accessed easily by network administrators with elevated privileges, it is not uncommon (often for audit purposes) for IT to have to seek special approval from someone senior to access the content of a home drive either while the end-user is still employed or after they have left. In these cases, IT will either access the active drive or request the back up tape to restore the content. 

The content in home drives, when backed up, remains as long as the backup media is accessible.

In Office 365, Global Administrators can access the ODfB accounts of any active user. They do this by going to the Office 365 Admin portal and, under the ‘Users’ section, clicking the end-user account name and then going to the ‘OneDrive’ tab where the option to ‘Get access to files’ is displayed’. Any access to ODfB accounts, by anyone (including Global Admins) is recorded in the audit logs.

[Note: At at January 2020, the old ‘My Sites’ options in SharePoint still exists. These options allow the Global Admins or SharePoint Admins to assign someone, or a Security Group, as a Secondary Admin for all ODfB accounts. This option is largely redundant because Global Admins can access the content anyway.]

The default retention period for ODfB content is 30 days after the end user’s account is disabled.

What exactly are you trying to achieve?

As noted, there are some fundamental differences between ‘home drives’ and ODfB.

Consequently, organisations ideally should re-examine their business requirements for access to and the retention of ‘personal content’ both while the user account is active and when it is made inactive, and not assume that old backup option remain valid.

For example, consider the use of backup tapes:

  • The primary purpose of backup tapes is to support disaster recovery. These made sense when IT owned the servers, but it makes less sense when Microsoft own them and are responsible for disaster recovery. Is Microsoft’s disaster recovery capability sufficient or suitable?
  • Backup tapes were (and still are) often used as a type of ‘archive’, allowing organisations to recover data from active and inactive home drives for an indefinite period of time.

The bottom line is – what business outcome/s do you want? Generally, these are likely to be:

  • The ability to recover content stored on personal drives after a disaster (not just when the end-user has deleted something).
  • The ability to access and retain content while the user is active or after they become inactive.

An additional business requirement might be to reduce the use of ‘home drives’ for business related content.

Retention options for content stored in ODfB

ODfB ships with two default retention options:

  • Recycle Bin. Any ODfB content deleted by an end-user goes to the Recycle Bin for 90 days.
  • Inactive content retention. When an end-user accounts is deactivated, the content remains accessible for a default period of 30 days.

Neither of these two options on their own, without modification, is likely to meet business requirements to achieve some form of back-up equivalent capability and the ability to access content in ODfB for a period of time.

It is likely that most business requirements (to replace backups) will be met instead via a combination of the following:

  • Creating a single Office 365 retention policy applied to all ODfB accounts that prevents content in those accounts from being deleted for a given period of time.
  • Extending the default retention period for the content in deactivated accounts from 30 days to a much longer period, for example 7 years.

Office 365 Retention Policy

To ensure that content is kept (and accessible, even after being ‘deleted’ by the user) while the user is active, and after they leave, (a) create a single Retention Policy in the Office 365 Compliance portal, ‘Information Governance’ section and (b) apply it to all ODfB accounts by choosing ‘https://tenantname-mysharepoint.com’.

ODfBRetentionPolicy.JPG

Once published, the retention policy creates a ‘Preservation Hold library’, visible only to the Global Admins, that stores any content that is modified or deleted by the end-user during the retention period.

At the end of the retention period, the content in the Preservation Hold library and anything else that has reached the end of the retention period is sent to the Recycle Bin where it is kept for 90 days before being permanently deleted.

ODfBPresHoldLib.JPG

This type of retention policy effectively replaces the need for a back up of home drives, provided the organisation:

  • Accepts the risk that Microsoft may not be able to recover all or some of the content in the case of a disaster. Note that this risk also applies to Exchange, SharePoint and MS Teams content.
  • Understands that, if it decides to attempt to back up ODfB, restoring from back up may not be as simple as it used to be when the organisation owned and managed the relevant servers. What, exactly, will you back up to, and how will you read the data?

ODfB Storage Retention

The second retention option relates to the ODfB accounts of departed users, or inactive accounts.

ODfB includes the option to retain files in ODfB for a specific period of time after the end-user account is deactivated. This is set in the ODfB Admin portal under ‘Storage’.

ODfBStorage.JPG

At the end of the period of time specified, the content is sent to the Recycle Bin after which it is deleted permanently.

Summary

Many organisations are likely to approach the retention of ODfB content in the same way they did for home drive content, by considering backup options first, often ‘because that’s what we’ve always done’.

Organisations implementing Office 365 should:

  • Define their business requirements for the retention of home drive/ODfB content
  • Examine, understand and consider if retention options in Office 365 result in the same outcome
  • Understand the potential risks of relying on Microsoft to provide a reliable service including in a disaster situation
  • Understand the complexity (and risks) of backing up (and recovering) content from Office 365.

In many cases, retention options in Office 365 may provide the required outcome at a much lower cost.

 

 

 

 

 

Posted in Compliance, Exchange Online, Governance, Information Management, Office 365, Office 365 Groups, OneDrive for Business, Records management, Retention and disposal, SharePoint Online

Managing the outcomes of records retention in Office 365

The retention of records in Exchange Online (EXO), SharePoint Online (SPO), OneDrive for Business (ODfB) and Office 365 (O365) groups can be achieved through the application of retention labels published in the O365 Security and Compliance admin portal.

This post describes:

  • How retention labels work (in summary), including the ‘per record’ rather than the container/aggregation retention model.
  • What happens to content in Office 365 when a retention period expires.
  • The options and actions that may influence the way retention labels/policies are configured, where and how they are applied, and the outcomes required.

The post highlights the need for information and records managers to be involved in all aspects of governance, site architecture and design, and decisions around specific settings and configuration, as well as being assigned specific roles, when Office 365 is implemented.

A quick summary of how O365 retention labels work

Records retention policies in O365 are based on ‘retention labels’ that are created in the O365 Security and Compliance admin portal under the ‘Classifications’ section. Multiple labels can be applied to a single policy.

  • Click this link to read Microsoft’s detailed guidance on retention labels.
  • Click this link to read Microsoft’s detailed guidance on retention policies.

Retention outcomes

Each retention label defines one of three potential outcomes at the end of the retention period, if retention is enabled, ‘keep forever’ is not selected, and the label is not used to classify the content as a record*:

  • The content will be automatically deleted. If the content is in SharePoint, it will first be sent to the Recycle Bin, from which it can be recovered within 90 days.
    • This option may be suitable for certain types of low value records.
  • A disposition review will be triggered to notify specific people. As with the previous point, SharePoint content will be sent to the Recycle Bin if a decision is made to delete it.
    • This option will require additional, human-intervention actions, as described below, if standard records management disposal review processes are followed.
  • Nothing.

Triggers

The date when the above action will occur is based on one of four triggers:

  • Date created
  • Date last modified
  • When labelled applied
  • A event. The ‘out of the box’ (OOTB) event types are:
    • Employee activity. (Processes related to hiring, performance and termination of an employee)
    • Expiration or termination of contracts and agreements.
    • Product lifetime. (Processes relating to last manufacturing date of products).
    • A new event can also be added.
    • See this post for Microsoft guidance on event-driven retention.

An additional alternative option is available: ‘Don’t retain the content, just delete it if it’s older than n days/months/years.’ This is similar to the automatic deletion option above and may be suitable for certain types of records. 

Declaring content as records

* The option to classify or ‘declare’ content as a record is not discussed further as relates to the way records are managed in the US. Microsoft’s guidance on labels notes that: ‘At a high level, records management means that: (a) Important content is classified as a record by users. (b) A record can’t be modified or deleted. (c) Records are finally disposed of after their stated lifetime is past.’ The standard on records management, ISO 15489, defines a record as ‘evidence of business activities, often (but not exclusively) in the form of a document or object, in any form’. This means that anything can be a record. The record may continue to be modified throughout its life. 

When do retention labels become active?

Retention labels become active only when they are published. As part of the publishing process, a decision must be made if the label will apply to all (a single option) or selected parts of the O365 ecosystem:

  • The Exchange Online (EXO) mailboxes of all or specific recipients, or excluding specific recipients.
  • All or specific SharePoint Online (SPO) sites, or excluding specific sites.
  • All or specific OneDrive for Business (ODfB) accounts, or excluding specific accounts.
  • All or specific O365 Groups, or excluding specific groups. Note that content in Microsoft Teams (MS Teams) is included in the O365 Groups options that include both the SharePoint content and email/Teams chat content.

Auto-applying retention labels

Both the retention label and policy sections include the ability to auto-apply a retention policy if certain conditions are met.

  • Sensitive information types. These are the same types that appear in the Data Loss Prevention (DLP) section, for example ‘Financial data’ or ‘Privacy data’.
  • Specific keywords.
  • Content types and metadata (E5 licences only). See this post by Joanne Klein for a description of these options.

The ability of the first two options to accurately identify content and apply a retention policy should be investigated before they are relied on.

When do retention policies start working

According to Microsoft’s guidance Overview of Retention Labels:

If you publish retention labels to SharePoint or OneDrive, it can take one day for those retention labels to appear for end users. In addition, if you publish retention labels to Exchange, it can take 7 days for those retention labels to appear for end users, and the mailbox needs to contain at least 10 MB of data.

  • In EXO, the default MRM policy needs to be removed before the new policy applies.
  • In ODfB, the policy is available to be manually applied on folders or documents. It does not automatically apply to content.
  • In SPO, the policy can be applied to document libraries or documents. To avoid removing the ability for users to legitimately need to delete documents in an active library it is recommended to apply the policy after the document library has ceased to be active.
  • Content in Office 365 Groups is covered by either the EXO (for email/teams chat content) or the SPO policy (applied to libraries).

Retention labels apply to individual records within aggregations

Records labels can be applied to aggregations of records (an entire email mailbox or folder, a SharePoint library or list, an ODfB account, O365 Groups) or individual records. However, the disposal process targets individual records (e.g., individual emails, single documents in SharePoint libraries, individual list items). 

That is, even when all the individual records are disposed of, the parent aggregation remains in place without any indication that the records previously stored in it (sometimes known as a ‘stub’) have been destroyed. 

This outcome has implications for the way the outcome of a retention label is set. It requires a choice between (a) delete automatically without review or (b) review before delete.

The latter option is made complicated by the requirement to review individual documents, including potentially in the original container (document library in SPO) and export metadata relating to those records if a record of the deletion is to be retained.

What happens when records reach the end of their retention period

As noted above, the outcome at the end of the retention period (trigger date + n days/months/years) will depend on the settings on the label.

  • Where the label was applied (EXO mailbox, SPO library or list, ODfB folder or document, O365 Group)
  • Whether the records would be deleted automatically or be subject to a disposition review.

If the records are to be deleted automatically:

  • SPO and ODfB records will be sent to the site/ODfB Recycle Bin for 90 days
  • EXO emails will be moved to a ‘Cleanup’ area for 14 days, before permanent deletion.
  • Aside from the audit logs (which by default only go back 90 days), no other record will be kept of the destroyed records.

If the records are subject to a disposition review, an email is sent to the person nominated. When that person clicks on the link in the email they are taken directly to the ‘Dispositions’ sub-section of the Records Management section of the O365 Security and Compliance centre.

It is arguable that retention policies with disposition review should not be applied to ODfB content as this will require the reviewer to review all the content that has been labelled by a user in their ODfB account.

  • For more information about this subject see this Microsoft page ‘Overview of disposition reviews‘. Microsoft note, on that page ‘To get access to the Disposition page, reviewers must be members of the Disposition Management role and the View-Only Audit Logs role. We recommend creating a new role group called Disposition Reviewers, adding these two roles to that role group, and then adding members to the role group.’

The dispositions dashboard shows the number of records that are pending disposition against each retention policy label:

O365_RM_DispositionsDashboard

Pending disposition tab

When the reviewer clicks on one of the retention policies listed, the following view opens for records ‘Pending disposition’:

O365_Dispositions_Pending

An important point to note here is that records are listed individually, not in logical aggregations or collections. It is possible however to use the Search option on the left to filter by author (emails) or SharePoint site and/or site library. It is also possible to export the details (which does not include any unique metadata applied to documents in SharePoint libraries).

All the records displayed may then be selected and a ‘Finalise decision’ dialogue box appears with the following options:

  • Dispose of the records.
  • Extend the retention.
  • Re-label the records.

Disposed items tab

The Dispositions dashboard includes a ‘Disposed items’ tab.

Microsoft note that this tab ‘… shows dispositions [that] were approved for deletion during a disposition review and are now in the process of being permanently deleted. Items that had a different retention label applied or their retention period extended as part of a review won’t appear here.’

Importantly, once records are permanently deleted, they no longer appear in the ‘Disposed Items’ tab. This means that no record will be kept of the records that were destroyed.

Shortcomings of the O365 dispositions/disposal model for records stored in SPO

Only individual records appear, not all the items in a document library

If the retention outcome is based on the ‘created’ or ‘last modified’ date, individual records in SPO document libraries will start to appear as soon as they reach the retention end date. The reviewer may need (or want) to view the original library, which they can identify from the link is in the dispositions review pane.

Retention policies prevent deletion

As a retention label prevents the deletion of content by users, and this may put them off using SharePoint, it is recommended that retention in SPO document libraries be based on when the label was applied NOT when it was created or last modified. This will help to ensure that all documents appear in the disposition review area at the same time.

Event based triggers may not be suitable for disposition review

If the retention outcome is based on an event, or is auto applied and a disposition review is required, those records will appear randomly when the event is triggered. It could be difficult for records managers to decide the disposal outcome in this way without referring back to the library.

The dispositions review pane does not display the original metadata

The  dispositions review pane displays only very basic metadata from the original library. Again, the reviewer may need to view the original library, export the metadata and store that in a secure location. Note that the exported metadata includes the URL of each original record including the library name.

The document library remains even when all contained records are destroyed

If the reviewer chooses to dispose of the records listed, only the content of the library (the individual documents stored in it) is deleted, not the actual library itself. No record (e.g., a ‘stub’ of the deleted item) is kept in the library of the deleted content.

The ‘Disposed items’ tab only shows records being destroyed

The ‘Disposed items’ tab only shows records in the process of being destroyed. It does not keep a record of what was destroyed. Records managers will need to retain the metadata of what was destroyed, when, based on what disposal authority, and with whose approval.

Dispositions really only provides a ‘heads up’ for further action

The Dispositions process may be instead used as a form of ‘heads up’ that records are starting to be due for disposal in a document library. This would allow the records managers (who should be Site Collection administrators) to review the library, export the complete set of metadata, and decide if the entire library can be deleted since it is no longer required.

Conclusions

Retention labels in O365 are an effective way of managing the retention and disposal of records in that environment, subject to the following points.

Email

Emails will likely continue to be managed as complete aggregations of records – the mailbox. Users cannot be expected to create logical groupings and apply individual retention labels to those records.

Organisational records policies may mandate specific timeframes for the retention of email (e.g., 1 year), while HR/IT security policies may mandate that whole mailboxes are retained for a period of time after employees leave. It is important to understand the difference between these two models

Options to automatically transfer emails to SharePoint document libraries via rules may be possible using Flow but these rely on individual users to set up.

Consideration should instead be given to using O365 Group mailboxes, rather than individual personal mailboxes, for specific work related matters. For example, ‘Customer Complaints’, or ‘XYZ Project’.

OneDrive for Business Accounts

ODfB accounts may be covered by two forms of retention:

  • Retention labels that apply to all ODfB accounts while the account is active. These must be manually applied by users.
  • A separate retention period set for ODfB accounts after a user leaves the organisation.

If there is a requirement to prevent the deletion of content by a user from their ODfB account, the better way to achieve this is using an eDiscovery case with Legal Hold applied.

SharePoint Online

As most records will be stored in SharePoint document libraries (including Office 365 Group-based SP libraries), multiple retention labels will be required to address different types of content or retention requirements.

Careful consideration should be given to whether records can be deleted automatically at the end of the retention period or should be subject to disposition review, noting that the automatic deletion provides no opportunity to capture the metadata of the records.

The ‘auto-apply’ or event-based retention option should be used sparingly to avoid a trickle of records for disposal – unless there is enough trust that these can be accurately marked and deleted without review.

Shortcomings in the disposition review process support the following decisions for SharePoint Online content:

  • The number of retention labels should be minimised to avoid a very long drop-down menu when a label is applied. If current record retention or disposal authorities contain a lot of classes, some of these could potential be combined into a single class (e.g., ‘Company Records – 7 years’), while the site name and document library name should provide some context to the content to ‘map’ back to the original classes.
  • Retention labels should be applied when document libraries (or lists) become inactive as this will avoid conflict with users who want to delete content and also ensure that documents are ready for disposition review at the same time.
  • Retention labels applied to SPO document libraries should include the disposition review option unless a ‘delete only’ label is considered suitable for certain document libraries that clearly contain working documents or Redundant, Outdated and Trivial (ROT) content.
  • Records managers should review the content of all or most original SPO document libraries, and export the metadata of those libraries for storage in a separate location (such as an ‘archives’ site), or in the original library with the retention label changed to ‘Never Delete’. The original document library can then be deleted.
Posted in Access controls, Electronic records, Office 365, Office 365 Groups, OneDrive for Business, Products and applications, SharePoint 2013, SharePoint Online

Migrating to SharePoint Online Part 2 (implementation)

In my previous post I described what we did to prepare for the migration of our SharePoint 2013 (SP2013) environment to SharePoint Online (SPO). In this post I describe the process we undertook and the lessons we learned along the way.

By August 2017 we had around 245 SP2013 sites across seven web applications: apps (13 ‘purpose-built’ sites); intranet (1 site); ipform (an old site that was closed several years back); projects; publication (30 sites); sptest (used for testing sites); and team.

The bulk of our sites (around 210 sites split almost equally) containing most of our corporate records were in either the teams and projects web applications.

The details of our root sites were recorded in several key artefacts:

  • A SharePoint Online list in our SharePoint Admin site, used for new site requests. This was always our ‘master’ listing of sites and included a range of additional metadata, including the business owner and a ‘yes/no’ if the site had been migrated or not. This was one of the first sites we migrated.
  • Another SharePoint list that recorded the details of site collections and subsites.
  • An Excel spreadsheet used to ‘map’ of all our root sites (one per cell_ grouped under business areas. This map provided a simple, printable visual map of all our SP2013 sites grouped by business area. We used colours to indicate when sites were migrated to SPO, providing an immediate visual reference.

Configuring and learning about Office 365 Admin and SharePoint Online (and OneDrive) admin

By August 2017 we had access to our Office 365 tenant admin environment, access to which is required to get to the SharePoint Admin portal initially (subsequently, the SharePoint Service Administrator could access it directly).

After setting up and configuring the Office 365 elements and SharePoint Online (SPO) environment, we created some initial test sites (via the Admin portal) to understand the new environment.

One of the early SPO sites was a re-created SharePoint User Group site, used to store general training and other useful information about the new environment. This site has remained our primary go-to point for all SharePoint users.

Our SharePoint developer also re-created various scripts, including scripts to automatically create and configure new sites from a request in a SharePoint Online list.

Monitoring changes in Office 365 and SharePoint Online

We learned the importance of monitoring – daily – the Office 365 message centre and also the Microsoft techcommunity site (which had been moved off a Yammer platform a year or so earlier), as well as the Microsoft Office 365 roadmap to ensure we were aware of any likely changes – many of which were introduced during the time we were migrating.

We quickly learned a few things:

  • Customisation was not a friend of migration. Fortunately, almost none of our sites were customised. However, page-based content would need to be re-created.
  • Any complex workflows, integration or data extraction (e.g., ETL for business intelligence purposes which needed to be re-linked) could delay migrations. As it turned out, these sites ended up at the very end of the migration process and a couple were still waiting for migration as at the date of this post.
  • New ‘modern’ sites based on Office 365 Groups needed careful planning to get them right early on. We decided that any request for an Office 365 Group would go through the same request process as SharePoint requests.

Towards the end of 2018, when they were introduced, we also learned that hub sites were preferred over subsites.

Project management

While the migration of SharePoint sites was included as part of an internal IT project, that project was focussed for most of the first year on a range of other core networking elements including the broader network architecture model and high level designs required for our new cloud environment.

It would only later start work on the migration of Exchange mailboxes to Exchange Online and personal drives to OneDrive.

SharePoint migrations continued through the life of the project.

Migration tool

There were a number of options to migrate our SP2013 sites to SPO. We decided against going with an external provider for a number of reasons and instead – after reviewing the market – acquired the ShareGate migration tool in September 2017.

Final architecture

By September 2017 we had finalised the architecture for our SPO environment. As we had been using web applications in our on-premise environment we needed to ‘map’ this to the new environment.

The new model was based on the following:

  • All team and project sites would be created under the ‘/teams’ path. Project sites would have the prefix ‘PRJ’ to identify them. (Some would also be created as Office 365 Group-based sites, with ‘O365_PRJ’ as the prefix). ‘Team’ sites had to be based on a logical business unit or team.
  • All other sites, including communication sites and sites that crossed over multiple business areas would be created under ‘/sites/’.

SharePoint migrations were ready to go.

First batch

Our earlier analysis indicated that around 50 of our SP2013 project sites were inactive (because the projects had since closed). As no-one was accessing any of these sites we decided to use the ShareGate tool to test the migration process and learn from the experience.

We migrated 51 project sites in October 2017. The migrations initially took place during the day but we then changed to an early morning migration (before 9 AM usually) to avoid any network traffic issues.

First lessons learned

The ShareGate tool worked as expected, and proved to be a very useful tool for other reasons too, such as moving libraries and lists between sites.

The early batch of SP2013 sites were migrated ‘as is’ in terms of their look and feel. They looked exactly the same but were now in SPO. That is, they did not get the new ‘modern’ page look and were not mobile friendly. This didn’t matter too much as the sites were rarely accessed.

After the first batch, we did the following for all new sites:

  • Added a new page (named ‘home2’) and swapped over the old ‘classic’ page (renamed to ‘homex’) with the new one.
  • Edited the replacement home page and add any text/images from the old site home page.
  • Fixed up the left hand navigation; indented libraries and lists on old sites were now under a heading with a drop down menu option. In some cases we left them ‘as is’, in other cases we promoted the indented libraries via the left hand ‘Edit’ option.

For any sites that had the publishing feature enabled on the site collection and site settings, we disabled this on the SPO site post-migration as there was generally no need to have these settings enabled.

Some sites had Active Directory security groups in their SharePoint permission groups. As these were not migrated to our Office 365 environment (for multiple reasons, including the complexity of this legacy environment), these had to be added back in a different way to provide the same level of access. In almost all cases, existing SP permission groups (Owner, Member, Visitor) were sufficient. The primary one we had to re-create was the AD Group for ‘everyone’; this was replaced by the ‘Everyone except external users’ option.

The other factor we had to consider were Office 365 licences. By October 2017 few people had these licences. Anyone who needed to access SharePoint would need a licence, but these might not be issued to everyone until mid 2018. This limited the number of sites that we could migrate. By mid 2018, more or less anyone who accessed SharePoint 2013 before could now access the SPO environment.

Next batch – to end June 2018

From November 2017 to end June 2018 we migrated another 57 sites, including a further 16 inactive project sites in December 2017. The primary reason for the low number was (as noted above) the need for staff to have Office 365 licences, which were not allocated more broadly until mid 2018.

At the same time, however, we were also starting to create a range of new SPO sites, including Office 365 Group-based sites and new communication sites.

Publication sites re-created as communication sites

Almost none of our publication sites could be migrated ‘as is’ to SharePoint Online because the page-based content, while it could be migrated, was not in modern pages or mobile friendly.

Accordingly, it was decided to re-create all the publication sites as SPO communication sites. In almost all cases this was a relatively simple process of creating pages and copying content from the old to the new.

Our intranet was the only except to this process and, as of end 2018, remains as a SP2013 site because of heavy customisation.

Other project impacts

From July 2018 two key projects impacted on the SharePoint migrations.

The first was the roll-out of new Windows 10 devices. While a Windows 10 device was not required to access SharePoint, some of our older Windows 7 devices had Internet Explorer 9 that had issues with SPO. These users were asked to use Chrome instead.

The second related project work (which was part of the overall Office 365 project that included SharePoint migrations) was the migration of Exchange mailboxes and personal drives to Exchange Online and OneDrive respectively. This part of the project encountered a problem with Windows 7 devices and as a result that part of the project activity was delayed.

Final migrations – from July 2018

From July to the end of 2018 we migrated all except around 10 of the remaining sites, at an average rate of around 20 per month. Many of these were simple migrations.

For each migration we followed the same process:

  • Engaged with the business area to provide awareness of, and where required, training in, the new environment. (By the end of 2018, this training included information about Office 365 and MS Teams to help site owners understand that SharePoint is not an ‘isolated’ product as it was before, but part of a much larger ecosystem)
  • Identified a suitable date and time to migrate the site (most of these were completed before 8 AM on a working day, but some were done over a weekend)
  • Alerted our Service Desk to the proposed change
  • Migrated the site
  • Made minor changes to the site (home page swaps mostly)
  • Made the old site read only with a pointer to the new site
  • ‘Released’ the migrated site to the business area, usually before 9 AM.
  • Provide post-migration support to the business area. In most cases the business area was able to use the new site immediately as the new site was very similar to the old one in look and layout.

The last sites to migrate included sites with complex workflows, integration or ETIL elements and several large, complex and sensitive sites.

New site request process

While we had always had a SharePoint-based site request form, the new environment meant that we needed an updated form. The (SPO) online form changed several times during 2018 as we learned from experience what worked and what didn’t.

The current form captures the following (not all columns/fields are listed):

  • Site Acronym (up to 12 characters – becomes the DocID prefix)
  • Site Type: (a) team or project site (no Office 365 Group), (b) team or project site (with Office 365 Group, (c) communication site
  • Approver
  • Business area owner
  • Owner/s
  • Member/s
  • Sensitivity (information security)
  • Suggested site URL

Each of these requests was reviewed by the SharePoint administrators who, if the site request is OK, then run a workflow for non Office 365 Group-based sites only. For Office 365 Group based sites, these were created by creating the Office 365 Group.

By the end of 2018 we had approximately the same number of migrated sites as new sites and our SPO environment was ‘live’ and active.

Almost all the old SP2013 sites were made read only. We expect that these will be deleted by June 2019.

Posted in Compliance, Data Loss Prevention - DLP, Exchange Online, Governance, Information Classification, Information Management, Information Security, Legal, Office 365, OneDrive for Business, Products and applications, Security, SharePoint Online, Training and education

SharePoint Online and OneDrive for Business – Preventing external sharing of data

A recent (September 2017) article suggested that OneDrive for Business (ODfB) (and by extension SharePoint Online (SPO); ODfB is a SharePoint-based service), a key application in Office 365 was a potential source of data leaks and/or target for hacking attacks.

I don’t disagree that, if not configured correctly, any online document management system – not just ODfB/SPO – could be the source of leaks or the target of external attacks. Especially if these systems, and the security controls that can protect the data in them, are not properly configured, governed, administered, and monitored.

But, I would ask, what controls do most organisations have in place now for documents stored in file shares and personal file folders, not to mention USB sticks, and the ability to send document via Bluetooth to mobile devices or upload corporate data to third-party document storage systems? Probably not many, because users have no other way to access the data out of the office.

As we will see, the controls available in Office 365 are likely to be more than sufficient to allow users to access to their documents out of the office, while at the same time reducing (if not eliminating) the sharing of documents with unauthorised users.

How to stop or minimise sharing from OneDrive for Business and SharePoint Online

There is one simple way to prevent the sharing of data stored in SPO and ODfB with external people – don’t allow it.

There are several ways to control what can be shared, each allowing the user a bit more capability. All these options should be based on business requirements and information security risk assessments, and Office 365 configured accordingly.

In this article I will start with no sharing allowed, and then show how the controls can be reduced as necessary.

External sharing – on or off

This is the primary setting, found in the main Office 365 Admin centre under Settings > Services & add-ins > Sites. If you turn this off, no-one can share anything stored in SPO or ODfB.

The option is shown below:

O365_SC_Sites_SharingOnOff

If you do allow sharing, you need to decide (as shown above) if sharing will be with:

  • Only existing external users
  • New and existing external users [Recommended]
  • Anyone, including anonymous users

The second option is recommended because it doesn’t restrict the ability to share with new users. The last option is unlikely to be used in most organisations and comes with some risks.

The next place to set these options are in the SPO and ODfB Admin centres.

OneDrive admin center

If the previous option is enabled, the following options are available for ODfB. Note that BOTH SharePoint and OneDrive are included here because the latter is a part of the SharePoint environment.

  • Let users share SharePoint content with external users: ON or OFF.
    • NOTE: If this option is turned OFF, all the following options disappear.
  • If sharing with external users is enabled, the following three options are offered:
    • Only existing external users
    • New and existing external users [Recommended]
    • Anyone, including anonymous users
  • Let users share OneDrive content with external users: ON or OFF
    • This setting must be at least as restrictive as the SharePoint setting.
  • If sharing with external users is enabled, the following three options are offered
    • Only existing external users
    • New and existing external users [Recommended]
    • Anyone, including anonymous users

If sharing is allowed, there are three sharing link options:

  • Direct – only people who already have permission [Recommended]
  • Internal – only people in the organisation
  • Anonymous access – anyone with the link

You can limit external sharing by domain, by allowing or blocking sharing with people on selected domains.

External users have two options:

  • External users must accept sharing invitations using the same account that the invitations were sent to [Recommended]
  • Let external users share items they don’t own. [This should normally be disabled]

A final ‘Share recipients’ checkbox allow the owners to see who viewed their files.

SharePoint admin center

The SPO admin center (to be upgraded in late 2017) has two options for sharing.

The first option is under the ‘sharing’ section which currently has the following options:

Sharing outside your organization

Control how users share content with people outside your organization.

  • Don’t allow sharing outside your organization
  • Allow sharing only with the external users that already exist in your organization’s directory
  • Allow users to invite and share with authenticated external users [Recommended]
  • Allow sharing to authenticated external users and using anonymous access links

Who can share outside your organization

  • [Checkbox] Let only users in selected security groups share with authenticated external users

Default link type

Choose the type of link that is created by default when users get links.

  • Direct – only people who have permission [Recommended, same as above]
  • Internal – people in the organization only
  • Anonymous Access – anyone with the link

Default link permission

Choose the default permission that is selected when users share. This applies to anonymous access, internal and direct links.

  • View [Recommended]
  • Edit

Additional settings (Checkboxes)

  • Limit external sharing using domains (applies to all future sharing invitations). Separate multiple domains with spaces.
  • Prevent external users from sharing files, folders, and sites that they don’t own [Recommended]
  • External users must accept sharing invitations using the same account that the invitations were sent to [Recommended]

Notifications (Checkboxes)

E-mail OneDrive for Business owners when

  • Other users invite additional external users to shared files [Recommended]
  • External users accept invitations to access files [Recommended]
  • An anonymous access link is created or changed [Recommended]

Sharing via the Site Collections option

In addition to the options above, sharing options for each SharePoint site are set in the ‘site collections’ section as follows. Note that the default is ‘no sharing allowed’. A conscious decision must be taken to allow sharing, and what type of sharing.

O365_SPO_Sharing1

When a site collection name is checked, the following options are displayed.

Sharing outside your company

Control how users invite people outside your organisation to access content

  • Don’t allowing sharing outside your organisation (default)
  • Allow sharing only with the external users that already exist in your organization’s directory
  • Allow external users who accept sharing invitations and sign in as authenticated users
  • Allow sharing with all external users, and by using anonymous access links

If anonymous access is not permitted (setting above), a message in red is displayed:

Anonymous access links aren’t allowed in your organization

SharePoint Sharing option

The SharePoint Admin Centre has an additional ‘Sharing’ section with the same settings as shown above for ODfB. It is expected that these multiple options will be merged in the new SharePoint Admin Centre due for release in late 2017.

Additional security controls

In addition to all the above settings, there are a range of additional controls available:

  • All user activities related to SPO and ODfB, including who accessed, viewed, edited, deleted, or shared files is accessible in the audit logs.
  • SPO and ODfB content may be picked up by Data Loss Prevention (DLP) policies and users prevented from sending them externally. This is of course subject to the DLP policies being able to identify the content correctly.
  • SPO and ODfB content may be subject to records retention policies set by preservation policies. These may impact on the ability to send documents externally.
  • SPO and ODfB content may be subject to an eDiscovery case.
  • Administrators can be notified when users perform specific activities in both SPO and ODfB.
  • Sharing (and access to the documents once shared) may be subject to security controls enforced through Microsoft Information Protection.

Conclusion

In summary, the settings above allow an organisation to strongly control what can be shared. If sharing is allowed, certain additional controls determine whether the sharing is for internal users or for users external to the organisation. If the latter is chosen, there are further controls on what external users can do. Audit controls and policies may also control how users can share information externally.

The key takeaway is that organisations should ensure that the sharing options available in Office 365 are based on the organisation’s business requirements and security risk framework.