Posted in Compliance, Data Loss Prevention - DLP, Electronic records, Governance, Information Classification, Information Management, Information Security, Legal, Microsoft 365, Records management, Retention and disposal, SharePoint Online

What happens if you mix label-based retention policies and non-label retention policies on the same SharePoint site?

Two types of retention policy can be created in Microsoft 365:

  • Label-based retention policies, where the label is used to define the retention and retention outcomes. Labels must be published in a retention policy, a process that includes determining where the labels will be applied and appear (‘explicit’) to end users.
  • Non-label-based retention policies, where the policy includes the retention details and the outcomes. As part of the policy creation, these policies are then applied to specific Microsoft 365 workloads where they are mostly invisible to end-users (except in Exchange mailboxes). In SharePoint and OneDrive for Business, these policies create a Preservation Hold library that is only visible to Site Collection Admins and above.

It is possible to apply both a label-based retention policy and a non-label retention policy to the same SharePoint site. In theory, this would allow for (a) everything on the site to be covered by an overarching retention policy and (b) specific libraries or lists to be covered by a label-based policy.

In practice, it gets a little complicated, as described in this post.

Creating the two labels

For the purpose of this post, I will apply the two types of policy to a SharePoint site (‘FinanceAP’) that contains specific types of financial information that needs to be kept for 7 years, but I want to allow other content on the site to be destroyed after 5 years.

Label-based policy

Retention labels are created in the Information Governance section of the Compliance admin portal in Microsoft 365. I created a label titled ‘Financial records’ with a retention period of 7 years. I then published that label to a retention policy named ‘Financial Records – 7 years’ and applied it only to the FinanceAP site.

More than one label can be published in the same policy, making this a useful option if your SharePoint architecture ‘maps to your file plan or Business Classification Scheme (BCS) and your records retention classes are based on either. It also allows you to create and add the same retention class for types of records that occur in multiple functions where the classes have the same retention – for example, ‘Meetings – 7 years’ or ‘Policy – 10 years’.

Once the policy has been published to a site or sites, the option (in Library Settings) to ‘Apply label to items in this list or library’ can be used to choose which label will apply to the content in the library, as shown below.

If the column ‘Retention label’ is checked, the retention label name appears in that column.

Non-label retention policy

Non-label retention policies are also created in the Information Governance section of the Compliance admin portal which also (a little confusingly) lists all the label-based policies as well.

The process of creating these policies includes the retention (e.g, 5 years) and retention outcome (delete) definitions, as well as the location where the policy will be applied.

For the purpose of this post I created a retention label named ‘Financial Working Records – 5 years’ and applied it to the same site (only) as the label-based policy.

I should expect now to find a Preservation Hold library (via Site Contents as a SharePoint admin) when something is deleted.

At this point, I have two retention policies, (a) one label-based and applied to the site, and (b) one that applies to the whole site.

What happens now?

In the document library where the label-based policy has been selected, I can see that the retention label (Financial Records) that has been applied to items in this library.

This means that I cannot delete this document unless (as an end-user with edit rights or admins) the retention label is removed. However, as we will see below, another policy is working behind the scenes.

In a document library where no label-based policy has been applied, I can see that no label appears under the Retention label policy. From an end-user point of view, it appears that the record can be deleted – or is it?

As this site is the subject of an ‘implicit’ or invisible retention policy that has been applied to the entire site, any attempt to delete anything will be captured by the back-end Preservation Hold library seen below via Site Contents (visible to Admins only).

Interestingly, any attempt to delete a document from a library where a label-based retention policy has been applied, which is ‘denied’ in the actual library, is recorded in the Preservation Hold library, although the document remains in the original library.

If anyone with access to the Preservation Hold library tries to delete that item there, they will receive this message:

The only way to remove this item is to remove the policy.

Posted in Compliance, Conservation and preservation, Electronic records, Governance, Information Management, Information Security, Legal, Records management, Retention and disposal, Security

Destroying digital records – are they really destroyed?

Most people should be aware that pressing the ‘delete’ option for a file stored on a computer doesn’t actually delete the item, it only makes the file ‘invisible’. The actual file is still accessible on the disk and can be retrieved relatively easily or using forensic tools until the space it was stored on is overwritten.

Traditional legacy electronic document and records management (EDRM) systems have two components:

  • A database (e.g., SQL, Oracle) where the metadata about the records are stored
  • A linked file share where the actual objects are stored, most of which are copies of emails or network file share files that remain in their original location.

In most on-premise systems, email mailboxes, network file shares, and the EDRMS database and linked file share are likely to be backed up.

When a digital record comes to the end of its retention and is subject to a ‘destruction’ process, how do you know if the record has actually been destroyed? And even if it is, how can you be sure that the original isn’t still stored in a mailbox, network file share, or a back up?

This post examines what actually happens when a file is ‘deleted’ from a Windows NT File System (NTFS), and questions whether digital records stored in an EDRMS are really destroyed at the end of the retention period.

The Windows NTFS Master File Table (MFT)

Details of every file stored on a computer drive will be found in the NTFS Master File Table (MFT).

In some ways, the MFT operates like a traditional electronic document management system – it is a kind of database that it records metadata about the attributes of the digital objects stored on the drive. These attributes include the following:

attriblist

As noted in the diagram above, the details stored by the MFT include the $File_Name and $Data attributes.

  • The $File_Name attributes include the actual name of the file as well as when it was created and modified, and its size.  This is the information that can be seen via File Explorer and is often copied to the EDRMS metadata.
  • The $Data attribute contains details of where the actual data in the file is stored on the disk (in 0s and 1s) or the complete data if the file is small enough to fit in the MFT record.

If the MFT record has many attributes or the file data is stored in multiple fragments on a disk (for example as a file is being edited), additional MFT ‘extension’ records may be created.

When a file is deleted, the MFT records the deletion.

  • If the file is simply deleted, the record will remain on the disk and can be recovered from the Recycle Bin.
  • If the file is deleted through SHIFT-DEL or emptying the Recycle Bin, the MFT will be updated to the ‘Deleted’ state and update the cluster bitmap section to set the file’s cluster (where the data is stored) as being free for reuse. The MFT record remains until it is re-used or the data clusters are allocated in whole or part to another file.

So, in summary, ‘deleting’ a file does not actually delete it. It may either:

  • Store the file in the Recycle Bin, making it relatively easy to recover, or
  • Change the MFT record to show the file as being deleted but leave the file data on the desk until it is overwritten.

How does an EDRMS store and manage files?

The following summary relates to a well-known Electronic Document and Records Management System (EDRMS). Other systems may work differently but the point is that records managers should understand exactly how they work and what happens when electronic files are destroyed at the end of a retention period.

Most EDRM systems are made up of two parts:

  • A database (SQL, Oracle etc) to store the metadata about the record.
  • An attached file store that stores the actual digital objects.

When EDRM systems are used to register paper or physical records (files and boxes), only the database is used.

When digital records are uploaded to the EDRMS:

  • The metadata in the original file, including the file type, original file name, date created, date modified and author are ‘captured’ by the system and recorded in the new database record.
  • Additional metadata may be added, including a content or record ‘type’.
  • The record will usually be associated with a ‘container’ (e.g., ‘file’). This containment makes the record appear to be ‘contained’ within that container, whereas in fact it is simply a metadata record of an object stored elsewhere.
  • The original record filename is changed to random characters (to make it harder to find, in theory) and then stored on the attached (usually Windows NTFS) file store, often in a series of folders.
  • A link is made between the database record and the record object stored in the file store (the MFT record).

When the end-user opens the EDRMS, they can search for or navigate to containers/files and see what appears to be the digital objects ‘stored’ in that container/file. In reality, they are seeing a link to the object stored (randomly) in the file store.

What happens when an EDRMS record is destroyed?

If there is no requirement to extend their retention, or keep them on a legal hold, records may be destroyed at the conclusion of a retention period.

For physical records, this usually means destroying the physical objects so they cannot be recovered, a process that may include bulk shredding or pulping.

For digital records, however, there may be less certainty about the outcome of the destruction. While the EDRMS may flag the record as being ‘destroyed’ it is not completely clear if the destruction process has actually destroyed the records and overwritten the digital records in a way that ensures its destruction to the same level as destroyed paper files. 

Also:

  • If the original associated NTFS file share becomes full and a new one is used, the original is likely to be made read only.
  • There is likely to be a backup of the EDRMS.
  • The original records uploaded to the EDRMS probably continue to exist on network files shares, in email, or in back up tapes.
  • Digital forensics can be used to recover ‘deleted’ files from the associated file share.

Consider this scenario:

  • An email containing evidence of something is saved to a container in an EDRMS.
  • The container of records is ‘destroyed’ after the retention period expires.
  • A legal case arises after the container is ‘destroyed’
  • A subpoena is made for all records, including those specific records.
  • Has the record actually been destroyed, or could it still be recoverable, including from backups or the digital originals?

Is it really possible to destroy digital records, and does it matter?

Yes, records can be destroyed by overwriting the cluster where the record is kept, and some EDRM systems may offer this option.

But:

  • Do EDRM systems overwrite the cluster when a digital record is destroyed in line with your records retention and disposal authorities, or simply mark the record as being deleted, when it is still technically recoverable?
  • Could the record still exist in the network file shares or email, or in backups of these or the EDRMS?
  • Might it be possible to recover the record with digital forensics tools?
  • Does it matter?

It might be worth asking IT and your EDRMS vendor.

References:

 

 

Posted in Disasters, Electronic records, Information Management, Information Security, Legal, Office 365, Records management, Retention and disposal, SharePoint Online, Training and education

Why is it so hard to ‘go digital’?

I visited a local fast-food outlet recently and could not help but notice the ‘Lever Arch’ binders in the small office behind the counter. A small two-drawer filing cabinet was also located below the desk.

20191002_125518

It made me wonder – in this day and age when pretty much everyone has access to the internet including via their smart phone, why are there any paper records?

And, why is it so hard to ‘go digital’, when so many better and safer digital options are available?

Reasons for not going digital

People probably want to keep paper records in this digital age for a few fairly common reasons, all of which I’ve encountered over the years.

  • Ease of access. It is much ‘easier’ to access a record if it’s in the folder with an obvious name, like ‘Rosters’.
  • Speed of access. You can access a paper record in a couple of seconds. Accessing the same record on a computer means logging on then searching or navigating to where it is stored (potentially including on personal removable storage devices).
  • Easier to archive. At the end of a given period the records can ‘simply’ be placed in an archive box and sent off for archiving.
  • Keeping digital records is too ‘hard’.
  • The company doesn’t offer any other option.
  • ‘Computers are hard’.
  • No obvious or pressing business reason to go digital.
  • A preference for paper, or belief that paper records must be kept.

Which of the above have you encountered? Let me know via this anonymous Form:

Or click this link:

https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAN__td1WRVUM0hJM0g2Q1NCWFdLS0JYM0k5QUlOUVUxRC4u

Keeping paper records can be risky

Keeping paper records can be all well and good, unless this sort of thing happens:

burger-king-fire-hed-2017-1260x840
Source: https://finance.yahoo.com/news/burger-king-used-photos-real-105654804.html

If you keep paper records when better digital options exist, you are taking a calculated risk that doing so is ‘OK’.

Of course, not all businesses (a) store the only copy of their physical records locally or (b) burn down (including by being constructed in fire-prone areas). However, these are not the only risks. Other risks include:

  • Flooding, from burst pipes, storms, or floodwaters. Water-damaged records are not easy to recover.
  • Damage from falling objects, including trees or other objects falling from the sky.
  • Theft or vandalism.
  • Business closure and leaving records behind in the abandoned building.
  • Any combination of the above.

What’s the back up for physical records?

What’s the back up for these paper records when disaster strikes?

Generally, unless the physical records have been transferred off-site, or they are the printed version of a digital original that can still be accessed, there isn’t one.

Is there a better, digital way?

Yes.

Printed records are likely to fall into several broad categories, each of which can be managed in their own way. For example, in the business above:

  • Policies and procedures, including ‘operating manuals’ and similar types of instructions are likely to be the printed version of digital originals. They can be made available on the company intranet or, if one doesn’t exist, sent via email.
  • Financial records (e.g., invoices). Again, these are likely to be the printed version of a digital original. If they were in printed form when received (e.g., by mail, with a delivery), the company should (a) ask for digital copies to be sent by email, or (b) scan them and store them digitally.
  • Rosters and general documents relating to groups of employees (as opposed to individual staff ‘files’). Rosters could still be printed for display purposes, but the original should be kept in digital form.
  • Staff files. The format of these may depend on the organisation, but there should be no reason for ‘local’ staff files to be kept in an organisation that has a centralised HR system.
  • Other types of business documents. If necessary, these could be scanned and kept in digital form.

And, of course, all of these could be kept in Office 365, including SharePoint for document storage and MS Teams for teams chat, including for front line workers.

Additional training and support may be required to help these areas ‘go digital’.

 

 

Posted in Classification, Compliance, Information Classification, Information Management, Office 365, Products and applications, Records management, Retention and disposal, Security

Office 365 Security and Compliance – classification label changes

Microsoft have improved the Classification section in the Office 365 Security and Compliance centre. The change will help to reduce confusion and make it easier for records managers and security administrators to focus on their individual needs.

Previous user interface

The primary change is to the menu interface. The previous menu options, shown in the screenshot below, showed only ‘Labels’ and ‘Label policies’.

O365_Classifications_Labels

When the previous ‘Labels’ option was selected, a new screen with two tabs ‘Sensitivity’ (default) and ‘Retention’ was displayed, as shown below.

O365_Classifications_Labels

The sensitivity or retention tab had to be selected to create or publish a new label. The user interface was unclear and the difference between creating and publishing a label was not obvious.

New user interface

The sensitivity and retention elements have now been separated and placed under the primary ‘Classification’ menu option as shown below.

O365_Compliance_ClassificationLabels23Aug19.JPG

Now, ‘Labels’ and ‘Label policies’ are two tabs under the relevant section as can be seen below.

 

O365_Compliance_ClassificationLabelsRetention23Aug19

O365_Compliance_ClassificationLabelsSensitivity23Aug19

The options to create and publish labels remain the same.

Posted in Information Management, Information Security, Products and applications, Records management, SharePoint 2013, SharePoint Online

Migrating to SharePoint Online – Part 1 (Planning)

We implemented SharePoint 2010 in early 2012 and then upgraded to SharePoint 2013 in early 2015. After acquiring Office 365 enterprise licences in April 2016 we began to play for the migration of our existing on-premise environment to SharePoint Online. After testing the migration process with inactive sites, we started to migrate active sites from early 2018. We expect to complete all the migrations by 31 December 2018.

This post, the first of three, outlines the factors that influenced and guided how we approached the migration. Our approach may not be the same as your approach, but many of the basic principles may be similar.

Overview of our SharePoint environment pre-migration

A key principle for our SharePoint environment since 2012 was to avoid customisation and dependencies, and use the product ‘out of the box’ (OOTB) as much as possible.

  • Customisation would almost always require some degree of development and ongoing maintenance. It also meant that upgrades could be more complex and expensive.
  • Dependencies of any sort – be they integration components or third-party add-ons – could also make upgrades more complex and expensive.

Governance model

We also implemented a ‘balanced’ controlled environment, following the technical design models for SharePoint 2010 described by Microsoft (extract in image above), which recommended that organisations strike balance across three key governance elements:

SharePoint2010GovernanceBalance

Source: https://docs.microsoft.com/en-us/previous-versions/office/sharepoint-server-2010/cc303422(v%3doffice.14)

  • IT Governance. Centrally managed or locally managed?
  • Information Management. Tightly managed or loosely managed?
  • Application Management. Strictly managed or loosely managed development?

In our environment, the ability to create new SharePoint sites and sub-sites required the completion of a (SharePoint) online form and was restricted to the SharePoint Administrators. This enabled us to prevent uncontrolled growth in the environment and to ensure that all new sites were created within a pre-defined – but not overly strict – architecture design model.

Upgrade to SharePoint 2013 in early 2015

Our SharePoint site collections were created across five web applications: team (approximately 120 sites), project (approx. 120 sites), publication, apps, and intranet. Most of the corporate records were stored in team or project sites, as well as a single ‘apps’ site. (Our apps sites (< 10) were set up to address small business problems that in the past might have been addressed by using Microsoft Access).

Thanks to our OOTB model, we were able to upgrade to SharePoint 2013 over a weekend, with almost no errors. The only site we could not upgrade was the intranet which remains (as at August 2018) in ‘compatibility mode’.

Note: It is not possible to migrate directly from SharePoint 2010 to SharePoint Online. It must be upgraded to SharePoint 2013 or SharePoint 2016 first.

The situation in 2016

In May 2016 we changed our Microsoft Enterprise Agreement to an Office 365 subscription model. Our reasons for going to Office 365 were driven by multiple factors, including the need for mobile access to information.

It is important to remember that SharePoint Online is only one element among many others in Office 365. That is, while it is technically possible to do it, SharePoint would not normally be migrated on its own to SharePoint Online. Any migration must take in account a range of considerations relating to the broader Office 365 environment, including (but not limited to):

  • Office 365 licences (and what this meant for our users with Office installed on existing computers which were being upgraded to new Windows 10-based devices as part of a separate project)
  • Active Directory syncing so users can access the environment.
  • Exchange mailbox migrations so SharePoint-based, email-linked Flow workflows can work.
  • OneDrive for Business, as a SharePoint service to replace ‘personal’ drives on network file shares.
  • Security controls and records retention policies, set from the Office 365 Security and Compliance admin portal, as well as audit logs in that same portal.
  • Office 365 Groups with associated SharePoint sites, Yammer groups (which can be linked with Office 365 Groups) and Microsoft Teams (which can also be linked with Office 365 Groups).
  • ‘Classic’ and modern team sites, Office 365 Group-based sites, and communication sites.
  • The SharePoint user portal.
  • The mobile app, and how sub-sites are accessed.
  • The ever-changing SharePoint Online environment in which anything described as ‘classic’ is likely to be deprecated at some point, and new features appear.

Migrating multiple web applications to one

We needed to plan our migration process, moving away from our five web applications to a new model. We new that, with the exception of our customised intranet, we would probably be able to migrate almost all of our sites relatively easily because we had always kept to the OOTB model.

Fortunately, Microsoft produced a very useful 12-page document which provided a good overview describing how it ran its own SharePoint migration, and good advice for how we might do our own migration.

SharePoint_to_the_cloud_MSpaper.JPG

Learn how Microsoft ran its own migration

We had a range of factors to take into account.

  • One of our initial decisions was not to migrate any active site until all Exchange mailboxes were migrated (and preferably, end-users had new Windows 10 devices). As it turned out, the decision to migrate mailboxes was delayed and as a result we would end up migrating most sites first.
  • We need to work out how to migrate our content as it was no longer possible to do a ‘lift and shift’. We investigated the market and made the decision to acquire a migration tool, ShareGate, to do the migrations ourselves. We would later find the same tool useful to migrate personal drives to OneDrive for Business.
  • We identified the likelihood that we would create new SharePoint Online sites in parallel with the migration of on-premise sites; this was partially because some existing on-premise sites with multiple sub-sites would be split into separate sites instead, but also because the new SharePoint was so much more versatile and would likely be popular.

The new architecture model

An important point to note is that the new SharePoint Online architecture model provided the opportunity to re-think our SharePoint model and, to some extent, clean up or leave unwanted SharePoint content behind. To quote the Microsoft site above, ‘the best migration is no migration’.

As noted above, we had five primary web applications in our SharePoint 2013 environment. These had to be migrated (or re-created, in the case of publication sites) under one of two paths (only – /teams or /sites) to one of three site option:

  • ‘Classic’ sites (the default for all team and project sites)
  • Office 365 Group-based team sites
  • Communication sites (re-created page-based content)

That is:

  • Migrated team and project sites would become classic team sites under either (a) /teams/sitename path or (b) /teams/prj_sitename path, respectively. There were some exceptions:
    • Some sites with multiple sub-sites would be split up into multiple independent sites (including using the new ‘hub’ sites).
    • A couple of team sites would become communication sites.
    • Team sites that crossed multiple organisational business areas would be created as classic team sites under the /sites/sitename path.
  • Most publication sites that used the publishing features would need to be re-created as communication sites under the /sites/sitename path. There were some exceptions:
    • Some publication sites would become team sites instead.
    • The intranet would be managed separately as, at the very least, it would need to be re-created in SharePoint Online. It could not be migrated ‘as is’.
  • Application sites would become team sites.
  • Some existing sites or sub-sites might be migrated to SharePoint sites linked to Office 365 Groups, with the naming prefix of either GRP_ or PRJ_.

The above ‘mapping’ model was an early decision that did not change.

Preparatory work

We also commenced work on the following elements of work:

  • Reviewing all existing sites to determine which sites would be migrated or discarded – see below.
  • Re-developing our SharePoint Architecture documentation for the Online version.
  • Investigating and documenting all Office 365 admin and Office 365 Security and Compliance admin configuration settings, and determining roles. This process, which required Global Admin access, included establishing records retention policies (from mid 2018) in the Security and Compliance admin portal.
  • Re-developing our existing SharePoint admin documentation for the Online version, including all the configuration settings. We included the OneDrive for Business config settings in this same document as it is a SharePoint service.
  • Understanding how the new environment worked, and would work.
  • Re-establishing our SharePoint Admin and SharePoint User Group sites in SharePoint Online.
  • We also created a range of ‘test’ sites to better understand the new environment.
  • Creating an initial schedule for the migration of sites, targeting inactive sites first.
  • Assigning the initial batches of Office 365 licences.
  • Developing a repeatable process to migrate sites using ShareGate. In our environment steps involved:
    • Identify need to migrate site
    • Register a new site request in our SharePoint Admin portal.
    • Register the task in our Jira task management system.
    • Create the SharePoint Online site (via a script linked to the request).
    • Migrate the on-premise site, make it read only with a re-direct notice on the front page (and a three month deletion notice*).
    • Prepare the migrated site, including swapping the classic default home page to a modern home page.
    • Hand over the site to the business owners and close the task

* In practice many of these sites still remained after 6 months.

As part of our review process, we identified around a dozen sites that had one or all of the following elements, that would mean we had to devote more time to their migration (‘custom workload’ in the Microsoft document above):

  • Complex workflows which would need to be re-created.
  • Integration with other systems (mostly via BizTalk).
  • Links with ETL processes.

We also identified around 50 sites that would not be migrated:

  • Sites that were unused or had no content of value (often because the original was still on a drive).
  • Sites that did not need to be migrated, for example if their content had been migrated to a different business system.
  • Test sites.

Sites that were no longer used but contained records that needed to be kept were to be migrated with the word ‘Archive’ to the end of the site URL name, assigned a site retention policy, and then made read only.

By August 2017, we had identified that 250 site collections would be migrated to SharePoint Online. We acquired ShareGate in September 2017 and were ready to start migrating.

In Part 2 of this series of posts I will describe the migration process and the lessons we learned along the way.

Posted in Classification, Compliance, Digital preservation, Electronic records, Governance, Information Management, Information Security, Office 365, Records management, Retention and disposal, SharePoint Online

Office 365 – Applying retention periods to SharePoint document libraries and disposal/disposition actions

Records retention policies are created in the Security and Compliance Admin portal, Classifications section of Office 365, as noted in my previous post of 9 March 2018 on the subject.

This post describes how these are applied to document libraries and what happens when the records reach their disposal/disposition period.

Note: In Australia we refer to the disposal of records. In the US this is called disposition.

Setting up retention policies

Organisations may have complex or quite simple records retention policies. An important point to keep in mind in Office 365 is how many policies should be displayed to the end user to choose from.

Ideally, there should be fewer than a dozen classes so they are easy to choose from (see below). There is nothing stopping you creating 100 or 500 policies, but all of them will appear in the drop down list to choose from. Microsoft say they are working on ‘grouping’ policies, so this may help to fix the issue.

For some organisations, it may be useful to distill or group retention policies down to a smaller number.

  • For example, specific retention policies for certain types of records, and one (or two) for ‘all other’ records. The key, as we will see below, is naming them so they are obvious and easy to apply.

Viewing available retention policies

Retention policies that have been created appear in the Security and Compliance Admin portal, under Classifications > Labels.

O365_Classifications_Labels

Note: Labels must be published before they become visible to end users.

When you click on Labels, you can then see all the retention policies that have been created (but not necessarily published).

The screenshot below shows just the very top policy (a test/demonstration policy with a 7 day retention period) in a list of policies.

O365_Classifications_Labels_List.png

Note: Policies can be auto-applied, provided the policy has sufficient ability to identify what records they should be applied to.

Published policies appear in the Data Governance, Dispositions section:

O365_DataGovernance_Dispositions.png

The Dispositions section displays policies that have been published and are visible to end users in the Office 365 areas selected when the policy was created (e.g., Exchange, SharePoint, OneDrive etc).

O365_DataGovernance_Dispositions_List.png

Applying the policy in a SharePoint document library

To apply the policy to a SharePoint document library, go to the document library, library settings, and you will see the option to add the retention policy: ‘Apply label to items in this list or library’.

O365_RetentionPolicy_LibrarySet1.PNG

The ‘Apply Label’ dialogue shows the option to apply the label to existing items (recommended) and a drop down which shows all the published retention policies.

O365_RetentionPolicy_LibrarySet2.PNG

In this example below, there are four policies including the test policy.

O365_RetentionPolicy_LibrarySet3

The policy now applies to all records stored in that document library.

Managing disposal/disposition

When the records reach the end of the retention period configured in the policy, the person designated to be informed about the retention will receive an email notifying them of the need to review the dispositions.

O365_Dispositions_EmailNotification.pngNote, the person (or mailbox) receiving this email MUST be assigned to the Records Management role in the Security and Compliance Admin portal, Permissions section. No-one else will see the records due for disposal otherwise (not even the Global Admins, unless they have also been delegated to that role).

The records person clicks on the link ‘Go there now’ and it opens the following section in the Office 365, Security and Compliance Admin portal, showing the documents that are pending disposition. A number of options are available to sort by Type, to search, and to filter by several options.

 

O365_Dispositions_DocListing

The following options appear if a single document is selected. Note the option to extend the retention period or apply a different label, as well as the ability to delete the item permanently.

O365_Dispositions_Doc_OneDocument

Filtering options are displayed below.

O365_DataGovernance_Dispositions_Filters

Finally, the records manager can choose all the documents in the list and complete three bulk actions as shown.

O365_DataGovernance_Dispositions_BulkActions.png

Positives and negatives

The positives of this method of disposing of documents are that all records from any location will appear in a single view that can be filtered and actions taken as required.

The negatives are that potentially thousands of documents might appear in this listing every single day making it difficult to decide what can deleted or not.

However, as it’s possible to filter by the retention policy, that at least should make it relatively easy to identify what can be destroyed. The more fine-grained the policies, the fewer records should appear.

Organisations that have function-based disposal classes should find that all records relating to the same function appear for disposal under that function.

Another potential negative is that records may not always appear in the same context, whether it be subject- or function-based. For example, a collection of documents (often known as a ‘file’) may not appear in the disposition listing as a collection but as a set of records that are only connected by the disposal policy name. Does this matter?

Recording disposal actions

A key requirement for most organisations is keeping a record of what was destroyed.

At the moment the only apparent option to do this is to apply filters and export the list, using the handy ‘Export’ option to keep a record of what was destroyed. That csv file can then be stored in a control library to ensure a record is kept. This type of action requires a degree of control to ensure it happens every time.

It may also be possible to identify what was destroyed – and by whom – in the audit logs. This is being investigated.

 

Posted in Classification, Data Loss Prevention - DLP, Information Classification, Information Management, Information Security, Office 365, Products and applications, Records management, Retention and disposal

Changes to security classification and records retention in Office 365

In May 2016, I wrote about the creation of security classification labels in the Azure Information Protection (AIP) portal (old post here). Quite a bit has changed since that post, in particular the naming of policies, away from ‘High’ to ‘Low’ Business Impact (e.g., HBI – LBI) to real-world words such as ‘General’ and ‘Highly Confidential’.

In October 2017, I wrote about the new retention policies that could be applied to all Exchange, SharePoint and OneDrive content in Office 365.

Changes to the Security and Compliance admin portal – Classifications section

On 23 February 2018, Microsoft’s Adam Jung posted a new article to the Microsoft Tech Community titled ‘Consistent labeling and protection policies coming to Office 365 and Azure Information Protection’.

The main outcome of this change is that information security protection and records retention policies, linked with Data Loss Prevention (DLP policies) are created from a single interface in the Security and Compliance admin centre > Classifications section (Labels). These policies are set in Office 365 are then synced to Azure (and vice versa).

To quote the Microsoft blog: ‘The upcoming experience means that the same default labels can be used in both Office 365 and Azure Information Protection, and the labels you create in either of these services will automatically be synchronized across the other service – no need to create labels in two different places!’

This post looks at the changes and some potential issues that may arise.

Security and Compliance Admin Portal – Classifications

Records retention policies for Office 365 content are set as labels in the Security & Compliance Admin portal of Office 365 under Classifications – Labels.

The Classifications area also includes a section for ‘Sensitive Information Types’, which simply lists a range of information types that are also used for DLP policies.

Note: Access to that Admin portal is restricted by default to Global Admins and anyone assigned to a specific security role. Records managers in organisations that have or are deploying Office 365 should have access to this feature.

Setting (Records Retention) Classification Labels

The options for setting a records retention label were described in detail in my post above, but for reference again, they are:

  • Name
  • Label settings
    • Disabled or enabled (off/on)
    • When enabled, the ability to set (a) a retention period, and (b) an action when the period expires.
    • Alternatively, it is possible to just delete content when it’s older than a given time.
    • An option also allows the content be to be classified as a ‘record’ when the label was applied, providing further protection against deletion, for example.
  • Review your settings

Merging of label options – Retention and Security together in a single label

The primary change to classifications is the inclusion of new options when you choose to ‘Create a Label’.

These options are now:

  • Label name
  • Protection settings (e.g., information security)
  • Retention settings
  • Advanced options settings
  • Review your settings

These options are described below.

O365ClassificationLabelsMar2018.JPG

The ‘Protection settings’ section includes the following options:

  • Enabled or disabled. (If disabled the next check box options do not appear)
  • Block users from sending email messages or sharing documents with this label
  • Show policy tip to users if they send or share labeled content (The text of the policy tip is editable)
  • Send incident reports in email
  • Advanced protection for content with this label (Customise settings option)

The ‘Retention settings’ are identical with the options already described above:

  • Disabled or enabled
  • Various settings when enabled.

The ‘Advanced options settings’ section includes the following options:

  • Enabled or disabled. (If disabled the next check box options do not appear)
  • Add a watermark (text can be customised)
  • Add a header (text can be customised)
  •  Add a footer (text can be customised)

The Microsoft article notes: ‘We are building labeling capabilities natively into the core Office apps – including Word, PowerPoint, Excel, and Outlook, and soon there will be no need to download or install any additional plug-ins.’ This comment references the problem of having to download a plug-in for the classification options to appear in installed versions of Office.

Does it make sense to merge security classifications and records retention?

In my opinion, putting information security and records retention policies in the same label doesn’t make sense.

Retention is almost never linked with the confidentiality (or otherwise) of the records but based on government or legislative requirements or business needs.

But that was probably not Microsoft’s intention; it was probably to make it as simple as possible to create and apply these policies.

It would have made more sense to have separate label options for ‘Retention policies’ and ‘Security policies’. This would potentially mean, however, having two labels (if a label is in fact required for retention purposes).

Organisations with complex retention policies might find that the mixing of both policies in the one view makes it harder to find the individual security related policies, and have the potential to cause some confusion.

For example, it is could be hard to spot the Highly Confidential label in this listing if there were more than (say) 50 retention classes:

  • Client records – 7 years
  • Confidential
  • Financial Records – 7 years
  • Highly Confidential
  • Internal Use Only
  • Meeting Records – 3 years
  • Working Paper – 1 year

It also raises the question (which I have asked and will update this post if I receive a response) as to whether two policies can (or should) be applied on a document.

If two labels cannot be applied, this could mean that organisations have to have even more labels to take account of the various combinations. For example:

  • General Financial Records – 7 years
  • Confidential Financial Records – 7 years
  • Highly Confidential Financial Records – 7 years

Not to mention the link to DLP policies, although that doesn’t appear as a label.

In my opinion, combining these two options, while perhaps making it easier at the ‘front end’, has the potential to create confusion for users, let alone complicate the administration of retention management.

Read the full Microsoft blog article in the link below

https://techcommunity.microsoft.com/t5/Security-Privacy-and-Compliance/Consistent-labeling-and-protection-policies-coming-to-Office-365/ba-p/161553

Posted in Compliance, Data Loss Prevention - DLP, Exchange Online, Governance, Information Classification, Information Management, Information Security, Legal, Office 365, OneDrive for Business, Products and applications, Security, SharePoint Online, Training and education

SharePoint Online and OneDrive for Business – Preventing external sharing of data

A recent (September 2017) article suggested that OneDrive for Business (ODfB) (and by extension SharePoint Online (SPO); ODfB is a SharePoint-based service), a key application in Office 365 was a potential source of data leaks and/or target for hacking attacks.

I don’t disagree that, if not configured correctly, any online document management system – not just ODfB/SPO – could be the source of leaks or the target of external attacks. Especially if these systems, and the security controls that can protect the data in them, are not properly configured, governed, administered, and monitored.

But, I would ask, what controls do most organisations have in place now for documents stored in file shares and personal file folders, not to mention USB sticks, and the ability to send document via Bluetooth to mobile devices or upload corporate data to third-party document storage systems? Probably not many, because users have no other way to access the data out of the office.

As we will see, the controls available in Office 365 are likely to be more than sufficient to allow users to access to their documents out of the office, while at the same time reducing (if not eliminating) the sharing of documents with unauthorised users.

How to stop or minimise sharing from OneDrive for Business and SharePoint Online

There is one simple way to prevent the sharing of data stored in SPO and ODfB with external people – don’t allow it.

There are several ways to control what can be shared, each allowing the user a bit more capability. All these options should be based on business requirements and information security risk assessments, and Office 365 configured accordingly.

In this article I will start with no sharing allowed, and then show how the controls can be reduced as necessary.

External sharing – on or off

This is the primary setting, found in the main Office 365 Admin centre under Settings > Services & add-ins > Sites. If you turn this off, no-one can share anything stored in SPO or ODfB.

The option is shown below:

O365_SC_Sites_SharingOnOff

If you do allow sharing, you need to decide (as shown above) if sharing will be with:

  • Only existing external users
  • New and existing external users [Recommended]
  • Anyone, including anonymous users

The second option is recommended because it doesn’t restrict the ability to share with new users. The last option is unlikely to be used in most organisations and comes with some risks.

The next place to set these options are in the SPO and ODfB Admin centres.

OneDrive admin center

If the previous option is enabled, the following options are available for ODfB. Note that BOTH SharePoint and OneDrive are included here because the latter is a part of the SharePoint environment.

  • Let users share SharePoint content with external users: ON or OFF.
    • NOTE: If this option is turned OFF, all the following options disappear.
  • If sharing with external users is enabled, the following three options are offered:
    • Only existing external users
    • New and existing external users [Recommended]
    • Anyone, including anonymous users
  • Let users share OneDrive content with external users: ON or OFF
    • This setting must be at least as restrictive as the SharePoint setting.
  • If sharing with external users is enabled, the following three options are offered
    • Only existing external users
    • New and existing external users [Recommended]
    • Anyone, including anonymous users

If sharing is allowed, there are three sharing link options:

  • Direct – only people who already have permission [Recommended]
  • Internal – only people in the organisation
  • Anonymous access – anyone with the link

You can limit external sharing by domain, by allowing or blocking sharing with people on selected domains.

External users have two options:

  • External users must accept sharing invitations using the same account that the invitations were sent to [Recommended]
  • Let external users share items they don’t own. [This should normally be disabled]

A final ‘Share recipients’ checkbox allow the owners to see who viewed their files.

SharePoint admin center

The SPO admin center (to be upgraded in late 2017) has two options for sharing.

The first option is under the ‘sharing’ section which currently has the following options:

Sharing outside your organization

Control how users share content with people outside your organization.

  • Don’t allow sharing outside your organization
  • Allow sharing only with the external users that already exist in your organization’s directory
  • Allow users to invite and share with authenticated external users [Recommended]
  • Allow sharing to authenticated external users and using anonymous access links

Who can share outside your organization

  • [Checkbox] Let only users in selected security groups share with authenticated external users

Default link type

Choose the type of link that is created by default when users get links.

  • Direct – only people who have permission [Recommended, same as above]
  • Internal – people in the organization only
  • Anonymous Access – anyone with the link

Default link permission

Choose the default permission that is selected when users share. This applies to anonymous access, internal and direct links.

  • View [Recommended]
  • Edit

Additional settings (Checkboxes)

  • Limit external sharing using domains (applies to all future sharing invitations). Separate multiple domains with spaces.
  • Prevent external users from sharing files, folders, and sites that they don’t own [Recommended]
  • External users must accept sharing invitations using the same account that the invitations were sent to [Recommended]

Notifications (Checkboxes)

E-mail OneDrive for Business owners when

  • Other users invite additional external users to shared files [Recommended]
  • External users accept invitations to access files [Recommended]
  • An anonymous access link is created or changed [Recommended]

Sharing via the Site Collections option

In addition to the options above, sharing options for each SharePoint site are set in the ‘site collections’ section as follows. Note that the default is ‘no sharing allowed’. A conscious decision must be taken to allow sharing, and what type of sharing.

O365_SPO_Sharing1

When a site collection name is checked, the following options are displayed.

Sharing outside your company

Control how users invite people outside your organisation to access content

  • Don’t allowing sharing outside your organisation (default)
  • Allow sharing only with the external users that already exist in your organization’s directory
  • Allow external users who accept sharing invitations and sign in as authenticated users
  • Allow sharing with all external users, and by using anonymous access links

If anonymous access is not permitted (setting above), a message in red is displayed:

Anonymous access links aren’t allowed in your organization

SharePoint Sharing option

The SharePoint Admin Centre has an additional ‘Sharing’ section with the same settings as shown above for ODfB. It is expected that these multiple options will be merged in the new SharePoint Admin Centre due for release in late 2017.

Additional security controls

In addition to all the above settings, there are a range of additional controls available:

  • All user activities related to SPO and ODfB, including who accessed, viewed, edited, deleted, or shared files is accessible in the audit logs.
  • SPO and ODfB content may be picked up by Data Loss Prevention (DLP) policies and users prevented from sending them externally. This is of course subject to the DLP policies being able to identify the content correctly.
  • SPO and ODfB content may be subject to records retention policies set by preservation policies. These may impact on the ability to send documents externally.
  • SPO and ODfB content may be subject to an eDiscovery case.
  • Administrators can be notified when users perform specific activities in both SPO and ODfB.
  • Sharing (and access to the documents once shared) may be subject to security controls enforced through Microsoft Information Protection.

Conclusion

In summary, the settings above allow an organisation to strongly control what can be shared. If sharing is allowed, certain additional controls determine whether the sharing is for internal users or for users external to the organisation. If the latter is chosen, there are further controls on what external users can do. Audit controls and policies may also control how users can share information externally.

The key takeaway is that organisations should ensure that the sharing options available in Office 365 are based on the organisation’s business requirements and security risk framework.