Archive for the ‘Digital preservation’ Category

SharePoint 2013 Site Disposal Policies

May 18, 2013

SharePoint 2013 includes the option to set a disposal date on site collections. This article describes how to configure a SharePoint 2013 site collection to include a site disposal policy.

Default settings

A site cannot be deleted (either manually or automatically) unless a Site Policy has been set up (exception – the SharePoint Administrator has permissions to do this).

Without a Site Policy, the default settings under the Site Closure and Deletion option (see below) are as follows:

  • Site Closure – ‘Close this site now’ click box default: greyed out.
  • Site Deletion – ‘This site will be deleted on:’ Default: ‘Never’.
  • Site Policy – Default:  ‘No Site Policy’.

Setting up a Site Policy

New site policies are created under Site SettingsSite Collection AdministrationSite Policies. Once created, the policy is applied under Site SettingsSite AdministrationSite Closure and Deletion. While you can create multiple policies, only one policy can be selected at a time under the Site Closure and Deletion option.

There are no default policies; the first time Site Policies is opened, the Site Policies section provides only one option – ‘Create’. Each policy must have a Name and may have a Description. The name and description can be the class description from a records retention schedule, using ‘after date created’ or ‘after date closed’ as the triggers (see below).

Site Closure and Deletion options

There are three options under Site Closure and Deletion:

  • Do not close or delete site automatically. The default option.
  • Delete sites automatically. This option deletes a site on a pre-defined date after it was created or closed.
  • Close and delete sites automatically. This option first closes the site and then deletes it on pre-defined dates.

In addition there is a check box ‘Site Collection Closure’ that allows the site collection to be made read only when it is closed.

Delete sites automatically

When this option is selected the following appears:

  • Set Deletion Event. The two options provided are ‘Site closed date’ and ‘Site created date’, plus n days, months, or years.
  • (Check box) ‘Send an email notification to site owners this far in advance of deletion:’ (i.e., to warn them of the pending deletion) – n days, months or years. Default setting is 3 months.
  • (Check box) ‘Send follow-up notifications every:’ (i.e., to remind site owners of the pending deletion) – n days, months, or years. Default setting is 14 days.
  • (Check box) ‘Owners can postpone imminent deletion for:’ (i.e., to postpone the proposed deletion) – n days, months or years. Default setting is 1 month.

Close and delete sites automatically

This option is identical to Delete Sites Automatically except that it also includes a date when the site can be closed – after which a deletion event date is set followed by the same three options above.

Site Closure and Deletion

As noted above, a Site Policy must exist before a site can be closed and deleted using these options. The Site Policy must be selected otherwise the default options (see above) apply.

  • If the Site Policy is based on the Delete Sites Automatically option, the option to ‘Close this site now’ becomes available. If the option ‘Site Closed Date’ was selected, the site will not be deleted (at the pre-defined time) until this option is selected. If the option ‘Site Created Date’ was selected there is no requirement to ‘manually’ close the site.
  • If the policy is based on the Close and Delete Sites Automatically option, the option to ‘Close this site now’ becomes available. This allows the site to be closed earlier, otherwise the deletion date will be automatically calculated from the site policy setting and displayed next to the Site Closure and the Site Deletion options.
  • If no policy is selected, the default settings will apply; this means that the site cannot be closed.

Further reading

Overview of site policies in SharePoint 2013 (Microsoft).

Advertisements

Applying recordkeeping policies to email – Microsoft Messaging Records Management (MRM)

June 1, 2012

The problem

The problem of managing emails as records is summed up in the following statements:

“Many organizations have yet to define an email retention policy. More than one‐quarter of organizations have not yet established any sort of email retention policy despite the fact that there are a growing body of statutory requirements and legal obligations to preserve business records, including those stored in email. Among the nearly three‐quarters of organizations that have established an email retention policy, only two‐thirds of these organizations indicate that their users are fully aware of the policy.” Michael Osterman, “Messaging Archiving and Document Management Markets Trends, 2009-20112”, dated May 2009.

‘Over 40 years after the invention of email, relatively few institutions have developed policies, implementation strategies, procedures, tools and services that support the longterm preservation of records generated via this transformative communication mechanism.’  Christopher J Prom, ‘Preserving Email’, DPC Technology Watch Report 11-01 Decemer 2011. www.dpconline.org/component/docman/doc_download/739-dpctw11-01.pdf

Storing business records in context

Traditional records management theory recommends that there should be a clear relationship between records about a particular subject or issue, regardless of format, and the business context that originated it. (AS ISO 15489-2002: 9.3 Records Capture)

In the paper world, this was achieved by the co-location of related records in a physical file.

In the electronic world, this is usually achieved through the application of metadata. Business classification and naming systems applied to electronic folders generally achieve this; as well, electronic systems also allow for a range of cross-subject metadata that allows records to be organised in different contexts.

Additional, business context-specific metadata can be applied to emails (including from integrated business applications – for example, an email saved to TRIM will show the TRIM record number in its email metadata properties).  However, this ability (as with Properties in Office documents) is rarely enabled or used.

Instead, and as with Office documents, we tend to let users ‘categorise’ their emails (and documents on network shares) through folders – although not all users do this.  (Interestingly, online email systems like Google’s gmail use tags instead of folders).

Are emails documents?

The short answer is yes (in the Australian legal evidence context), but they are documents that, in a way like xml-based Office documents like docx, are made up of structured data that displays as a single ‘document’.

Part of the problem with emails as records is the perception (on the part of users who have never had to face court) that they are not documents, but messages.  The ability to use the system to send or receive ‘private’ messages exacerbates this perception.

The problem of storing emails as records

Emails have been a constant problem and challenge for records managers and recordkeeping since they first appeared in the early 1990s.

The three main approaches to keeping emails have been to (a) print to paper, (b) save to a recordkeeping system, and (c) save to a drive.

Print to paper, while relatively common in many organisations even now, is probably the poorest (and some might say ‘silliest’) option in the digital world as (a) it is dependent on users, (b) emails usually lose their message headers, (c) emails are unsearchable in their electronic form, (d) emails remain on the Exchange system and are discoverable.

Saving emails to a recordkeeping system, while better than printing, is an inadequate option because (a) it is usually dependent on users to do it, (b) the email still remains in the Exchange system, and (c) it can sometimes result in the email being saved in a different format that is not necessarily suitable for long-term preservation (e.g., TRIM’s .vmbx).  There is also the problem of users saving ‘dumb’ emails with (valuable) attachments, which can make the attachment harder to find, identify or access.  Some systems (such as SharePoint 2010) include email-enabled storage locations.

Chris Prom, in a blog posting titled ‘Practical E-Records ‘Facilitating the Generation of Archives in the Facebook Age’, notes that:

‘…the formal recordkeeping systems previously used by many organizations for electronic records have died or have one foot firmly in the grave.  At the same time, the habits that individuals use in producing, consuming, storing, filing, searching, and interpreting records are themselves undergoing constant change.  People adopt new communication technologies at an ever-quickening pace.   Divergent personal practices, rather than the centralized electronic systems, are the harsh reality that confronts our profession’.

Saving to a drive is also a poor option, and is usually based on user preferences to want to ‘keep’ emails.  Emails saved to drives (a) will still remain in the Exchange system, (b) may lose their header information, and (c) are not necessarily saved in appropriate or accessible formats.

In relation to the last point, Outlook does not make it easy for an end user to decide, with usually five options to choose from – which is the right one?  Users will usually choose whatever is the default (.msg), but this isn’t necessarily the best long term option (which is MIME or EML – the latter described by the National Archives of Australia (NAA) as ‘an acceptable open file format for long term storage).

In all cases, keeping these emails in the business context to which they relate has been a constant problem for records managers.  As a consequence, there is a tendency on the part of almost all businesses to leave and manage emails where they are (i.e., in Exchange).

Microsoft Exchange 2010 – Messaging Records Management

To try to address this problem, Microsoft introduced ‘Mailbox Manager Policies’ in Exchange Server 2003.

This was followed by ‘Message Records Management’ with Managed Folders in Exchange Server 2007 (a feature that remains in Exchange 2010).

Exchange Server 2010 includes a new model of managing emails as records, called ‘Messaging Records Management’.  Microsoft describe it as follows:

‘Messaging records management (MRM) is the records management technology in Microsoft Exchange Server 2010 that helps organizations reduce the legal risks associated with e-mail. MRM makes it easier to keep the messages needed to comply with company policy, government regulations, or legal needs, and to remove content that has no legal or business value. This is accomplished through the use of retention policies or managed folders’. (Source: http://technet.microsoft.com/en-us/library/dd335093)

As Microsoft notes, however (on the same page), MRM does not prevent users from deleting messaging; it is really only designed to remove them at the end of a given period.  Microsoft recommend ‘journaling’ emails where there are specific business reasons to keep them for longer (such as legal proceedings or the need to ensure specific email is kept), or applying the Legal Holds functionality.

The key elements of MRM are Retention Policy Tags (RPTs) and Retention Policies.

There are three types of Retention Tags: (1) Default Policy Tags (DPT), (2) Retention Policy Tags, and (3) Personal  Tags (which are an ‘opt-in’ on the email client).

  • Retention Policy Tags (RPTs) are used on default folders (e.g., inbox, junk mail, sent, deleted). Users cannot change the RPT but can override it with a Personal Tag.
  • Default Policy Tags can be applied by users to untagged items.  A Retention Policy can contain only one default policy tag.
  • Personal Tags can be applied by users to their own custom folders or individual emails.

In most cases, users make the decision, and the retention applies on where the email is located.  If there is actual or anticipated litigation, a Retention Hold can be applied to the user’s mailbox; however, this does not prevent users deleting emails, it only overrides any retention policies.  The Legal Hold option should be applied to prevent deletion.  Once this option is applied, Legal Hold ‘captures any deleted or edited items into a special folder that’s neither accessible nor changeable by the user’.

All retention tags include: a Tag Name, a Tag Type, an age limit (in days) with an action to take, and comments.

The actions available are:

  • Delete And Allow Recovery – This action will perform a hard delete, sending the message to the dumpster. The user will be able to recover the item using the Recover Deleted Items dialog box in Outlook 2010 or Outlook Web App.
  • Mark As Past Retention Limit – This action will mark an item as past the retention limit, displaying the message using strikethrough text in Outlook 2007, 2010 or Outlook Web App.
  • Move To Archive – This action moves the message to the users archive mailbox.(see below)
  • Move To Deleted Items – This action will move the message to the Deleted Items folder.
  • Permanently Delete – This action will permanently delete the message and cannot be restored using the Recover Deleted Items dialog box.

Once the tags are created, they can be added to a Retention Policy and this policy, in turn, is then applied to specific mailboxes – one policy per mailbox.

The ‘auto-tagging’ feature, once 500 items have been tagged, will automatically tag items in a user’s mailox based on their past tagging activities.

So, is MRM the answer to managing emails as records? 

Yes and no.  From a recordkeeping perspective, MRM:

  • Does nothing to ensure that records are kept in the business activity or functional context to which they relate, unless (of course) the emails are the only form of record that exists for the business activity.
  • Does not stop users from deleting emails.

On a positive note, MRM:

  • Attempts to address the problem of email retention.
  • Allows the application of a retention policy to emails that might be stored in a business context Outlook mailbox or fold As well, Exchange features like Legal Hold and Journaling allow further controls to be implemented.

Archiving

Exchange 2010 now includes a ‘personal archives feature’, which allows users to save emails to their own archive instead of saving emails to drives or using Personal Storage (.pst). A good article on this subject can be found at this location: http://mohamedridha.com/2011/11/07/exchange-2010-online-archiving-and-retention-tagspolicies-a-practical-example/

Sources (all retrieved 1 June 2012)

Ensuring long term access to digital information

May 10, 2010
(This is a version of an article written for the RMAA magazine Informaa Quarterly, due to be published in May 2010).
In February 2010, the Blue Ribbon Task Force on Sustainable Digital Preservation and Access (BRTF-SDPA, brtf.sdsc.edu), a US-based group established in 2007 and funded by several private and public organisations, published a report titled ‘Sustainable Economics for a Digital Planet: Ensuring Long-Term Access to Digital Information’.
The BRTF-SDPA report examined the long-term preservation of digital information from an economic perspective, noting that ‘… economically sustainable preservation (of digital information) is … an urgent societal problem’.  The report quotes a 2008 IDC report stating that the volume of information now created exceeds all available storage.
The BRTF focussed its attention on digital information created within four key areas: scholarly discourse; research data; commercially owned cultural content; and collectively produced web content.  The report did not examine digital information produced by public sector agencies because there are already ‘… well articulated mandates for preservation and well defined organisations with clear roles and responsibilities’ to preserve the digital information produced by those agencies.
The report confirms the frequently cited preservation and conservation mantra that the main business case for preservation is use.  The dilemma for those making decisions about preservation is that access – and therefore use – is impossible without preservation; however, if there is no demand for access, there will be no preservation. What to preserve is the problem.
Identifying what should be preserved for later use requires significant effort that requires the agreement of a range of stakeholders – those who own, will select, preserve, pay for preservation to take place and who will eventually benefit.  The interests of these stakeholders need to be aligned as much as possible; and yet those who make preservation decisions now must attempt to do so without any real idea of what future stakeholders may want to access.
The report makes the point that a key threat to ‘persistent access’ is the costs involved, particularly where the costs outweigh the perceived benefits.
The report presents digital information as economic goods that have four essential attributes: the derived demand for access rather than preservation; their nature as depreciable durable assets that can suffer from physical degradation and loss of functionality; the ubiquity of access (known as ‘non rivalrous consumption’) which can lead to ‘free riding’; and the temporarily dynamic and path dependent nature of the digital preservation process throughout the lifecycle of the information.
These attributes, according to the report, mean that problems may be encountered aligning incentives to preserve among beneficiaries, owners and preservers.  The closer the alignment, the more likely that appropriate preservation actions will be taken.  Weak or misguided incentives to preserve are the greatest risk to preservation.
According to the report the six key conditions necessary to ensure the economic sustainability for digital information are: recognition of the benefits; selecting materials with long-term value; providing incentives for preservation; establishing effective governance arrangements and allocating resources; and ensuring that timely actions are taken before digital information is lost.
As the report notes, solving the economic challenges of digital preservation is neither easy nor insuperable.  A careful balance needs to be established between the perceived future  value of digital information, incentives for its preservation, and the roles and responsibilities of key stakeholders.