Posted in Electronic records, Products and applications, Records management, Retention and disposal, Sharepoint 2010, SharePoint 2013

Managing Physical Records in SharePoint 2010

It is sometimes said by records managers that one of the drawbacks of SharePoint 2010 (and SharePoint 2013) is that it doesn’t manage physical records, or at least not very well.

This article describes how to manage physical records in SharePoint 2010/2013 using a list that includes barcodes.

Up front and out of the box, the main limitations are the ability to: (a) print labels for paper files and boxes (including labels with barcodes) (b) attach a portable scanner to scan barcodes and (c) maintain a chronological history of file movements. These limitations may not be a problem for many organisations seeking a low-cost way to manage mostly inactive files and boxes, especially if they currently use an Access database or Excel spreadsheet to do this now.

Options to address each of these limitations are discussed in this article.

Metadata options

There are almost no limitations on the type of metadata you can use to describe physical records. I would suggest having the following metadata:

  • Title (e.g., name on the file or box)
  • Record Type (drop down list – file, box, etc)
  • Reference ID (manually entered unless you use the default ‘ID’ option, possibly with a separate ‘year’ column)
  • Date created (date first registered, default system date)
  • Start date (start of date range)
  • End date (end of date range)
  • Business Owner
  • Current location (can be a custom list of pre-defined locations)
  • Date at current location
  • Record status (drop down list: active, inactive, missing, destroyed etc)
  • Internal barcode (if one exists)
  • External barcode (usually the one provided by a storage provider)
  • Records description (description of contents, usually of a box)
  • Box number
  • Box barcode (may be the same as the Internal or External Barcode)
  • Retention schedule (can be from a pre-defined list)
  • Disposal action (can be from a pre-defined list)
  • Date to be destroyed (manually entered based on retention schedule and disposal action)
  • Destroyed? (checkbox)
  • Date destroyed (manually entered date)
  • Legal hold? (checkbox)

If physical files are registered in the system, then only details of the box and disposal action would need to be entered. When the box is ready to be sent offsite, details from the offsite storage provider’s label (often provided in advance) can be entered in the external barcode and other metadata (including current location) updated.

An optional barcode can be obtained by checking ‘Barcode’ in the Information Management Policy settings for the list item. Once checked, the barcode appears when the item is displayed in view but not when it is edited or exported.

Printing labels

All list metadata (except the barcode) can be exported to an Excel spreadsheet, which can then be used to print labels. Microsoft provide details how to create and print labels for a list in Excel here:


Using a portable (or attached) scanner to scan barcodes and update your records register, especially with a new location (or during a physical records audit) saves a great deal of time instead of entering details manually. It is possible to scan directly to an Excel spreadsheet however several steps may be required to match or validate the barcode with the details in the SharePoint list.

For example, if series of files are being scanned to a box (or location), the usual practice is to scan the box (or location) barcode first, then scan each individual file. If scanning directly to an Excel spreadsheet, you would do it the other way around – have the list of existing files in the spreadsheet, then scan the box or location barcode to the cell next to the file number. This detail can then be copied and pasted back to the SharePoint list.

Chronological movement history

The only option to maintain a chronological movement history, out of the box, would be to add versioning to the list. This means that each time the item is edited, a new version is created, keeping the previous details. The movement history will not display in a single view in the list but clicking on ‘Version History’ will show any changes made in single view. Only ‘major’ version numbers are supported in lists (i.e., 1, 2, 3, 4).

Posted in Digital preservation, Electronic records, Exchange 2010, Governance, Information Management, Legal, Products and applications, Records management, Retention and disposal

Applying recordkeeping policies to email – Microsoft Messaging Records Management (MRM)

The problem

The problem of managing emails as records is summed up in the following statements:

“Many organizations have yet to define an email retention policy. More than one‐quarter of organizations have not yet established any sort of email retention policy despite the fact that there are a growing body of statutory requirements and legal obligations to preserve business records, including those stored in email. Among the nearly three‐quarters of organizations that have established an email retention policy, only two‐thirds of these organizations indicate that their users are fully aware of the policy.” Michael Osterman, “Messaging Archiving and Document Management Markets Trends, 2009-20112”, dated May 2009.

‘Over 40 years after the invention of email, relatively few institutions have developed policies, implementation strategies, procedures, tools and services that support the longterm preservation of records generated via this transformative communication mechanism.’  Christopher J Prom, ‘Preserving Email’, DPC Technology Watch Report 11-01 Decemer 2011.

Storing business records in context

Traditional records management theory recommends that there should be a clear relationship between records about a particular subject or issue, regardless of format, and the business context that originated it. (AS ISO 15489-2002: 9.3 Records Capture)

In the paper world, this was achieved by the co-location of related records in a physical file.

In the electronic world, this is usually achieved through the application of metadata. Business classification and naming systems applied to electronic folders generally achieve this; as well, electronic systems also allow for a range of cross-subject metadata that allows records to be organised in different contexts.

Additional, business context-specific metadata can be applied to emails (including from integrated business applications – for example, an email saved to TRIM will show the TRIM record number in its email metadata properties).  However, this ability (as with Properties in Office documents) is rarely enabled or used.

Instead, and as with Office documents, we tend to let users ‘categorise’ their emails (and documents on network shares) through folders – although not all users do this.  (Interestingly, online email systems like Google’s gmail use tags instead of folders).

Are emails documents?

The short answer is yes (in the Australian legal evidence context), but they are documents that, in a way like xml-based Office documents like docx, are made up of structured data that displays as a single ‘document’.

Part of the problem with emails as records is the perception (on the part of users who have never had to face court) that they are not documents, but messages.  The ability to use the system to send or receive ‘private’ messages exacerbates this perception.

The problem of storing emails as records

Emails have been a constant problem and challenge for records managers and recordkeeping since they first appeared in the early 1990s.

The three main approaches to keeping emails have been to (a) print to paper, (b) save to a recordkeeping system, and (c) save to a drive.

Print to paper, while relatively common in many organisations even now, is probably the poorest (and some might say ‘silliest’) option in the digital world as (a) it is dependent on users, (b) emails usually lose their message headers, (c) emails are unsearchable in their electronic form, (d) emails remain on the Exchange system and are discoverable.

Saving emails to a recordkeeping system, while better than printing, is an inadequate option because (a) it is usually dependent on users to do it, (b) the email still remains in the Exchange system, and (c) it can sometimes result in the email being saved in a different format that is not necessarily suitable for long-term preservation (e.g., TRIM’s .vmbx).  There is also the problem of users saving ‘dumb’ emails with (valuable) attachments, which can make the attachment harder to find, identify or access.  Some systems (such as SharePoint 2010) include email-enabled storage locations.

Chris Prom, in a blog posting titled ‘Practical E-Records ‘Facilitating the Generation of Archives in the Facebook Age’, notes that:

‘…the formal recordkeeping systems previously used by many organizations for electronic records have died or have one foot firmly in the grave.  At the same time, the habits that individuals use in producing, consuming, storing, filing, searching, and interpreting records are themselves undergoing constant change.  People adopt new communication technologies at an ever-quickening pace.   Divergent personal practices, rather than the centralized electronic systems, are the harsh reality that confronts our profession’.

Saving to a drive is also a poor option, and is usually based on user preferences to want to ‘keep’ emails.  Emails saved to drives (a) will still remain in the Exchange system, (b) may lose their header information, and (c) are not necessarily saved in appropriate or accessible formats.

In relation to the last point, Outlook does not make it easy for an end user to decide, with usually five options to choose from – which is the right one?  Users will usually choose whatever is the default (.msg), but this isn’t necessarily the best long term option (which is MIME or EML – the latter described by the National Archives of Australia (NAA) as ‘an acceptable open file format for long term storage).

In all cases, keeping these emails in the business context to which they relate has been a constant problem for records managers.  As a consequence, there is a tendency on the part of almost all businesses to leave and manage emails where they are (i.e., in Exchange).

Microsoft Exchange 2010 – Messaging Records Management

To try to address this problem, Microsoft introduced ‘Mailbox Manager Policies’ in Exchange Server 2003.

This was followed by ‘Message Records Management’ with Managed Folders in Exchange Server 2007 (a feature that remains in Exchange 2010).

Exchange Server 2010 includes a new model of managing emails as records, called ‘Messaging Records Management’.  Microsoft describe it as follows:

‘Messaging records management (MRM) is the records management technology in Microsoft Exchange Server 2010 that helps organizations reduce the legal risks associated with e-mail. MRM makes it easier to keep the messages needed to comply with company policy, government regulations, or legal needs, and to remove content that has no legal or business value. This is accomplished through the use of retention policies or managed folders’. (Source:

As Microsoft notes, however (on the same page), MRM does not prevent users from deleting messaging; it is really only designed to remove them at the end of a given period.  Microsoft recommend ‘journaling’ emails where there are specific business reasons to keep them for longer (such as legal proceedings or the need to ensure specific email is kept), or applying the Legal Holds functionality.

The key elements of MRM are Retention Policy Tags (RPTs) and Retention Policies.

There are three types of Retention Tags: (1) Default Policy Tags (DPT), (2) Retention Policy Tags, and (3) Personal  Tags (which are an ‘opt-in’ on the email client).

  • Retention Policy Tags (RPTs) are used on default folders (e.g., inbox, junk mail, sent, deleted). Users cannot change the RPT but can override it with a Personal Tag.
  • Default Policy Tags can be applied by users to untagged items.  A Retention Policy can contain only one default policy tag.
  • Personal Tags can be applied by users to their own custom folders or individual emails.

In most cases, users make the decision, and the retention applies on where the email is located.  If there is actual or anticipated litigation, a Retention Hold can be applied to the user’s mailbox; however, this does not prevent users deleting emails, it only overrides any retention policies.  The Legal Hold option should be applied to prevent deletion.  Once this option is applied, Legal Hold ‘captures any deleted or edited items into a special folder that’s neither accessible nor changeable by the user’.

All retention tags include: a Tag Name, a Tag Type, an age limit (in days) with an action to take, and comments.

The actions available are:

  • Delete And Allow Recovery – This action will perform a hard delete, sending the message to the dumpster. The user will be able to recover the item using the Recover Deleted Items dialog box in Outlook 2010 or Outlook Web App.
  • Mark As Past Retention Limit – This action will mark an item as past the retention limit, displaying the message using strikethrough text in Outlook 2007, 2010 or Outlook Web App.
  • Move To Archive – This action moves the message to the users archive mailbox.(see below)
  • Move To Deleted Items – This action will move the message to the Deleted Items folder.
  • Permanently Delete – This action will permanently delete the message and cannot be restored using the Recover Deleted Items dialog box.

Once the tags are created, they can be added to a Retention Policy and this policy, in turn, is then applied to specific mailboxes – one policy per mailbox.

The ‘auto-tagging’ feature, once 500 items have been tagged, will automatically tag items in a user’s mailox based on their past tagging activities.

So, is MRM the answer to managing emails as records? 

Yes and no.  From a recordkeeping perspective, MRM:

  • Does nothing to ensure that records are kept in the business activity or functional context to which they relate, unless (of course) the emails are the only form of record that exists for the business activity.
  • Does not stop users from deleting emails.

On a positive note, MRM:

  • Attempts to address the problem of email retention.
  • Allows the application of a retention policy to emails that might be stored in a business context Outlook mailbox or fold As well, Exchange features like Legal Hold and Journaling allow further controls to be implemented.


Exchange 2010 now includes a ‘personal archives feature’, which allows users to save emails to their own archive instead of saving emails to drives or using Personal Storage (.pst). A good article on this subject can be found at this location:

Sources (all retrieved 1 June 2012)

Posted in Electronic records, Products and applications, Records management, Sharepoint 2010

What records managers need to know about Sharepoint 2010

The following is a revised version of an article that appeared in the July 2011 edition of Informaa Quarterly, IQ, the quarterly journal of the Records and Information Management Professionals Australasia (RIMPA).

Sharepoint 2010, is starting to be used by more and more organisations. And it is being used to create and manage records.

The following are the things that records managers need to know about this product.

Check the version

First, Sharepoint 2010 is not Sharepoint 2007. Sharepoint 2010 has recordkeeping functionality built in that its predecessor lacked.

If someone is talking about a Sharepoint implementation, records managers need to know which version – 2007 or 2010.

Know what it does – and doesn’t. 

Microsoft has spent a lot of effort building recordkeeping functionality into Sharepoint 2010.

Records managers need to understand this functionality and how it works, and why they need to be part of the Sharepoint 2010 implementation process at the beginning of the project. They need to know and understand what it can do, and what it can’t.

For example, it doesn’t manage physical records out of the box. It doesn’t integrate well with Outlook. And it doesn’t do security classifications.*

It is very easy overlook or ignore the recordkeeping functionality in Sharepoint 2010 when it is being implemented. The product looks and behaves very much like Sharepoint 2007. So much so, it would be (and probably has been) easy for IT to implement it without any reference at all to records managers.

The first time records managers might find out about Sharepoint is when they find the intranet has been built using it and they are given a team site used to manage records.

The bits that matter

The following is a quick summary of the key recordkeeping elements Sharepoint 2010.


Sites and team sites in Sharepoint are more or less the same concept as a web site, grouped within an overall Site Collection.

Just like a web site, Sharepoint 2010 sites and team sites are locations accessed via an URL. But, unless access to the site or team site is open to everyone (such as the organisation’s intranet), access will be controlled (often by a site administrator) through a series of permissions. Records managers need to know and understand what this means for the management of records.

Sites may be made up of a range of different content types (see below) used by end users for all sorts of records and ‘non-records’.

One of the main reasons given by IT not to involve records managers in a Sharepoint 2010 implementation is likely to be that ‘there are no records stored on the sites’. This is a bit like saying that there are no records stored in network shares or in Outlook folders.

Of course, it depends on what industry sector you work in and the litigation risks associated with that sector and/or the records created by the organisation.

Without good planning and the involvement of records managers, users may create records on sites without any recordkeeping functionality at all.

Records managers may not know anything about these sites until well after they have been created and content added. By this time, the cat is out of the bag and it could be quite difficult (but not impossible) to retrofit the recordkeeping functionality that could have been applied at the beginning.

Lesson for records managers: Learn about the recordkeeping functionality of Sharepoint 2010 and get involved early with any implementation project.

Managed Metadata

Good metadata should be second nature to records managers. We know that metadata is a good way to describe records that will allow the organisation to find and make better use of records, including disposition decisions.

Sharepoint 2010 is a dream product for metadata lovers. The opportunities to apply metadata to – and in – records are almost limitless.

The close integration between Sharepoint 2010 and Office 2010 allows recordkeeping metadata to be be embedded in documents and then re-used on sites in metadata columns.

A key feature in Sharepoint 2010 is the Managed Metadata Service (MMS). The MMS allows the creation of multiple hierarchical descriptions of information that can then be applied to records across sites.

End users can also tag records with their own metadata which is then surfaced in the MMS, allowing it to be added to the rest of the organisation’s metadata as required. This is particularly useful when a new subject appears in the organisation.

In some ways, the MMS resembles folder names in Outlook or network shares. It is the way end users see their world.

Lesson for records managers: Understand how end users describe their world and how Sharepoint 2010 can capture that information. Recordkeeping metadata sometimes doesn’t make sense to end users.

Content Types

Content types are the main recordkeeping ‘currency’ in Sharepoint 2010.

Sites consist almost entirely of content types, including when a user clicks on ‘New – (Content Type)’ in a Library or List, or creates content in the form of a wiki or event.

Records managers who are familiar with TRIM record types will see a certain similarity with Sharepoint 2010 content types. But the resemblance is superficial and it would be a mistake to try to make content types behave like TRIM record types.

TRIM record types are generally few in number and mostly consist of documents, files (containers) and boxes. Retention information is generally applied at the file (container) level, and documents inherit that information. Documents are generally single, digital objects of some sort.

In the Sharepoint 2010 model, content can be literally anything. One way of thinking about content types is that each one can ‘map’ to a disposal class in a retention schedule.

Some content types can (and should) be created by customising the top level content type. This is because each content type can then have its own specific metadata and retention information.

Other content types, such as wikis and calendars, can be left ‘as is’ as they are likely to be managed and retained the same way across all sites. For example, calendars could all have a retention period of 2 years after last entry.

Records managers need to understand that *every* content type can be assigned two key elements. These are: (a)a range of recordkeeping metadata that is mostly invisible to end users, but can include end-user defined metadata, and, (b) Information management (IM) policies.

IM policies, and in particular retention policies, are one of the *key* differences between Sharepoint 2010 and 2007. This functionality is not overly obvious if you don’t know where it is or what it does.  It should not be confused with ‘expiry dates’.

The importance of getting IM policies right becomes more obvious when they are not configured correctly. When this happens, (such as by configuring the policies on the parent, generic content type), it can become impossible to do anything with them, and all records using the same content type could have either no recordkeeping rules applied, or it will be same regardless of the type of record it is.

This may not be a bad thing for generic types of records, such as calendars, but could be disastrous if every single document saved into the site collection has the same retention policy.

A *key* information management policy element is Retention. This check-box functionality is easily overlooked but it is a critical piece of recordkeeping as it defines how long the record will be kept. As noted already, each content type can (and perhaps should be) mapped to a class in a retention schedule (or to Normal Administrative Practice (NAP), for Australian government agencies).

Retention policies allow for multi-stage disposition changes to records. They also allow records either to be disposed of on a site after a given period, or sent to the Records Centre for review and/or disposal or further retention.

Sharepoint 2010 includes the US-style concept of ‘declaring’ something as a record. While this functionality is alien to most Australian records managers, it has potential use for allowing changes to the status of a record.

Records stored on network shares or in email folders provide a useful analogy. If the organisation has an EDRMS, end users must transfer the record from the drive or folder to the EDRMS. In most cases this process will leave the document or email in its original location.

The records declaration function in Sharepoint 2010 would allow for a document that is created or saved to a site as a ‘non-record’ (eg, a ‘working document’) to be ‘declared’ as a record on the site, resulting in a change to retention requirements for the same record, rather than expecting an end user to save it to a different system.

Information management policies also include the ‘auditing’ function, allowing auditing on a range of activities.

Lesson for records managers: Learn about Content Types and understand how Information Management policies work and can be configured.

Where are the containers?

Potentially one of the most confusing things about Sharepoint 2010 for records managers seeing the product for the first time is the apparent absence of containers.

Collections of document-type records can be stored in libraries. Libraries may contain folders (another content type) defined by users, or document sets which resemble containers.

It is important to keep an open mind about the concept of a container when learning about Sharepoint 2010.

The product offers multiple ways to organise records in a given context, including through libraries, folders and document sets, but also (and perhaps most importantly) through metadata.

The way end users organise their records is likely to depend on their (or their business unit’s) context. Different users, and records managers, may see the world in different ways and organise information accordingly.

Sharepoint 2010 provides the flexibility to meet these sometimes competing interests, including by allowing end users to organise information that makes sense to them, and then send it to the Records Centre at or after a given time, to be organised in a way that makes sense to records managers.

Lesson for records managers: Look outside the square at how records might be organised. Pre-defined containers are not necessarily the only option.

The Content Organiser

As noted above, information management policies in content types in Sharepoint 2010 can be used to define retention rules for all types of content.

In the same way that rules that can be applied in email applications to route emails to specific folders, the content organiser in Sharepoint 2010 can be used to route any content type to a specific, pre-defined location in the records centre.

This is done by using the feature to send the content type to a different location such as the records centre after a given period of time.

Lesson for records managers: Learn how content types work. If you are involved at the beginning of a Sharepoint 2010 implementation you have a better chance of setting up and configuring content types correctly so they will eventually be managed in the records centre. This can include the ability for records managers to review new, undefined content that is sent to a ‘drop off’ area in the Records Centre.

The Records Centre

The Records Centre in Sharepoint 2010 is, in effect, a separate site in the site collection (or farm) to store corporate records. An organisation may have more than one records centre.

The centre should be owned and managed by records managers. There are various ways to configure a records centre.

One option would be to create a library (or libraries) for records, each of which contains a pre-defined folder structure mapped to the organisation’s Business Classification Scheme (BCS).

As these folders can have IM policies applied to them, they can be mapped to specific classes in the organisation’s retention schedule. In this way, they start to look very much like containers!

As Sharepoint 2010 allows for retention rules to be applied to either libraries and folder OR content types, each of these folders can have retention rules applied to them.

Lesson for records managers: Own the records centre!