Getting Started with Email Archiving in Exchange Server 2010

In my first job in IT I was given an email address and a mailbox with a 20mb quota. Thats 20 megabytes. Unthinkable in today’s business world, particularly when you consider Exchange Server 2010’s default storage quotas of +2Gb and the Microsoft “large mailbox vision”.

email-logo

The PST Nightmare

Naturally with such a small mailbox quota the entire organization made heavy use of PST files for storing email. When you had read an email and wanted to keep it, you moved it to the PST file immediately. Every week or so you would go to your Sent Items and move all your recent sent email into the PST as well. Deleted items were emptied automatically when Outlook closed.

Everyone had their own “system” for maintaining their email, but the results were basically the same – you stayed under your mailbox quota by creating a mess of PST files, and permanently deleting anything you didn’t want to keep.

Of course these days PST files are a royal pain for everyone involved. The end users become frustrated when PST files are corrupted seemingly at random, or lost when they switch computers.

The IT administrators become frustrated with assisting end users with those problems, and the escalating cost of storing PST files on file servers.

And the business itself is often frustrated by information that is leaked or lost in PST files, unable to be discovered, searched, or secured.

Email Archiving Solutions

And so an entire industry segment evolved to deliver email archiving solutions to businesses, such as Symantec Enterprise Vault and GFI MailArchiver.

The benefits of these types of email archiving systems are obvious; the valuable information stored in email can be safely and centrally archived in a format that is secure, searchable, and also carries other benefits such as single-instancing, compression, and the ability to locate archives on different tiers of storage and with different backup/retention timelines applied.

Exchange Native Archiving

Unsurprisingly, Microsoft then delivered native email archiving capabilities in Exchange Server 2010. At RTM the feature was somewhat disappointing with a few desirable options missing, such as the ability to locate the archive mailbox for a user on a separate database to their primary mailbox.

However in Exchange Server 2010 SP1 that issue, and many others, were resolved and native archiving became an attractive option for organizations that were beginning to evaluate email archiving solutions.

Personal Archive Mailboxes

Exchange 2010 native archiving involves the provisioning of a personal archive, or archive mailbox, for end users. This is in effect just another mailbox, separate to their primary mailbox, stored in an Exchange mailbox database.

Archiving is enabled on a per-mailbox basis, and each user that is enabled for archiving requires an Enterprise Client Access License (CAL). This isn’t enforced at the software level, it is your responsibility to ensure that you have the correct number of Enterprise CALs for the number of users you intend to enable for archiving.

Further reading:

Archive Databases

Thanks to the improvements in Exchange Server 2010 SP1 the archive mailbox for a user can be located on a different mailbox database to their primary mailbox. The mailbox database can be on the same mailbox server, or a different mailbox server, than the database that hosts the primary mailbox, as long as both servers are located within the same Active Directory site.

This allows you to locate mailbox databases that are dedicated to archive mailboxes on different servers, or on lower tiers of storage (although Exchange 2010 is already designed to run on low cost disk anyway), and apply different backup schedules and retention periods.

And because the archive mailbox database is otherwise the same as any other Exchange 2010 mailbox database, it can have the same high availability by being replicated within a Database Availability Group.

Further reading:

Importing PST Files to Archive Mailboxes

For organizations that already have PST files on the network, one of the goals in the implementation of an email archiving solution may be to import those PST files into the archive so that they can be removed from the network for good. For Exchange Server 2010 native archiving there are two solutions available for this.

The first solution is to perform PST imports into the archive mailboxes. This has been available since Exchange Server 2010 SP1, however it requires that the IT administrators be able to locate the PST files and either manually run PowerShell commands or write a script to perform the import operations.

The second solution is to use the PST Capture Tool that Microsoft released in early 2012. This tool assists with the discovery and import of PST files on the network into either the primary or archive mailbox for Exchange 2010 and Office 365. Although a little more complicated to set up it does solve some of the issues associated with the previous manual approach.

Further reading:

Archiving Email Automatically via Retention Policies

When a mailbox user has been enabled for archiving they are assigned a retention policy.

Exchange 2010 mailbox with retention policy

A retention policy is made up of a group of retention tags – at least one default tag, and then any number of additional policy tags or personal tags.

Although the tags that you are licensed to use in retention policies differ between the Standard and Enterprise CAL, since archiving is an Enterprise CAL feature anyway I will assume that you are entitled to the full range of retention tags.

There is already one default retention policy installed with Exchange Server 2010 so you should take a look at it first before you begin enabling mailboxes for archiving. The outcome of the default policy is that it will archive any items more than 2 years old, unless the mailbox user assigns one of the available personal tags instead.

Exchange 2010 default retention policy

Alternatively you can create your own retention tags and policies to suit your organization’s archiving goals, such as this 30 day archive policy I’ve configured in my test environment.

Exchange 2010 new retention policy

You can add existing mailboxes to the new retention policy when you are creating it, or add them later.

Exchange 2010 add mailbox to new retention policy

Managed Folder Assistant

The Managed Folder Assistant is the agent that processes mailboxes and applies retention tags and performs actions on the mailbox items.

In Exchange 2010 RTM the Managed Folder Assistant operated on a schedule, however in Exchange 2010 SP1 this was changed to a “workcycles” approach instead, where the assistant runs and stops as necessary depending on the load the mailbox server is under at the time. This improves the reliability of the Managed Folder Assistant by increasing the likelihood that it will process all mailboxes in larger environments where a scheduled approach may not be suitable.

You can also manually run the Managed Folder Assistant from PowerShell.

Start-ManagedFolderAssistant -Identity alan.reid

The outcome is quite obvious. Here is the mailbox for Alan Reid prior to the Managed Folder Assistant running.

Exchange 2010 mailbox before archiving

And here is the same mailbox after the Managed Folder Assistant has run.

Exchange 2010 mailbox after archiving

Summary

As you can see from this brief overview, Exchange Server 2010 native archiving can be used to effectively manage the size of mailboxes by automatically moving items into personal archives.

However proper planning should always take place before implementing any email archiving solution. In the case of Exchange 2010 native archiving, considerations include:

  • Is the organization licensed correctly for personal archives?
  • Do the client computers have a compatible version of Outlook installed?
  • Where will the archive databases be located, and will they be replicated within a Database Availability Group?
  • How will the archive database be backed up?
  • Which retention tags and policies will be assigned to mailboxes?

 

Comments

  1. Christian says

    Hi Paul, excellent guide, thanks for share it.

    There’s any chance to have an OST file of that Archive database in order to access it offline? In all my test I can access to “archive” only when connected.

    Thank you!

    • says

      In Exchange 2010 native archiving the archive mailbox is only available online, it is not cached by the Outlook client.

      If cached/offline archive access is a strong requirement for your organization then you may wish to look at some of the other third party archiving products that do support that scenario.

  2. gengaiyan says

    Hi Pall,

    It’s very quit explanation, thanks for.sharing.

    Is it will create whitespace in mailbox database once retention policy run?

    Also do you have document for ediscovery ?

    Thanks
    Gangaiyan

  3. David says

    Thanks for the overview, I’m looking for a solution that restricts access to users archive with an auditing and authorisation workflow element for management of access. I have yet to fins anything, do you have any ideas?

    Thanks

    David

  4. Roger Haines says

    Hi Paul,

    Great article. I’m just getting started with Exchange 2010 and it’s archiving capability. Is it right that there is no search capability of the archived messages? We currently used Outlook 2007 and it’s my understanding that support for that version of Outlook was added in a roll-up Office 2007 update, but would not support the archive policy retention tags built into Exchange 2010. Right now I’m just trying to understand how it’s configured to be used. I have it enabled on the server, for my user account and selected a database to be used. Now how do I go about configuring it on the client side? I haven’t been able to locate a document that describes how to do that.

    Thanks for any help you can offer.

    Roger

  5. Harmeet Sandhu says

    Hey Paul,

    You dont need to do anything on the client side. The archive folder will appear once you restart outlook.

    Harmeet

  6. Jim says

    Hi Paul,

    We’re just starting to define our end user documentation and policies for deployment soon. I recall reading that when you toggle the archive option on an enabled mailbox, the archive is outright deleted and not recoverable, is that correct? What about terminated staff… we normally disable the account for 30 days, then delete the account, but the mailbox is retained for another 30 days; what happens to the archive mailbox during the same time window?

    Also, if we chose a 3 year retention and deleted anything older, what would be the effective difference between setting Single Item Recovery to 3 years vs using Legal Hold?

    Thanks,

    Jim

  7. Michael says

    Hi Paul

    I have a question regarding the actual archiving itself.
    We setup archiving policy tags that sais that all items older than 6 months have to be archived.
    I applied the policy last week but nothing happens

    Do i have to wait another 5 months and 3 weeks before the policy kicks in?
    On what does it base itself for archiving?
    The creation date, the receive date, or the date when the policy is applied?

    Thanks a lot for the help!

    Michael

      • Michael says

        Hi Paul

        Yes I Did.
        I’ve created a retention policy with a default retention policy tag (move to archive after 6 months for all other folder) and assigned this policy.

        That’s why i ask when die policy kicks in and for what items.
        The items just older than six months or the items older than six months from that date the policy is applied?

        Many thanks

        • says

          And you’ve created an archive mailbox for the user?

          It is based on the age of the item. In my example towards the end of the article the mailbox “alan.reid” had a bunch of mail that immediately got archived after I manually ran the managed folder assistant (which runs on a continual basis anyway, depending on server workload at the time).

  8. Michael says

    Hi Paul

    Yes and that is just the strange thing.
    He has items much older than six months.

    I’ve also started the mananged folder assistant yesterday but nothing happens.
    Really strange.
    For now i will ask the user to manually archive and we will try again with a policy for 1 week.

    Many thanks for the help!

  9. Lisa Russell says

    I have implemented Auto Archiving and while its great it does not appear for some users, also while it appears for a user on one computer, it doesn’t when they log onto another computer – weird considering its not local and associated with Outlook.
    so my questions are:
    1- When right clicking on user mailbox and selected to enable Archiving, why does it appear for some users and not all?
    2- Why would it not appear for a user when they log onto a different computer/

    thanks

    • says

      If the mailbox is on a server that is running an earlier version of Exchange you wouldn’t be able to archive-enable it.

      The visibility of the archive mailbox also depends on having a compatible version of Outlook installed on the PC.

  10. Luke Humphreys says

    Hi Paul Great Article!

    Was wondering if you have posted anything regarding upgrading to Exchange 2010 SP1? Or you are aware of any good articles.

    I’ve inherited an exchange 2010 environment and been tasked with enabling archiving. The business currently archives using an unreliable third party product and have intermittent issues restoring from archive.

    I’m looking for something around best practice and also concerned at any issues that can arise. Although, I will be fully backing up the server before I make any changes.

    Should I just build a new 2013 server and migrate to that?

    Thanks in advance,

    Luke

    • says

      An Exchange 2013 migration would require Exchange 2010 SP3 anyway, so you’re not going to get away without upgrading 2010.

      Basically you’ve got a bunch of reading to do. Read the release notes for SP1, SP2, and SP3, and then the release notes for any post-SP3 update rollup you plan to apply as well.

      (You don’t need to install SP1, then SP2, then SP3, you can go straight to SP3, the point is more that you need to read all the fixes, updates, and changed behaviours for each)

      Also read the Exchange team blog posts for each SP release as they explain a lot of things.

  11. AlbertWT says

    So how does this feature compares with the Symantec Enterprise Vault archiving ?

    I couldn’t find any PDF or website comparing the feature side-by-side. Not even Symantec is able to comment on this comparison.

  12. EugeTP says

    Hi Paul,

    great article.
    we are planning to setup a separate DAG environment for archiving. right now we have DAG for regular mailbox. (e2010 sp1). is it possible to have a separate DAG for archive? do you have articles that i can read and follow to build from scratch.

  13. Trevor P says

    We got everyhting up and running now.. Exchange 2010 is a huge boost to us over 2003 :) However, there is one thing hapenning that we don’t understand.

    We have Archiving enabled on a user, with the proper retention policies applied, archive quotas are set to unlimited. We can see emails moving from their mailbox to thier personal archive according to policy as things should..

    however… it seems that the retention policies are also being applied to their Personal Archive! Things are disappears out it.

    In Outlook, if we highlight the Inbox located in the Personal Archive,then select Folder, then Policy; it shows that the Retention tag for “Inbox” items is being applied to it! Indded, individual messages in that folder also show a Retention Policy and an Expires warning on them I tought this should never happen? This is the Personal Archive folder, not the actual user mailbox.

    Any ideas? I’ve trolled all the groups and can’t find anything like this. How do you disable Retention Policy on the Personal Archive if this is “standard” behaviour?

  14. Ruben Kirakosyan says

    Hi Paul,
    I am in the middle of migration from ex10 to ex13. There are some mailboxes which are archive-enabled in ex10. So the problem is, that, when I try to migrate that mailboxes from ex10 to ex13, it get queued, and nothing more. I also tried to do it from cmdlet “New-MoveRequest -Identity ‘ayla@humongousinsurance.com’ -TargetDatabase DB01 -ArchiveTargetDatabase -DB03″, but there is an error which said, that my archivedatabase coundn`t find. I guess, that it is because my databases are on different machines. I haven`t found any information how to do such kind of task. Could you help me?

  15. says

    Hey Paul,
    Great article and thanks for sharing! We are getting ready to enable archiving in our Exchange 2010/Outlook 2010 environment. Trying to find information on best practice post configuration settings on the cached Outlook 2010 client after the archiving retention policy is enabled? For example, it is recommended to delete the OST file and recreate after archiving is enabled? Thanks in advance, Rod

  16. David Fletcher says

    Paul,

    How do you restore an archive mailbox. The archive is in another db. Do I have to restore the both the original mailbox and then the entire db that the archive is in?

    • says

      Only the DB that held the archive mailbox would need to be restored as far as I know. I’ve never actually had to restore an archive mailbox so if there are any caveats to the normal restore process I don’t know them sorry.

  17. Mike Dunne says

    Hello Paul,
    Do you know if you can use personal archiving and litigation hold at the same time? Can both be active?
    thx

    Mike

  18. Stephen Wilde says

    Thank you very much for this detailed article, Paul. Our Exchange DBs (4 of them) are on one SAN volume (RAID 10). Is a Best Practice to create archive DBs on a separate volume (RAID 10 or 5) ?

    Thanks much.

    • says

      There’s no best practice that I’m aware of specifically for archive DB placement, but you should certainly put your archive DBs on storage that provides sufficient IOPS/performance to run.

      One way I’ve seen orgs reduce costs with archive DBs is by placing them on less resilient storage, for example they may put their primary mailbox DBs on RAID10 and archive DBs on RAID1. Similarly, in DAG deployments they may have 4x copies of primary mailbox DBs and only 2x of archive DBs.

      It all comes down to what the business is willing risk to save money :)

  19. Marc Wenger says

    Hi Paul,
    This article (http://blogs.technet.com/b/exchange/archive/2011/08/05/prevent-archiving-of-items-in-a-default-folder-in-exchange-2010.aspx) says Notes will still get archived even if we setup a DPT to exclude Notes due to a conflict of rules. It goes on to mention a “workaround”: for each mailbox Notes folder, apply a personal tag to prevent archiving. Can PowerShell be used to set the retention on notes? What would such a PowerShell script look like? Thanks!

  20. Yasir says

    Hi, i have configure auto-archive for my outlook user, after auto-archive completed a separate file is created but existing OST file has still the same size, it removes email from exchange server but not from OST file. i need suggestion regarding when i enable archive its automatically remove emails from OST and also from server. kindly suggest me solution.

    Awaiting for your prompt response.

    Thanks

    • Trev says

      This is by design. OST files do not automatiically shrink when you remove or delete items from them. Like most other DB type files, they “keep” the space on disk, but mark the area as “available” in the file. This actually increases performance as that disk is already preallocated and the OS doesn’t have to go looking for extra space to give to the file. Your emails are removed from the OST file; the space is just marked as free.

      The only way to shrink the OST file that I am aware of is either
      1) compact the file (google : OST file compact)
      2) remove the OST file completely and let Outlook rebuild it. This can be CONSIDERABLY faster, especially if you’ve removed a large number of emails. This does NO harm to the users mailbox as everything is already on Exchange anyway.

  21. User_Feo says

    What is the best way to handle the archiving process for terminated users(mailboxes)? The plan: Archive the mailboxes of terminated users, immediately delete their account, have a retention period of 30 days. The questions, will mailbox remain the archive db, even if the account is deleted, and I presume the emails get deleted the retention period is met. Am I on the right track?

    Thanks!

  22. Tom says

    Paul,
    I have come across many topics involving Exchange, Mailboxes, and archiving that all include your signature. the one i have not yet found is the one that includes a good or even decent PS command line for archiving a mailbox from the exchange server. i have seen a few which include the command line:
    New-MailboxExportRequest -Mailbox alan.reid -FilePath \\esp-ho-ex2010a\pst\alan.reid.pst

    while i have modified that line to meet my needs, the issue i have been facing is that the system does not see the “Microsoft Mailbox Replication Service” as running. i found yet another site that marked “Set-Service NetTcpPortSharing -StartupType Automatic” as a fix to this issue, and while it turns the Net.TCP Port Sharing service to Automatic start (which then the admin needs to start), this does not resolve the issue.

    In a nutshell, i need to archive off an ex-employee’s .ost file and I much rather prefer to do this from PS.

    suggestions?

  23. Joe says

    I’m coming a bit late to the party but just came across this article on archiving. I’ve noticed that you can’t move inbox to archive. My experience is that most users don’t take the time to move emails to folders and their inboxes fill up. How do you go about archiving inbox items based on age?

  24. Leonard says

    Hi Paul ,

    I definitely need you help for some Exchange Server 2010 / Sp3 configuration :

    I have setup Database Limits : Issue warning -5000 Mb , Prohibit send – 5100 Mb and Prohibit send receive -5200 Mb .
    Now to optimize Exchange i have to lower these limits in 1000-1100-1200.
    I have plan to enable archive online for each user , right now many of some users has mailbox quota about 5000 Mb .
    How to setup a retention policy that MOVE to ARCHIVE all old emails ( from all folders ) and leave in Primary mailbox only 1000 or 900 Mb of quota ?
    So after that i can change database limits to 1000-1100-1200 .

    What is happened to a user that has 5000 Mb mailbox quota if i setup database limit to 1000 1100 1200 without archiving old mails ?
    The difference 5000-1000=4000 Mb will be deleted ?

    thanks
    leo

    • says

      Enforcing quotas doesn’t delete anything already in the mailbox. It will prevent them from sending/receiving new email if they’ve exceeded those quotas though.

      What you’re looking for is a report of the age and size of the items in the mailbox, so you can estimate how effective an archiving policy will be at reducing the mailbox sizes to the desired quota levels. You can do that with EWS, such as by using this code sample by Glen Scales (Exchange MVP and expert on EWS) http://gsexdev.blogspot.com.au/2012/10/reporting-on-item-age-count-and-size-in.html

      • Leonard says

        Hi Paul ,

        Thanks for your fast reply , but this script is very hard for me to be implemented , so i have to find another way. Maybe archiving mailboxes with retention days starting from 90 days and lowering , until i get the proper size for Primary mailboxes.

        many thanks
        Leo

  25. Joe says

    Hi Paul, I was hoping you can help point me in the right direction. I can’t seem to configure a retention tag to archive the inbox. I can set move to archive for all other folders, but when I try to create an archive for the inbox it fails. I will only let me delete and recover or permanently delete inbox emails.

    Is there a way to create archive tags for users inboxes?

  26. Kevin says

    Paul, I think I have setup everything fine for the exchange archive for my users, and I just bought them office 365 Pro plus so they are on a compatible version but when I go into outlook the auto archive is there for archiving to a pst, but the policy buttons aren’t and even if I customize the ribbon to put them on, they are greyed out. When I right click on a folder the auto-archive tab is still there instead of the policy tab. What could be wrong?

Leave a Reply

Your email address will not be published. Required fields are marked *