How to Defrag an Exchange 2010 Mailbox Database

Exchange Server 2010 mailbox databases grow in size as the data within them grows. But they will never shrink when data is removed from them.

For example if you have a 20Gb mailbox database file and move 4Gb worth of mailboxes to another database, the file will remain at 20Gb in size.

However, the database itself will have 4Gb of “white space” in it, which is space that is available for new data to be written without growing the size of the file.

Your options to reclaim that space are to either:

  • Create a new mailbox database and move all the mailboxes to that database
  • Perform an offline defrag of the existing database to shrink the file

Each option has pros and cons. An offline defrag involves an outage for all users on that database, but may be more convenient if there is not additional storage available to allocate to the Exchange server to hold the new database.

On the other hand a mailbox migration has fewer risks, can be less disruptive as a whole, but will generate a lot of transaction logging that needs to be kept under control so it may take longer (ie several nights/weekends to migrate) as opposed to just one outage for a defrag.

Determining Free Space in an Exchange 2010 Mailbox Database

In Exchange 2010 you can see how big your mailbox databases are, and how much white space they have, by running the following command in the Exchange Management Shell.

[PS] C:\>Get-MailboxDatabase -Status | ft name,databasesize,availablenewmailboxspace -auto

Name             DatabaseSize                    AvailableNewMailboxSpace
----             ------------                    ------------------------
MB-HO-01         18.26 GB (19,604,766,720 bytes) 9.544 GB (10,247,766,016 bytes)
MB-HO-02         15.63 GB (16,785,670,144 bytes) 3.696 GB (3,968,761,856 bytes)
MB-HO-Archive-01 648.1 MB (679,542,784 bytes)    134.6 MB (141,164,544 bytes)

In the example above the database MB-HO-01 is 18.26Gb in size but has 9.544Gb white space due to archiving that has occurred. If you want to reclaim that disk space then the file can be shrunk by using eseutil to defrag it.

In this example I will demonstrate how to defrag a mailbox database for a single Exchange 2010 Mailbox server that is not a member of a Database Availability Group.

Do not follow the procedure in this document if your Mailbox server is a member of a DAG. Before you defrag any mailbox database please read and understand the pros and cons of this operation and make the best decision for your specific situation.

Preparing to Defrag an Exchange 2010 Mailbox Database

The first thing to be aware of when planning a defrag is that you can only perform this task when the database is dismounted. This means that users with mailboxes on that database will not be able to access their email while you are defragging it.

The second thing to be aware of is that you need some available disk space to perform the defrag. This is because a new file is written during the defrag process, so for a period of time both the old and new files will exist, as well as a temporary file that eseutil creates.

So to plan for an Exchange 2010 mailbox database defrag you need an amount of free space equivalent to 1.1x the predicted size of the new file.

In this example that would be:

18.26 – 9.544 = 8.7

8.7 x 1.1 = 9.57

In other words, I’ll need about 10Gb of free disk space to run this defrag. Since I don’t have that much free space on the same drive as the database I will need to specify a different temporary location when I run eseutil. This can be another local drive or a UNC path, just be aware that if you are using a UNC path the defrag will take longer due to network latency.

Before proceeding you should be sure that you have a good, working backup that you can use for recovery if something goes wrong during the defrag.

Using ESEUtil to Defrag an Exchange 2010 Mailbox Database

Open the Exchange Management Shell and navigate to the folder containing the database file.

cd D:\Data\MB-HO-01

Dismount the mailbox database.

Dismount-Database MB-HO-01

No run ESEUtil to defrag the file.

[PS] D:\Data\MB-HO-01>eseutil /d MB-HO-01.edb /t\\testserver\defrag\temp.edb

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 14.01
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating DEFRAGMENTATION mode...
            Database: MB-HO-01.edb

                  Defragmentation Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................

Moving '\\testserver\defrag\temp.edb' to 'MB-HO-01.edb'...
                     File Copy Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................

Note:
  It is recommended that you immediately perform a full backup
  of this database. If you restore a backup made before the
  defragmentation, the database will be rolled back to the state
  it was in at the time of that backup.

Operation completed successfully in 3788.218 seconds.

Mount the database again.

mount-Database MB-HO-01

You can now see that the file is smaller, and all the white space is gone.

Get-MailboxDatabase -Status | ft name,databasesize,availablenewmailboxspace -auto

Name             DatabaseSize                    AvailableNewMailboxSpace
----             ------------                    ------------------------
MB-HO-01         8.328 GB (8,942,190,592 bytes)  5.219 MB (5,472,256 bytes)
MB-HO-02         15.63 GB (16,785,670,144 bytes) 3.696 GB (3,968,761,856 bytes)
MB-HO-Archive-01 648.1 MB (679,542,784 bytes)    134.6 MB (141,164,544 bytes)

As you saw when ESEUtil completed you should run a full backup of the database at your next backup window.

Comments

  1. Garry says

    I would recommend creating a new database and just moving the mailboxes. You achieve the same thing but with very little or no dommwn time.

      • says

        I’d love to use extra mailbox databases to rebalance our Exchange server, I really would, but Microsoft in their infinite wisdom have limited our version of Exchange to 5 active mailboxes.

        Oh, ok, we chose to limit ourselves to it, for $$$ reasons.

        Still, good advice there regarding planning mailbox database usage and spreading mailboxes across them. It all makes perfect sense and when we do our next Exchange server we’ll be better prepared, as well as be able to use it to receive mailboxes from our current unbalanced Exchange server.

        Thanks for the pointers Brian.

      • chatnakhon says

        D:\Data\MB-HO-01>eseutil /d MB-HO-01.edb /t\\testserver\defrag\temp.edb

        what is \\testserver\defrag\temp.edb ?

        thank you

  2. says

    You article is totally correct in every technical way Paul (like always) but I would have to agree with Garry. There are some unwanted sides effects to offline defrag and I would always prefer mailbox move when possible.

    • says

      What side effects are you most concerned about? There are risks with everything of course, so having backup and rollback plans is important.

      Also moving mailboxes means you’ve got to manage all the transaction logging that generates, as well as have enough storage allocated to Exchange to hold both the old and new databases together (as compared to a defrag where you can just use temp space somewhere like I demonstrated). Moving a lot of mailboxes could take several nights to complete whereas a defrag can usually be done in just a few hours even for a big database.

      I agree that moving the mailboxes is probably the *lowest* risk but it does add other considerations to the situation.

  3. says

    Thanks for taking the time Paul. Let me first clarify that I don’t by any mean disagree with the content of your article. My point was that there might be other ways that could be considered before an /D.

    My major concerns are what happen when offline defrag encounter errors and can’t be completed and the fact that restores from earlier backups are more or less impossible (without third party tools or a lot of work).

    The last “official” comment on this by the team is rather old but I would say it still applies. http://blogs.technet.com/b/exchange/archive/2004/07/08/177574.aspx

    • says

      Sure, these are all things for the admin to consider before diving in. Each scenario will be different.

      Hopefully the admin has working backups to fall back on if a defrag fails. Whether backups are working or not is a whole different problem ;-)

      You’ll notice too on that article you link to that they spell out the scenarios in which they DO recommend a defrag, and reclaiming disk space is one of them.

      All good discussion points. I’m going to update the article with a few extra “Before you proceed…” notes just so people are aware there is other things to consider.

  4. Brian Legg says

    Great article,
    In Exchange 2010, if we move mailboxes to a different database and we are using both WindowsBackup 2008r2 and DataProtection Manager, when do the logs get committed, with DPM or Windows Backup?
    Also, I noticed when i moved 1.5GB of data out, my free space is still only 100mb+.
    Can you comment?
    Mailbox 82.01 GB (88,055,283,712 bytes) 102.2 MB (107,151,360 bytes)

    I want to migrate all the mail out of this store, but unfortunatly I only have 100GB on the drive, total, 15GB free, so when i move I can only move about 5GB per day?

    • says

      Hi Brian, I don’t understand the first part of your question. You’re backing up the same Exchange server with both Windows Server Backup and DPM?

      For the second part, the white space isn’t created until online maintenance has had a chance to finish.

  5. Genghis says

    Hi Paul,

    I’m on a verge of Defragging my Exchange 2010 and is about to follow your information but there’s one issue. You clearly outlined that “Do not follow the procedure in this document if your Mailbox server is a member of a DAG”. Unfortunately in my environment, my Mailbox server is a member of a DAG and you didn’t state an alternative way to defrag.

    Can you please provide me some information on how to defrag my Exchange 2010 Mailbox Database which is a member of a DAG?

    • says

      Hi Genghis, the issue with DAGs is that the database is replicated to multiple servers for high availability. But when you defrag your database (the active one) you then need to reseed that to each of your passive copies as well.

      So while that reseed is occurring you’ve broken the resiliency of your DAG – the database no longer has multiple copies/replicas and is at risk if the primary/active copy fails for some reason.

      So if you have a DAG, the safer way to do it is to set up a new database with its own replicas, migrate the mailboxes to the new database, then remove the original/empty database.

      • Genghis says

        Hi Paul,

        Firstly, thank you for your quick response and I found your information very helpful. I was wondering if you have any online articles that explain in depth about Defragging Exchange 2010 Mailbox Database in a DAGs environment. I just want to know things inside out before I start commencing the whitespace issues.

        Your help is deeply appreciated.

      • Genghis says

        Hi Paul,

        Did you have the chance to find any useful documents about how to Defragging Exchange 2010 Mailbox Database in a DAGs environment?

      • says

        No I haven’t, but my previous advice still stands. It is safer in a DAG environment to move mailboxes to a new, replicated database, than it is to temporarily break the resilience of an existing database by defragging and reseeding it.

  6. Michael says

    Interesting dicussion on Exchange DB growth — couple of questions.

    1. If moving the mailboxes to a new DB fixes whitespace — will it prevent future whitespace issue from occuring once in a new DB?
    2. Is this ‘normal’ behavior for Exchange 2010 to create whitespace and not have the ability to reclaim it and one is left with offline defrag and/or the moving of mailboxes to a new DB? This does not seem practical given an organization may have hundres if not thousands of mailboxes?
    3. Anyone figure out the root cause of what may be causing the whitespace from being reclaimed thus leaving us with options to degrag or move boxes?

    • says

      Hi Michael,

      1. “Whitespace” is created when data is removed/purged from the database. For example moving a mailbox off the database, or users deleting items (and then the items passing the deleted item retention period etc). Whitespace is not something you can prevent in that sense.

      2. Yes, this is normal. The database, through its own online maintenance, zeros out the unused database pages and they become whitespace that new data can be written to. If you ignore the whitespace it will eventually be reused as new data is written to the database.

      During an offline defrag the database is dismounted, and then a new database file is written with the contents of the old database file minus the empty database pages. That is how the file is made smaller.

      Yes this is inconvenient, and in some organizations is unacceptable due to SLAs. It isn’t something that would be considered routine maintenance.

      3. The “root cause” is as above. This is normal, expected behaviour of the database.

      As a side note, while it would be great if defrags could be done online (maybe using VSS) the only times I find myself doing them is when absolutely necessary. I prefer to put more effort towards monitoring and proactive maintenance to keep mailboxes well distributed across available databases and try to manage growth that way. Archiving can help with that too.

  7. vadim says

    This is almost a good article. Almost, because its not exactly complete, a number of the most important things are left out, those things that are the reason why people are browsing Internet instead of opening MS help or an admin guide for Exchange. Here is an example.

    I got a 200GB store. I moved a number of mailboxes out of it to other stores, at least 30GB worth. Waited for maintenance to run its course. Checked this (rather obscure) parameter “availableNewMailboxSpace”. Still the same 100MB or so. Moved antoher 10 GB worth of mailboxes. Should be plenty of ‘white space’ now, right? Not so fast. Still the same and actually DB grew meanwhile another 3 GB to 203GB overall. Whats the…?
    OK. Forget this. While there is space on the disk – moved ALL the mailboxes out’ohere, For the heck of it, dismounted the store and ran offline defrag. Completed successfully, ran maintenance, check DB. Guess what? Size – 203 GB, SpaceAvailableForNewMailboxes – 128 MB. In an empty, defragged store!
    According to MS documentation and this very article something got to be wrong. Well, Im afraid not. That is besides MS poor documentation, bugs and design deficiencies.

    I have some guesses but anyone here have some explanation?

    • Brian says

      Very interesting, did you purge out the disconnected mailboxes after moving them? Because it actually does not move them but rather copies them to the other database file and then marks the original as disconnected.

      • vadim says

        Yes, ‘no, I did not’. Thats what I thought maybe is the problem. Not very obvious. To me at least. So ‘move’ is not really a ‘move’ in Ex2010, for strange reason its a copy (??). So no white spaces. Actually, deleting email is also ‘no white space’ deal – after deletion it still stays in the store according to retention time (at least this makes sense). So defrag is not really a defrag – it would not do anything except wasting your time and thrashing the disk unless first the “white space’ is really created. Manually.
        To do that you need to run this wonderful command if you want to create ‘white space’ after moving (sorry, copying) boxes to elsewhere:

        Get-MailboxStatistics –Database “dbname” | Where-Object {$_.DisconnectReason –eq “softdeleted”} | ForEach {Remove-StoreMailbox –Database $_.database –identity $_.mailboxguid –MailboxState softdeleted}.

        Same to purge ‘disabled’ boxes with replacing “softdeleted” parameter with ‘disabled’. If you got a little scared with this monster its worthwhile trying more civilized looking comand:

        clean-mailboxdatabase -identity “Mailbox Database”
        Allegedly it should do the same though I did not try it and dont know if it will purge those ‘softdeleted’. ‘disabled’ or both types.
        Hate to mention but still running either command won’t do a thing for you. You see, there is another obvious thing (oh, you did not know that??) – to make any difference you still need to change the setting for retention period of deleted items to “0” (at least this is on the properties of the mailstore in question on the ‘Limits’ tab of GUI). Otherwise those shell commands will run fine but wont do a thing and they apparently are under no obligation to tell you that something is wrong.

        Did not experiment with it but pretty sure just changing retention period to “0” wont help with items that were deleted or moved (sorry, copied) before you made it to “0”.

        So to really clear and reclaim space in Ex2010 db you need to follow these steps (that I beleive should be included in any good article on Ex2010 defragmentation):
        1. Move whatever mailboxes you can to other dabatase.
        2. change retention period for database for deleted email and\or maibloxes to ‘0’
        3. purge ‘ghost’ mailboxes with one of the above commands and possibly purge deleted email (another shell command that is availble on Internet)
        4. dismount store
        5. run eseutil \d (possibly with other options, like location of the temp db)
        6. if after all this you still have any interest in it, check the size of db and mount it

        If you are in a DAG I suspect you need to run eseutil \d on each copy of the store while its offline. I am and I did not want it so I moved all mailboxes out, still ran all the purge procedures (have to, or the following step will not work) and deleted the offending store. Recovered almost 50% of space. Most positive thing – I understand now why MS does not really recommend offline defrag.

        Its not a place to express personal feelings but with Exchange 2010 my feeling is like – you bought Mercedes and… Oops… its a Hyundai. Well, Ex2010 is not a Hyundai product, its mostly “Made in India” but I think you got my point. And no similar products with “Made in Germany” label on the market, so I guess we just stuck with it.

  8. Raj says

    Paul,

    I have to perform Offline defrag on e one of f my Ex2010SP1 ru1 DB. I have performed offline defrags on earlier versions of exchange however, not in a DAG situation. If I perform the following steps what are the repercussions:
    – Suspend the database copy(on exserver2) for the “Passive/Healthy” copy. Dismount this copy and perform offline defrag. Once the defrag is complete, mount it, and resume copying/ replication. Once all the files are copied over, make this the active copy ( mount). Now db ( copy on exserver1) becomes passive. Ressed the db from server2 to server1.
    Is this a workable solution? this way the end users will be unaffected… I think?

    Any feedback would be appreciated.

    thanks

    Raj

    • says

      Hi Raj, as Brian mentions below you can’t dismount *just* a passive copy of a database.

      And also you can’t run an ESEutil defrag on a database that is not in a clean shutdown state, in other words has been properly dismounted.

      So I’m afraid what you propose is not possible. The safest way to reduce the database size of a replicated database with the least possible user impact is to create a new replicated database in the DAG and move the mailboxes to that new database, then remove the old one.

  9. Brian says

    Not quite sure i follow you, but i just did this yesterday and here is what i did.
    The active server is EXMAIL1

    Suspend-MailboxDatabaseCopy DVSDB2\EXMAIL2
    Suspend-MailboxDatabaseCopy DVSDB2\EXMAIL3

    dismount-database “dvsdb2″

    eseutil /d dvshosted.edb /t “f:\temp\temp.edb”

    mount-database “dvsdb2″

    Update-MailboxDatabaseCopy -Identity DVSDB2\EXMAIL2 -DeleteExistingFiles
    Monitor to verify replication maybe even test replication

    Update-MailboxDatabaseCopy -Identity DVSDB2\EXMAIL3 -DeleteExistingFiles
    Test replication

    Run a full backup to truncate the logs.
    We use DPM so an incremental work to truncate and commit the logs.

    The Lord was my helper so all went well!
    Hope this helps.
    Brian

    • vadim says

      Brian, I think the Raj was asking not how to defrag the single db but would it affect other copies in DAG (and as a result users) if one db is suspended and then dismounted (and then defragged). I also wonder if thats the case. Just dismounting the db will simply dismount it for the whole DAG. Seems you can’t really ‘dismount’ a passive copy since its not a mounted db, just a copy. So can’t defrag the db without taking db offline for users. Also, if you do need to dismount db you don’t really have to suspend replication – just dismount db and replication will stop itself.
      The question though is would it be possible to run defrag on passive copy when replication is simply suspended, without dismounting the db? Then users will not be affected.

      So the question still stands – in your case (or anybody else) did you manage to defrag a passive copy of db without interrupting service for users? If ‘yes’ how exactly it is done, whats the logic?

      And another question (also for you or for anybody else) – I noticed that very often as is here in Brian response people would throw in a handful of PS commands to show how to perform some tasks. The question is – “whats the reason?”. Like in this case simply right- and then left-clicking on the db in question will suspend and dismount the db. Another two clicks and it will be mounted it and resumes replication.

      So why run a set of some obscure commands (the syntaxes of which you cant memorize unless you run them every single day)? Should not be efficiency considering a need to memorize or lookup a reference for the specific command and then rerun it again after mistyping it and choosing appropriate switches versus just clicking a couple of times on the mouse with one finger while keeping a cup of coffee in another hand.

      So is there particular reason to employ PS commands in such cases (aside from opportunity to enlist the Lord as a helper)?

  10. Brian says

    Actually we are a hosted exchange provider and when exchange 2010 is installed and run in /hosting mode then there is no Exchange Management Console. Hence, everything is done via the PS. I wish it were not so, but over the past year it has forced me to become familiar with the commands and how they work.
    In fact our other installaitons that are not hosted modes have different PS command structure for various things.
    Commands such as
    Get-MoveRequest -TargetDatabase dvshosted
    does not work
    it needs to be
    get-mailbox -database “dvsomdb” | get-moverequest

    Mainly the differences are due to the isolation mode of the hosted exchange clients..
    And no, I would not attempt something something via PS just to enlist the Lord as my helper, but I would be remiss if I did not give him praise.

    Apart also from the hosted mode, an understanding of the PS is beneficial. I can run commands such as:

    Get-Mailbox -database “DVSOMDB” | Get-MailboxStatistics | Sort-Object TotalItemSize -descending |Select-Object DisplayName,ItemCount,@{name=”MailboxSize”;exp={$_.totalitemsize}} -first 10

    To see who the big 10 offenders of email use are. I don’t believe this can be done in the EMC.

    With all due respect…. I acknowledge that I am far from being an expert, I was just trying to share what I had learned and what was possible.

    Regarding his/your question, re: “The question though is would it be possible to run defrag on passive copy when replication is simply suspended, without dismounting the db? Then users will not be affected.”
    How would this be possible since the command to dismount-database is server independent? ie…
    dismount-database “dvsdb2″
    So I don’t belive it is possible.

    Brian

    • Raj says

      thanks for all your replies and observations. I am going to call MS support tomorrow to find if there is a way to offline defrag the passive copy. I know it is a shot in the dark, but if I find anything worthwhile, i will share it with you all.

      thanks again for all your help and insight.

      Raj

    • vadim says

      Brian, thanks for explanation, I did not know that in hosting mode the GUI is not available at all. Thats BAD. And difference in commands in different modes is simply unacceptable in my opinion. Can only imagine how many hrs you spent trying to figure it all out (with appropriate increase in ROI). I would agree the understanding of PS is beneficial. Running scripts on rare occasions you need them would not be possible without it. So I do understand PS. I just dont want waste my time on it with common tasks that can be done easily through the GUI. In fact, when and if I have to run Exchange in hosting mode (though now it seems I would think hard justifying it intstead of employing something significantly cheaper from the ‘Linux\Unix world”) I would spend some time and write a simple VB interfaces to all the common tasks.

      Regarding the second question – I’m afraid your logic is a litle faulty. The whole point defragging a passive copy (afer stopping replication) is that you DONT dismount the database. Logically, that could be possible. DB that is not replicating does not need to be an open file. But I would still agree with you though for other reasons. I feel that whichever Lord the creators of 2010 enlisted for help was not as good as the one you are employing or may be they did not have any :-) . But my expectations of the product are getting less and less.

      Lets see now what Raj will tell us. Good time to make bets.

  11. Raj says

    After speaking with tech support here is a summary of the discussion:

    If we choose to perform offline defragmentation on the passive copy database ( approx 250gb) as an option to reduce the down time for the users we have the following challenges:
    -Once the copy is suspended the passive copy of the database will be in a “Dirty Shutdown” state.
    -we will have to bring the database to Clean shutdown status using ESEUTIL /R ( replay appropriate log files) and then we can run OFFLINE defrag on it.
    -Once the DB is “offline defragmented”, we cannot “resume copy” as the “disk signatures” have changed. This copy will have to be mounted and the “active” copy will have to be “passive”. (This will result in loss of data as the defrag process will take between 6-8 hours- all changes from the time copy was suspended to the time defragged database was mounted will be lost)
    – To recover the data, we will have to use the database file of the Active Database( not- defragged copy) and mount it in Recovery Database and do a merge operation between the Recovery Database and the current Active Database ( this could take many hours as each mailbox/folder/subfolders will have to be evaluated and altered as per need).
    After this process is complete, the db can be reseeded to the passive copy.

    It is doable probably not worth all the trouble.

    thoughts?

    • vadim says

      Thanks, Raj, for squeezing it out from MS. Seems we have our anwsers. If DB is small then it might be possible to defrag it, with or without DAG. But it also seems it would be rarely a better solution than just moving mailboxes to a new DB in such case. If DB is large than is seems almost prohibitive to defrag offline as I cant imagine organization that have a lot of mailboxes that would tolerate bringing down email for 7-10 hrs (with a possibility of the whole db getting corrupted in the end). In extreme case if I dont have space on existing servers I would rather just build a temporary server, move mailboxes there, delete original DB to release space and then move mailboxes back. One of the three GOOD features of E2010 (aside from DAG and improved ESE engine perforamce) is asynchronous mailbox move that allows to move mailboxes without disturbing users. I think this is the way to go.

      • vadim says

        Well, I’m afraid deciding to run Exchange 2010 one is committing himself to a world of incredible complexities. I, for one, after years running MSExchange starting from MS Mail product found myself now checking around for another solution. At any rate to me it looks that moving mailboxes (either to other server or to newly attached storage like iSCSI) is a much simpler and safer process than offline defragging, particularly that it does not take users offline. I just had a choice and went with this solution exactly because of that. But its nice to have a choice and sure in some cases defragging just might happen to be acceptable. Only I’m afraid it will not happen before I forget all the pitfalls and intricacies of the procedure and I doubt I will have any wish by then to go over it again. But I will certainly remember how to move mailboxes. Whatever way is easier to reach the goal is the best.

      • says

        I meant the process Raj got from Microsoft when I say complex and high risk.

        Otherwise it is simple really.

        Non-DAG? Either do offline defrag (some downtime), or move mailboxes to a new database (no downtime for 2010->2010)

        DAG? Either do offline defrag of active DB and reseed (some downtime, not recommended as it breaks resilience until reseeded), or move mailboxes to a new replicated database (no downtime, and maintains resilience)

        :)

  12. David says

    Cannot see how anyone would consider doing an offline defrag of a production database, issue of SLA’s, potential issues during defrag leaving stores offline for god knows how long.

    Keep it simple, move mailboxes over the course of a few days/weeks to new mailstore, any other approach should be saved for the LAB or an MS training course…

    Real world solution for real world problem defragging is not…..

  13. Kevin says

    Paul, thanks for this very useful article.

    I have administered an Exchange database from the old days (ver 5.5) up to 2010 today. I have successfully performed an offline defrag in all the versions I have used up to, but excluding the present one. Typically, the defrag has run for anywhere from 4 to 8 hours, but then again our edb file today is about 4x the size of the last one I defragged (I have also always written the temp edb to a drive within the server containing the original .edb).

    Recently, we had an errant CRM package process store 120GB of emails into a single mailbox. That mailbox has been emptied now, but as you know, the whitespace remains. The current edb is 162.6GB, and the whitespace is 112.4GB, so the defragged edb should be around 52.2GB. My question is, how could I calculate (at least a reasonable ballpark) the amount of time the defrag will take? I will be creating the temp edb on the same server as the original edb. I cannot afford for it to be down much more than 8-10 hours. If it will take longer than that, I would consider simply creating a new edb and moving mailboxes to it for that reason alone. My only concern with moving mailboxes to a new edb and then removing the present edb is what might happen to OWA and smartphone connections once a mailbox is moved to the new edb.

    Your comments would be appreciated.

    • says

      Kevin, on my servers defragging down to 50Gb would only take a few hours, certainly less than 8-10.

      If you’re moving mailboxes to a DB on the same server (or even just within the same site) there should be nil impact on users, except for potentially during the final step of the move request and then only for a few minutes. OWA users *may* need to logon again if they happen to be using it right at that moment (but most likely won’t need to), and ActiveSync devices shouldn’t even notice.

  14. Dave Purscell says

    Fantastic discussion. Are there any “Best Practices” or user recommendations for distributing users to various databases to simplify on-going maintenance? Mailboxes per database? Max REASONABLE database size (especially in a WAN environment)? Databases per volume? Etc…

    The customer where I am wondering about this has approximately 150 user mailboxes. Due to the nature of their industry, some of the mailboxes are unreasonably big… and unlikely to be convinced to be smaller.

    We currently have them split into 4 databases. All databases are DAG’d to another server at a remote location. Originally they were based on user location. Site 1 users were in the Site 1 database with the active copy at Site 1 and passive copy at Site 2. Site 2 users were in the Site 2 database with the active copy at Site 2 and passive copy at Site 1. Remote offices were in other databases.

    I’m considering splitting the Site 1 and Site 2 databases because of the database growth and maintenance issues. 90 GB might not seem huge until you have to reseed it over a VPN. It would also allow me to collapse whitespace in the process.

    • says

      I don’t know about best practices for numbers of mailboxes per database. It depends how many people you’re willing to have impacted by a DB outage. But also depends on how big those mailboxes are, how fast they grow, etc. I have lots of cases where two databases of the same size have wildly different mailbox counts on them.

      As far a size goes, Microsoft talks about 2TB databases in Exchange 2010 but the caveat is always factors such as backup and recovery SLAs. Work out what your SLAs are and find the max size that lets you meet them.

  15. Carlos Alonzo says

    I have a trouble, I received the folloving message
    Slow Find Row Rate is ## which is NOT less than the threshold value of ## FAILED

    What´s happened? What can I do?

  16. Faisal Khan says

    Thanks Paul,
    Thinking of defrag my db becuase of disk space issue. Database white shace shows 200MB. Does it worth defrag it and hope it ll free up more space than shown or should I try moving DB to large drive?

    Faisal Khan

  17. Vis Naidoo says

    I am running out of disk space on the drive that stores the mailbox database. The DB size is 750GB.

    After moving 1000 users mailboxes to another storage DB on another drive successfully, the above drive size still remains the same. This is DAG environment

    How do I reduce the 750 DB to recover disk space.

  18. Vis Naidoo says

    Hi Guys, all this info has made me understand my environment in a completely new way. I am new to Exchange 2010
    Kind regards

    Vis

  19. Amit says

    Hi,

    Can some one have pointers on MS Exchange 2010 DAG servers mailbox database degragmentation. How to proceed for DAG clusted mailbox databases defragmentation. Is it possible in MS Exchange 2010 ?

    Please let me know,

    Thanks,
    Amit

  20. Adam says

    Hi Paul, I have a Exchange 2010 SP1 DAG environment with 8 Databases. I need to perform a Whitespace reclaim on about 4 of the Databases. The method I am going to use is create a new DB in the DAG and then move all users from Orginal DB to New Temp DB. Dismount the orginal DB and delete the edb files from the Orginal DB. Re mount the Orginal DB that will create a new edb file and then re seed with file copy. Move all users from New Temp DB to New Orginal DB.
    A few items I am not sure about
    1) Do I need to purge the softdeletes from the Orginal DB ?
    2) When deleting the orginal edb file do I also delete the search and logs for that DB (Circular logging enabled) ?
    3) Will the Outlook clients still pick up the maibox at the Clean Orginal DB ?

      • Adam says

        Hi Paul

        The temp DB location will be located on a LUN for the use for performing a recovery of an database for security investigations of past data. Hence the reson I need to move the users back to the orginal location once I have created a new edb file.

      • Rosario says

        Paul, we are going through this very same scenario at the moment being. A collegue of mine thought he had to reclaim white space by creating a new DB, move all mailboxes on it and delete the original DB. Now we have the problem that we can no longer RESTORE Backups from the original DBs! because HP Data Protector stops with the error:

        [170:414] Database dbserverxyz_DB1 could not be found in current Exchange topology.

        I know, with Exchange 2007 we did the trick to delete the DB and recreate it with the same name, and I do not recall having had any problems with Tape-Restores. But in these days it was Exchange itself to manage the Recovery-Storage-Group. And may be this was simply based on the DB’s file name on disk.

        Is there someone out who knows whether in EX2010 we could repeat this very same trick of deleting the original DB and recreate it with it’s original file name? Or has this gone into “Topology”, i.e. some unique ID in the Active Directory that, once lost/deletet, you could never recover, not even with the same DB file name.

        Any hints are very appreciated. As we are stuck we opened a case with HP and I will report back, what they will tell us.

        Thanks a lot, Rosario

  21. Jib says

    Good article and sharing, Paul.

    I got to learn from you and the others in here even I still not read to the end of posted.
    Performance tuning on Mailbox server in my interesting now.

  22. says

    Hi Paul
    Thanks for the excellent article, The scenario; My drive where the maildb and logs are stored is completely full 200mb free :(. (I know poor administration but I wont go into that now) The issue is I ned to move mailboxes to a now mail store that I have created, but in oder to do that I need some free space on the drive where the original mail store resides. in order to keep the mail store active.
    The mail db is 200GB but I have 400GB of log files going back many months.
    The question I have is this; Is it wise to temporarily move old log files off the full drive to allow for space while I move the mailboxes to a new mail store?

    • says

      That implies your backups haven’t been running. You should back up that database immediately. A full backup will then cause the logs to truncate and you’ll get back most of that disk space.

      • says

        Thanks for the response. We use HP’s Iron Mountain Live Vault backup to backup our servers including the exchange servers. Which should truncate the logs after backup and I see the backup shows it has completed successfully so I guess I will have to go sort that out thanks again

        • says

          Quick update
          As the drive was so full backup attempts were failing as there was not enough space for the VSS.
          So a quick work around is to enable circular logging to clear the space and then a back can be completed. Probably not the best solution as backup data will be missing but at least the mail store is working and users don’t appear to have lost any data.

        • says

          Get your backups working ASAP yeah.

          Circular logging should also be turned off once you’ve got the disk space situation under control.

          I take it this means your transaction logs and database are on the same drive? You should probably look into separating them onto different disks to mitigate risk of data loss if a disk fails.

          Also may I suggest you look at implementing this script to monitor your database backups :-)

          http://exchangeserverpro.com/set-automated-exchange-2010-database-backup-alert-email

  23. Erick F. says

    Hello,

    I have a 290GB (right now shows 350GB+) database that I am looking to defrag. I did a lot of archiving and end up with a lot of white space. My question is how long will it take to run the defrag, 2 hours, 4 hours? I will be using a network on the /t directory. I just want to get a feel of how long it will take.

    I do have all my backups up to date.

    Thanks a lot!

  24. SBM says

    I have a situation a little different then I have seen anybody talk about. I began moving mailboxes from a database to another database very slowly a few months ago. When our Exchange servers were setup (3 of them) in a DAG one was setup with less disk space then the others. So as I am moving these mailboxes 2 of the servers are doing OK on disk space, but this one now is down below 20 GB. We have an active and 2 passive copies of the database that the mailboxes are being moved out of, so I guess my question would be, is it safe for me just to remove the copy that is on this server with less disk space and only have the 2 copies while I finish moving the mailboxes over to the new database that has 3 copies?

    I don’t see any other way I can do it, besides adding more disk space to this server. My second question is, is there added risk because of having this original database that is around 80 GB and I have moved about half the users and the other half are sitting on it, with all this whitespace. It is more apt to possibly having a problem, in other words should I make it a hurry to get them moved? So far all the moves I have done have gone just fine without issue.

    My 3rd question would be all of the disconnected mailboxes that are happening during the move, so far ones I have moved have just deleted after the 30 days as they should, but right now I have around 100 disconnected ones since they have been moved recently. When I finish all moves to the new database and have 250 or 300 disconnected mailboxes, is that OK, in other words can I just delete that database older database at that point or do I need to wait out the 30 days or manually do a soft delete on these, what is recommended.

    And my last question for now. When I created this new database, I set it up with circular logging to save space. We are performing backups and have many successful recent backups of it and the original database, which did not have circular logging enabled. Once I get all of the mailboxes moved over to this new database is there an issue with turning circular logging back off? I don’t think there is, but I would rather not cause downtime or do something to the new database, but I prefer to have all of the log files, since we are performing backups and they will be truncated anyhow. Sorry for the length of this post and thanks for the assistance!

    -SBM

    • says

      1) It is your choice how many database copies you run with. If you reduce from 3 to 2 you just increase the risk of a single database/server failure undermining your high availability.

      2) Whitespace is not a risk.

      3) You can remove a database even if it is still retaining deleted mailboxes that haven’t expired yet.

      4) If you’re taking full backups then it doesn’t matter whether circular logging was on or off, its still a full backup that can be used for recovery purposes.

  25. Ken says

    When you guys say move the database? Does that mean moving the database data and log “Path” to a new drive?

    • says

      The database files and log files are two separate things. You can move them both together or just move one or the other. Your choice, depending on what you’re trying to achieve.

  26. Pradip Marek says

    Can you do a exchange 2010 public folder defrag in the same way as per above instructions

    I have a 70GB public folder database on PROD server and and a 270MB database on the DR server. Public folder only had 15MB of actual files in it. Im hopefull I will gain 69GB of free space

  27. Yuan says

    Hi Paul,

    I have a situation where the total dumpster size is around 160GB and the retention period is set to 14 days. I want to purge them asap, so I set the retention period to 0 and the online maintain is scheduled 1-5am each day, but the next day, total dumpster size is around 130 GB. It took about three days to reduce the dumpster size close to none.
    So, I am thinking it is possible that online maintain of a database cannot complete within a single day and may take several passes to complete, but, how do I find it out for sure? I cannot find out any log related to database online maintain.

    Thanks
    Yuan

  28. Dima says

    Many thanks for articles. Do you have any suggestions for this question in DAG environment? Thank You.

  29. Ian Cortez says

    i love the article… it helps me a lot…im having problem in database growth. for example DB1=450GB, move some users to new DB2 and is now around 10GB, but DB1=450GB still. Correct me if im wrong but i would to this way is if you have an extra disk.

    D:1 TB ——– “DB1.edb” = 450GB
    E:1 TB ———“DB2.edb

    since you have enough disk to accommodate. Then move users from DB1 to DB2, after successful move then dismount the DB1 and delete the DB1.edb then you can reclaim the disk space..

    Thnks..
    IAN

  30. YCL says

    Any suggestions for what I could do if I don’t have the free space to run the defrag?
    All my secondary drives and network locations are not able to provide enough scratch space to be able to run the defrag utility.

    I would have throught the MS server was smart enough to use the ‘white space’ when i needed more room for a mailbox or etc. but apparently. not.

    Thanks,

  31. Dave Reaser says

    Paul,

    I was considering doing a defrag but I’m wondering if creating a new database and moving the mailboxes might be my best option.

    Using AppAssure from Dell for backups truncating has stopped and i’m down to about 10gb of space on the drive. Poor design initially allowing only 233gb for disk space.

    I am considering adding a 1tb drive and then creating a new db on that drive. Then moving mailboxes.

    Do you have a guide on this process or a link to the steps to do this?

  32. Ronny says

    Thanks a lot for this Paul!

    Followed this last night as we have to do an offline defrag and worked a treat.
    We really have no choice but to do this as our disk is filling up!

  33. Jim says

    Thanks for the details, Paul. We are nearing the point where we won’t be able to restore the entire database to our recovery partition. I’m slowly migrating users into new databases so I can remove the original large database. My question goes to recoving user mailboxes, though.

    If I need to restore a mailbox, and if Exchange is randomly provisioning the mailboxes to any of the allowed databases, how does one know which database needs to be restored to recover the data? Do we need to get more granular and regulary run scripts which somehow sorts the mailboxes into specific databases based upon attributes? Say DB1 is lastnames beginning with A-F, DB2 is G-M…etc?

Leave a Reply

Your email address will not be published. Required fields are marked *