Managing Database Switchovers for Exchange Server 2013 Database Availability Groups

Databases that are being replicated within an Exchange Server 2013 database availability group can generally be considered to be either active or passive at any given time.

There are also other states that the database may be in, such as seeding, or due to health problems, but for the purposes of this article we’re going to be looking only at healthy database copies.

The active database copy is the one that is mounted on a DAG member and is being used to serve clients. The passive database copies (up to 15 of them) are those that reside on other mailbox servers within the DAG. Changes are replicated from the active database copy to the passive database copies.

exchange-2013-dag-database-switchover-01

The active database copy can be “moved” between DAG members that host a copy of that database. Although some people consider this to be an actual “move” of the data from one server to the other, in actual fact what is occurring is the active database copy is dismounted, and one of the passive database copies is then mounted.

There are two ways that the active database copy can be moved to another DAG member:

  • Failover – this is an unplanned event, such as a failure of the server hosting the active copy
  • Switchover – this is a deliberate, administrator-driven event, such as during server maintenance

In this article we’ll look specifically at the database switchover.

Consider a two-member DAG with two databases. Each server hosts one active and one passive database copy.

exchange-2013-dag-database-switchover-02

To move Mailbox Database 1 to E15MB2 we can simply highlight it and then review the status of the database copies. Note that in this case the passive copy on E15MB2 is healthy, with no copy queue length issues and a healthy content index state as well.

Under those conditions we can proceed with the switchover by clicking the link to Activate the database copy.

exchange-2013-dag-database-switchover-03

You’ll be prompted to confirm the action, and then a progress bar let’s you know when the operation is complete.

exchange-2013-dag-database-switchover-04

Click Close when it is finished. In this case we can see that the database is now active on server E15MB2.

exchange-2013-dag-database-switchover-05

 

Comments

  1. Barry says

    Hi Paul,

    I have built a DAG group but only have the one mailbox database. Is it best practice to have 2 databases?

  2. G says

    Hi Paul

    Awesome articles, they are real help for people like me getting to grips with Exchange 2013 for the first time.

    I just have one question regarding mailbox databases. I read above that it is not necessarily best practice to have more than one mailbox database, but are there advantages to say have two mailbox databases replicated on a DAG, say you have two combined CAS and MBX servers (my set up). Is there a performance gain or redundancy gain of having ‘mailbox database 1’ active and mounted on server A with a passive copy on server B and another ‘mailbox database 2’ mount and active on server B with a passive copy on server A. Lets say both DB’s have all your user and other mailboxes spread across both databases.

    Any advice or opinions greatly appreciated

    Thanks

    • says

      Exchange 2013 has some store improvements that mean lower IOPS for passive database copies, so in theory yes there is less load on each server if you distribute your active DBs 50/50 like that.

      However you should always size your hardware to suit maximum performance requirements, ie when both DBs are active on one node, which will occur regularly due to maintenance, failures, automatic failovers, etc.

  3. G says

    Hi Paul

    Thanks for the swift response! My hardware is way over spec, we utilised a dell poweredge as it was just lying around doing nothing that is comprised of 4 server systems for the exchange environment, 2 combined cas and mbx servers both with 64 gb of ram, 2 6 core cpus each, 140 gb raided volume each for the os and one 2tb raided volume each for the mailbox databases and logs. Given the above do you think it would be worthwhile me creating the extra mb db as stated in my previous comment?

    Thanks

    • says

      Well Exchange 2013 can support quite large databases. If you’re satisfied that performance is not going to be an issue then there are other reasons to consider multiple databases, such as backup/restore times.

  4. says

    Paul
    I have two mailbox servers and eight databases that I try to spread between them, but after applying the latest rollup for Exchange 2013 I notice that the databases all seem to migrate to one server or the other. Is this normal?

  5. Tony Perez says

    Hi Paul,
    First I want to thank you for all your posts, it help us to get better in our jobs. Quick question can I add a secondary FSW to the DAG?, and how I can accomplish that?. I want to set two FSW’s one physical and one Virtual.

    Thanks,

  6. Carlo says

    Hi Paul,

    Greetings!

    This is very helpful. i just have some question in mind. I have my Exchange 2013 DAG in place.
    When is the best time to replicate my mailbox database to other dag member?
    I’m planning to replicate after office hours. And also, what is the impact when i replicate to other mailbox server?
    Does my exchange users may encounter downtime or a delay of the send\receive email?

    Please share your thoughts on this.

    • says

      When you add a database copy on another DAG member the replication begins and runs continuously from that point onwards, unless it is suspended for reasons of maintenance or due to a fault.

      Replication does not cause any downtime for the end user or delays on email, unless the server is overloaded which is more of a general problem than anything.

  7. Carlo says

    Thanks Paul,

    I have some other issue encountered, i am experiencing an intermittent send/receive emails for all migrated users.

    Workaround:
    1. Restart this services Microsoft Exchange Information Store.
    2. On OWA – i just restart my IIS services.

    Please share on thoughts

    Godspeed!

  8. Levin says

    I have a question on exchange 2013 dag failover,
    1. I understand that in exchange 2013 I can give same URL name for eg.mail.contoso.com in both datacenter for all services, and because of this I can assign two IP address to mail.contoso.com that is IP at primary datacenter and IP at secondary datacenter. But how can I make sure that my oulook or any other clients continuously connected to primary datacenter IP? if I give both IP in DNS how oulook will choose primary datacenter IP as primary connection pont? I hope it will randomly select and may not be suitable for my situation.

    2. Exchange 2010 datacenter switchover process is mentioned in the TechNet article,
    http://technet.microsoft.com/en-us/library/dd351049(v=exchg.141).aspx

    how can I manually activate datacenter in case of exchange 2013? I don’t have a third location for witness server for automatic failover. I hope it will be same as exchange 2010.

    • says

      You would need to do geo-DNS or geo-loadbalancing, otherwise yes the clients will randonly pick one CAS IP and connect to it regardless of where the active database copy is.

      However, that isn’t necessarily a bad thing if the WAN between datacenters is fast enough. And it should be reasonably fast if you’re stretching a DAG across it.

  9. Jerry Beasley says

    Great info! Just started a migration from Domino to Exchange and I noticed that over the weekend the mailbox databases failed over to another server. Is there a log somewhere that will tell me when\why this happened? Thanks

  10. Brian says

    Great article!
    Is the procedure to fail over to a datacenter basically the same as in 2010 where you need to stop nodes and activate the alternate FSW?

    Thank you
    Brian

  11. Pam says

    As always, very helpful & well done. I have also read your related article “How to Install Updates on Exchange Server 2010 Database Availability Groups” and am wondering if there are any important changes for Exchange 2013.

  12. VP says

    I have a 2010 Exchange server which currently contains my mailboxes and in a DAG I have 2 2013 servers. All 3 servers currently live in the same data center A. Once I move all the mailboxes over to 2013 I plan on sending one of the 2013 servers to a remote data center B. Are there special preparations I should take to do that? I expect to overnight the server so it would be up in at least 48 or worst case 72 hours.

    It will have about 300 mailboxes, 800 Gig among 5 dbs. My witness server is already in a different data center C.

  13. Caveman says

    Hi Paul,

    I have a mailbox database that has a copy at another site. Replication takes pretty long and there is almost always a copy queue. What happens if I activate the copy which still has a copy queue length of non-zero value? Furthermore, what will happen if I activate the original database? Will the changes made to the copy (which still has copy queue) be reflected to the original database?

      • Chris says

        I’ve been speaking with a few Exchange consultants and they’ve been informing me that AUTO failover is not possible between two sites. Mainly because of client access and DNS records that would need to be changed. Seems like there would be a lot of user intervention to be able to failover to a DR site. Do you agree?

        • says

          Depends which version of Exchange and how they’ve deployed it. For Exchange 2013 (which this article is about) it is entirely possible to have automatic database failovers to other sites without manual intervention required.

  14. Andy says

    Hello Paul,

    i just found this great Article Series. However there is one question that makes me puzzeled.
    Is there any benefit of a DAG Deployment if Exchange is already hosted on a Hyper-V failover Cluster with Shared Storage ? The only benefit i can see is to have a somehow “desaster” copy available on any external storage.

    • says

      A single VM is not a highly available solution. Virtualization can protect you from some failure scenarios, but not from many others. Even something as simple as performing routine patching of the server is impossible to do without downtime if you’ve only deployed a single server.

      • I agreed with Paul says

        In my current environment even though we have storage failover we also have a DAG, I ran to an scenario, where one of the exchange server, didn’t reboot after a patch update. But we didn’t have any downtime because the as soon the witness server found that server was down, migrated the copy to the second Server.

        • says

          “as soon the witness server found that server was down, migrated the copy to the second Server”

          That isn’t really how it works, but I’m glad your DAG is doing its job.

    • says

      It depends on how smoothly the switchover goes, but a few seconds would be considered normal. End users shouldn’t notice anything, especially if they’re running Outlook in cached mode.

  15. Davis says

    Thank you Paul! Learned a lot from your posts.

    I have a Disaster Recovery situation and would like your advice. I understand that only one database can be active at any one time.

    However, are there workarounds to activate both databases at DR site and Production site for a period of time?
    Reason: When we conduct DR tests, we would only like a subset of users to access mail boxes in the DR site via the Internet link from our offices. The rest of users should access their mail boxes from their our office internal production connectivity so they will not be affected by Internet links latencies, etc.

    In normal circumstances, a DR test should simulate a real disaster where all users will access DR site mail boxes via the Internet Link be it slow or performance degradation because a real disaster has indeed happened.

    But, we wouldn’t want to alarm the users so we were thinking of only having a subset of users to test the DR mail boxes while rest works business as usual. We didn’t want all 1000 over users to access email over the Internet during a DR test.

    • says

      The switchover/failover is per-database, so you can choose to only switchover or failover specific databases.

      However, for any given database, only one copy of that database can be active at any given time. If you have two copies of the same database active at the same time you have effectively created a split brain condition which is a very bad thing.

      In your situation I would simply advise users that a DR test is being undertaken and that some minor performance degradation may occur as a result. In reality any cached-mode Outlook user should remain blissfully unaware of any latency issues that may occur.

  16. Dave says

    Paul,

    We have two DAG members, each with 4 CPUs and 40GB RAM. For over a month, we were running in co-existence with Exchange 2007, having all of our mailboxes distributed between four databases on 2013 while the public folders remained on 2007. Last weekend we completed the public folder migration and there have been challenges ever since. The four databases with the user mailboxes are active on one server with passives on the other while we have one database with all the public folders active on the other with passive copy on the server with the active user mailboxes. This “public folder mailbox” database is 145GB and our public folders consists of around 75k folders and 13 million emails. We seem to get the occasional event 164 for this public folder database which causes it to activate on the other server with all the active user mailbox databases. This hoses up all of our Outlook clients (cached mode but public folders are online mode). A few minutes later the event 164 will appear on that server and the public folder database switches back. A few minutes later everything is fine again. This causes almost 10 minutes of downtime for our users because Outlook is not responding. The storage is on an iSCSI SAN with dedicated RAID 5 disks for each email server. However the active and passive copies on one server all reside on one RAID 5 drive (3 spindles) and the active passive copies of the other server are on another set of RAID 5 disks (3 spindles). The tlogs of both servers are sharing a RAID 1 (2 spindles). I/Os, CPU, and memory don’t appear to be excessively high when event 164 appears. Any suggestions that we should try?

    Log Name: Application
    Source: ExchangeStoreDB
    Date: 5/26/2015 5:38:50 PM
    Event ID: 164
    Task Category: Database recovery
    Level: Error
    Keywords: Classic
    User: N/A
    Computer: xxxxx
    Description:
    At ‘5/26/2015 5:38:43 PM’ the Exchange store database ‘DB_PF’ copy on this server timed out on periodic status check. For more details about the failure, consult the Event log on the server for other storage and “ExchangeStoreDb” events. A successful failover restored service.

  17. Dave says

    On a side note, do most people still run an antivirus agent on their Exchange servers? I’m following the MS recommendations for exclusions, but you never know if that agent is behaving accordingly. We have desktop AV and a hosted spam/virus filtering solution for incoming/outgoing email so just wondering if it’s even worth it to have AV on the Exchange server.

  18. Rusty says

    Great article! I would recommend that you add a snippet about enabling DAC mode and what it does though. I had an issue recently where our primary data center (one mailbox server and witness) went down, and I had an extremely difficult time activating the DAG at our secondary data center (one mailbox server only) without DAC mode enabled. DAC mode makes switching to a secondary site during an outage much much simpler and also helps to prevent database divergence when recovering your primary site should a connection between the two not yet be available.

Leave a Reply

Your email address will not be published. Required fields are marked *