Cannot Activate Database Copy: Content Index Catalog Files in Failed State

When activating a database copy in an Exchange Server 2010 Database Availability Group it may fail with an error message that catalog index files are in a failed state.

exchange-2010-cannot-activate-database

——————————————————–
Microsoft Exchange Error
——————————————————–
Cannot activate database copy ‘Activate Database Copy…’.

Activate Database Copy…
Failed
Error:
An Active Manager operation failed. Error The database action failed. Error: An error occurred while trying to validate the specified database copy for possible activation. Error: Database copy ‘MB-HO-03′ on server ‘HO-EX2010-MB2.exchangeserverpro.net’ has content index catalog files in the following state: ‘Failed’.. [Database: MB-HO-03, Server: HO-EX2010-MB1.exchangeserverpro.net]

An Active Manager operation failed. Error An error occurred while trying to validate the specified database copy for possible activation. Error: Database copy ‘MB-HO-03′ on server ‘HO-EX2010-MB2.exchangeserverpro.net’ has content index catalog files in the following state: ‘Failed’..

When you view the copy status of the mailbox database the content index is in a failed state.

[PS] C:\>Get-MailboxDatabaseCopyStatus | fl name, contentindexstate

Name              : MB-HO-01\HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-04\HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-03\HO-EX2010-MB1
ContentIndexState : Failed

Name              : MB-HO-02\HO-EX2010-MB1
ContentIndexState : Healthy

There are two likely causes of this issue.

  1. The content index has failed due some error or fault occurring
  2. Indexing has been disabled for that database (a common configuration for databases hosting journal mailboxes)

Fixing Failed Content Indexes

To resolve the issue update the content index on the server on which it has failed.

[PS] C:\>Update-MailboxDatabaseCopy "MB-HO-03\HO-EX2010-MB1" -CatalogOnly

The content index should now be in a healthy state.

[PS] C:\>Get-MailboxDatabaseCopyStatus | fl name, contentindexstate

Name              : MB-HO-01\HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-04\HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-03\HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-02\HO-EX2010-MB1
ContentIndexState : Healthy

After the content index is healthy you can attempt to activate the database copy again.

Performing Database Switchovers When Indexing is Disabled

If indexing has been intentionally disabled you can still perform a database switchover using the Exchange Management Shell, by running Move-ActiveMailboxDatabase with the -SkipClientExperienceChecks switch.

[PS] C:\>Move-ActiveMailboxDatabase MB-HO-03 -ActivateOnServer HO-EX2010-MB2 -SkipClientExperienceChecks

Assuming no other issues exist with the database the move should complete successfully.

About Paul Cunningham

Paul is a Microsoft Exchange Server MVP and publisher of Exchange Server Pro. He also holds several Microsoft certifications including for Exchange Server 2007, 2010 and 2013. Connect with Paul on Twitter and Google+.

Comments

  1. Chris Lee says:

    It’s work~ Thanks for your help.

  2. Thanks Paul, I’ve seen this before but it’s nice when the error and solution match right up.

  3. Hi,

    Thanks for detailing that out; however my databases are stuck in a permanent ‘crawling’ state and cannot run the -catalogonly command.

    Any suggestions would be greatly appreciated.

  4. Hi paul, works like a charm!! thnx for the good solution. +1

    Regards,

    Leon

  5. This isn’t working for me. The command completes without errors, but the state of the database doesn’t go back to healthy, and the server keeps logging ExchangeStoreDB 123 events (which says to run the Update-MailboxDatabaseCopy command to reseed the catalog). This has affected two databases in the past two days. The first one was a new database, so I fixed it by deleting and recreating it, but that’s not a long-term solution. Any other ideas? I’ll keep searching. Thanks.

  6. I found two other articles concerning the same problem, with slightly different solutions, which sounded promising…

    http://msexchangeguru.com/2010/12/01/cannot-activate-database-copy-content-index-catalog-files-in-failed-state/

    http://blogs.msdn.com/b/pepeedu/archive/2010/12/14/exchange-2010-database-copy-on-server-has-content-index-catalog-files-in-the-following-state-failed.aspx

    Unfortunately, neither of these could resolve my problem. In the end, I had to just stop the Microsoft Exchange Search Indexer service on both nodes, then delete the Catalog folder, and restart the service. I just hope this doesn’t happen too often.

    Mike

  7. Hey Paul,

    Thanks for this info, it has worked both times I have had this issue. One question I have that I haven’t been able to find anywhere is what causes this to happen in the first place and how can I avoid it?

    • Not entirely sure, to be honest. At least I can’t remember off the top of my head right now.

      • Noone is very sure about why that happens. We’ve seen it on nearly empty databases more often than on filled ones (for God’s sake!), and see it happen when the GC is not available in a Failover Cluster.

        Greetings,

        Rob

  8. Hi Paul,

    This works!! Thanks a lot.. Your posts has been a big help to the exchange community..

  9. Get-MailboxDatabaseCopyStatus | where {$_.contentindexstate -eq ‘Failed’} | Update-MailboxDatabaseCopy -CatalogOnly

  10. Thanks Paul. Worked great for me.

  11. Elliston says:

    Worked for me after changing my database paths and the copy wouldnt become active. Thanks – really great to have helpful people in the commuinity.

  12. Hey,

    Did you have information on failed index state as error message ” The Time stamp of the Catalog is too old for database”

    Please us know,

    Thanks,
    Amit

  13. if there are more than 1 mailbox server in to Exchange environment then you need to run following commnad to know actually which copy of the databse having content indexing issue.

    get-mailboxserver |Get-MailboxDatabaseCopyStatus | fl name, contentindexstate

    Based on the result, it will give us that which databse on which server is having status fail of content indexing

  14. Hi,

    Could you have details on contentindex state failed on active node database server, As of we reseed only passive copy with active node. In case its active node database server how it possible to make its contentindex state failed to healthy

    does any one have answer ?

    Thanks,
    Amit

  15. in that case, you need to go to the Exchange script folder where you have Active copy of DB with contect index Failed . you may find it is in default location which is C:\Program Files\Microsoft\Exchange Server\V14\scripts

    after going in to Script directory through EMS, run below command

    .\ResetSearchIndex.ps1

    It will ask you to press enter for continue. so say YES.

    Once it done, intial state of Contectindexing will show as crawling. later it will turn in to Healthy state.

    Cheers!

  16. When ran reset index for mailbox databases, seems this error
    Stop-Service : Service ‘Microsoft Exchange Search Indexer (MSExchangeSearch)’ cannot be stopped due to the following error: Cannot open MSExchangeSearch service on computer ‘.’.
    At C:\Program Files\Microsoft\Exchange Server\V14\Scripts\ResetSearchIndex.ps1:105 char:13
    + stop-service <<<< MSExchangeSearch
    + CategoryInfo : CloseError: (System.ServiceProcess.ServiceController:ServiceController) [Stop-Service],
    ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStopService,Microsoft.PowerShell.Commands.StopServiceCommand

    I guess search service not respoding for reset,

    Any idea ?

    Thanks,
    Amit

  17. Thank you for that article – it helps me in a short way :-)

  18. It works! But before typing this commands I needed to restart my Exchange services. Thank you very much!!!

  19. Ian Salgado says:

    Hi,

    I have a Database which has a Catalog Index = FAILED

    The database IS NOT part of a DAG. It is a stand alone DB which is use for archiving emails (archiving function in Exchange 2010)

    The DB is currently in a DISMOUNT state because of the CATALOG INDEX FAILED & cannot be mount.

    I tried applying a command such as:

    Update-MailboxDatabaseCopy “Mailbox Database 01\EX1″ -CatalogOnly

    It does not work as the DB is not in a DAG.

    I have also try the following command:

    [PS] D:\Program Files\Microsoft\Exchange Server\V14\Scripts>.\Troubleshoot-CI.ps1 -Database “Mailbox Database 01\EX1″ –Action DetectAndResolve

    However, I did not have any luck with this one either.

    How can i correct this catalog index & hence be able to mount the DB ?

    Thank You in advance

    Ian Salgado

  20. Monty St. says:

    Thanks for that. Concise and very helpful.

  21. Keith Risinger says:

    Worked like a charm. Thank you Paul for this and other helpful information you provide.

  22. John Azevedo says:

    Worked like a charm!

    Thanks!

  23. Thank you!

  24. Jamsheed Kadooran says:

    HI Paul

    it’s worked, thanks for your gr8 support.

    BR!

  25. Sule Yaya says:

    Hi Paul,

    Thanks for always being a source of help & support for all of us.

    Sulex

  26. Paul taylor says:

    I have opened up a powershell command window as administrator, I have then typed get-mailboxserver |Get-MailboxDatabaseCopyStatus | fl name, contentindexstate and I get the following error

    get-mailboxserver : The term ‘get-mailboxserver’ is not recognized as the name of a cmdlet, function, script file, or
    operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
    again.
    At line:1 char:1
    + get-mailboxserver |Get-MailboxDatabaseCopyStatus | fl name, contentindexstate
    + ~~~~~~~~~~~~~~~~~
    + CategoryInfo : ObjectNotFound: (get-mailboxserver:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

    I am just learning Powershell so why wont it work?

  27. Ciprian Bacanu says:

    Hi Paul,
    thanks for this post. I wonder what’s the solution when there’s only one server i.e. only one database.
    I noticed index state was failed, I reseed search index but perf mon crawler is telling me there is one mailbox to be seeded. This lasts from the 1st of November. Also, I have error id 4087:
    Failed to move active database ‘Mailbox Database DB1′ from server ‘ExSrv1′. Move comment: Database failover initiated by Responder ‘SearchLocalCopyStatusEscalate’ target resource ‘Mailbox Database DB1′

    Error: An Active Manager operation failed. Error: The database action failed. Error: This operation is not supported on standalone mailbox servers.

    Could you tell me what’s going on? Will crawling process stop? If not, how could I find the mailbox?

    Best regards,
    Ciprian

    • Ciprian Bacanu says:

      Hi Paul, Happy New Year! Finally solved issue, please delete this comment.

      Best regards,

      Ciprian

      • Hi Ciprian – can you tell me what the fix was?

        Thanks in advance.

        • Ciprian Bacanu says:

          Tarek, hello,
          Unfortunately I didn’t find out what was the mailbox that caused the trouble so I started out and created a new database where I moved some of mailboxes (I took reverse the size they had). No joy after rendex. I created a new one and moved another ones. At the third one I noticed that after reindex databases were healthy. So, I moved back users mailboxes and delete the temporary databases.
          I don’t have a lot of users and a had also a lot of space available – take care that moving a mailbox to another database does not free the space.
          I managed to find out the maibox looking the migration logs.

          Ciprian

  28. Hi Paul
    It’s one of the best Website which I used forever.
    Thank you for your short and good guides to resolve problems

    Best Regards

  29. Well, you can use Long Path Tool, it works good for such problems……….

  30. Thank you !

    Have a wonderful weekend !

    Greetings from Lisboa !

  31. Hi Paul

    I have content index state on one DB as crawling for both the active and the passive noe. Any help would be appreciated

  32. Upendra says:

    Hi Paul,

    Really it is very useful.

    I have one question , Can you explain how does replication works in DAG, if suppose if any log file got corrupted in passive copy in DAG , how identify and how will resolve it.

    Highly appreciated your quick response.

    Thanks,
    Upendra.

  33. Hi Paul,

    Have you seen any issues with Exchange 2010 SP3 in EMC >> Server Config >> Mailbox. Under Database Copies > Copy Status not updating correctly with a refresh.

    I end up logging into another Exchange server just to see the current correct copy status.

    Thanks,
    Jeff

  34. WORKED LIKE A CHARM…BIG THANKS.

  35. Excellent. reseeding the Catalog only worked like a charm. Thank you!!

  36. Incredible to see how MS does nothing to address this little problem – the work on the overnext problem … erm… Exchange version seems to be more important than to auto-fix problems that are obvious; be it in the ource or by auto-healing.

    Sad to see that a company which claims to be powerful is so weak – but to be honest, I did not expect more…

    Rob

Leave a Comment

*

We are an Authorized DigiCert™ SSL Partner.