Recovering a single Domain Controller from a USN Rollback

Some time ago I wrote about my experience recovering a customer’s Active Directory from a USN Rollback condition that had been caused by some virtualisation work.  There has been some discussion in the comments in that post about what to do when you have a single domain controller that thinks it is in a USN Rollback condition (eg has disabled outbound replication and paused the NetLogon service).

Logic would suggest that once a DC knows it is the only DC in the Forest that it would shake off the USN Rollback blues and start humming away normally again.  Not the case unfortunately.

Rob P recently spent some time and effort with Microsoft support and came up with a solution that can be applied.

!!!Warning!!! !!!Warning!!! !!!Warning!!!

I’m not 100% sure why I’m warning you, but I’ll take Rob’s word on the matter.  Apparently this fix is quite dangerous and not for the faint of heart.  My heart is not the least bit faint, particularly when it comes to my VMWare test environment, so I didn’t mind testing this out.  At the very least you should make sure you have a backup of the server you can go back to if this doesn’t work.

To get a single domain controller out of USN Rollback:

  1. Open Regedit
  2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters
  3. Locate the key “Dsa Not Writable”=dword:00000004
  4. Delete the entire key
  5. Enable replication by running repadmin /options servername -DISABLE_OUTBOUND_REPL and repadmin /options servername -DISABLE_INBOUND_REPL
  6. Reboot

Once your domain controller has rebooted you should find that NetLogon is running again and repadmin /options no longer shows replication as being disabled.

I performed this test on a Windows Server 2003 R2 domain controller and I imagine it works fine on Small Business Server 2003 as well.

About Paul Cunningham

Paul is a Microsoft Exchange Server specialist for one of Australia's largest companies, and is the Publisher of ExchangeServerPro.com. He is also an MCP, MCSA, MCSE, MCTS, and an MCITP for Exchange Server 2007/2010. Connect with Paul on Twitter, LinkedIn and Google+.

Comments

  1. Qazi says:

    Hi,

    I recently had this issue on our site where we had two domain controllers, one on a physical machine and the other a virtual machine (running on ESX). We were in a situation where the AD would not allow anyone to logon because of replication and USN errors. I followed Microsofts solution and forced one DC down but was unable to get it to become a DC again as the USN rollback error starting causing issues (atleast users are able to login). So at the moment we have one DC that has the USN error and I am unable to create a second DC on the domain. I ran the repadmin /showutdvec command and it returned two lines. The first line shows the one DC and the USN number. The second line has a long name (seems like random alphanumeric characters) with a different USN number. Now I am not sure if the second line is the USN of the second DC that we killed. I am not an expert on USN so I am not sure if I should delete this or keep it as it is and try your solution. Any ideas?

    Also, thanks for posting these!! I have been looking for a solution for a week now!!

  2. Paul says:

    Hi Qazi, take a look at this http://www.exchangeserverpro.com/2007/06/02/event-id-2095-and-the-usn-rollback-adventure/ and make sure you’ve followed the procedure to remove the demoted domain controller from the directory with NTDSUtil etc. Once you’ve done that, if your sole remaining DC still thinks it is in USN rollback then I would proceed with deleting the registry key above, but only after taking a full backup of the server.

  3. Qazi says:

    Hi,

    Just to let everyone know that I tried the above solution and it did work for us in a live environment! I would however suggest that anyone attempting this should backup the server and AD and the registry before attempting anything. Good luck.

  4. Ryan says:

    Had this problem after restoring from snapshot an AD DC on VMWare.

    Did all steps as stated and it worked. At first there was some replication errors but they got sorted out automatically.

    Thanks!

  5. Dan says:

    Have similar issue based on a dc snapshot restore on VMWare. See the key, however this is a Server 2008 Standard …anyone know if same applies…

  6. Michel Bitton says:

    Thank you for this fix. I am dancing for joy since my DC running exchange 2003 server got messed up (twice actually) first by a hard drive failure and using a partition restore and a month later trying to migrate the physical machine to an ESX VMware server and I was greatly concerned by how to handle this.

    May the schwarz be with you!!!!!

    Michel.

  7. Happy GoLucky says:

    You are a life saver! Fixed my problems and I’m back in business.

  8. Tarek says:

    Thank you ,Fixed my problems

  9. Mirco says:

    Thank Thank Thank You
    This Workaround saved my entire weekend…
    My Family thanks.

  10. Justin says:

    So after performing this procedure, your DC replicates again. However, you now have a DC in a state where is potentially has many objects or attributes missing from it. (and they won’t be replicated back because other DCs think this one is up-to-date) Some bad advice here… The quarantine put into place was meant to prevent the DC from replicating again for a reason.

  11. Ken Rudd says:

    Worked like a charm on a 2008 DC. Note that, while this was the only DC in the domain, it was a child domain in a forest with a root domain and one other child.

    This was the result of a Virtualization re-home

    Woot, thanks again!

Leave a Comment

*