Microsoft has released Service Pack 3 for Exchange Server 2010. This is a significant release that delivers some key functionality to customers such as support for Windows Server 2012, support for co-existence with Exchange Server 2013 CU1, and general bug fixes and security updates.
If you are planning to upgrade your Exchange 2010 servers to SP3 you should be aware that there is an Active Directory schema update involved. If that is a concern for your environment, but you still want the bug fixes and security updates, you might consider sticking with Service Pack 2 and applying Update Rollup 6 instead.
At the time of this writing there are some points in the various release notes that aren’t correct or fully updated yet that Microsoft are still working on or that are worth some clarification:
- Exchange 2010 SP3 is listed as including all security bug fixes up to SP2 UR5-v2. It actually includes all security and bug fixes up to SP2 UR6.
- The SP3 release notes state you can only install on Windows Server 2008 SP2 or 2008 R2. You can actually install on Windows Server 2012, although exact pre-requisite guidance may not be available yet.
- The support for Windows Server 2012 includes both the installation of SP3 on Server 2012, and the interoperability of Exchange 2010 SP3 with Server 2012 domain controllers.
- The support for Windows Server 2012, which ships with PowerShell 3, does not mean that Exchange 2010 SP3 also supports upgrading to PowerShell 3 on other operating systems.
- The co-existence support for Exchange 2013 does not apply to Exchange 2013 RTM, but rather Exchange 2013 CU1 (cumulative update 1) due for release in Q1 of 2013 (within about 6 weeks from the time of this writing)
Preparing to Upgrade to Exchange 2010 SP3
You can download Exchange 2010 Service Pack 3 here and extract the files ready to be installed on your servers.
Upgrade your servers in the following order:
- Client Access servers (beginning with the internet-facing site)
- Hub Transport and Edge Transport servers
- Mailbox servers
- Unified Messaging servers
You should also plan to update any management tools installations you have on admin workstations or servers, and also check your third party applications that integrate with Exchange in case they also need updated management tools.
I’m going to walk through the upgrade process in some more detail next, and also provide some general guidance afterwards about the Service Pack 3 installation steps as well as what to expect in terms of timing and service interruptions.
Applying the Schema Update
If you have an AD forest topology with multiple domains, or process restrictions that require schema updates to be managed a certain way, you can apply the Exchange 2010 SP3 schema update on a 64-bit domain controller that is in the same AD site as the Schema Master, using an account with Schema Admins and Enterprise Admins rights.
C:\Admin\Ex2010SP3>setup.exe /PrepareAD
Otherwise the schema update will be applied when you upgrade the first Exchange server.
Updating Client Access Servers to Exchange 2010 SP3
Client Access servers are the first server role to update, and you should begin with the internet-facing site if you have multiple sites in your organization.
For Client Access servers that are in a CAS Array you should remove some of the servers (eg half of them) from the load balancer configuration, upgrade them, re-add them to the load balancer, then repeat the process with the remaining Client Access servers in that load balanced array.
For an example of how to do this with Windows NLB see the following article:
For other load balancers refer to your vendor documentation for how to take servers out of the load balanced array for maintenance and updates.
Updating Mailbox Servers to Exchange 2010 SP3
I admit I was concerned when I read the release notes for Exchange 2010 SP3 that state:
The database schema has been updated in Exchange 2010 SP3. As a result, when Mailbox servers are upgraded to Exchange 2010 SP3, the databases are upgraded to the Exchange 2010 SP3 version of the database schema…
…During the upgrade, the database is dismounted, and all mailboxes in that database are taken offline.
This seemed to be a major issue to me until I performed the upgrade in my test lab. The statement above is correct for standalone mailbox servers, which is expected.
However for an Exchange 2010 Database Availability Group the upgrade process can be performed with no downtime following the normal process of moving active databases off DAG members while they are being updated.
You can use the standard process as demonstrated here:
However, be aware that once a database has been made active on an Exchange 2010 SP3 member of the DAG, it can’t be made active on a pre-SP3 DAG member again. This means that you will need to roll through your entire DAG upgrading to Service Pack 3 to retain the full availability resilience your DAG is designed to provide.
Upgrading Other Server Roles to Exchange 2010 SP3
For Hub Transport, Edge Transport, and Unified Messaging servers there are no special steps required other than to manage your upgrades in a way that aligns with whatever high availability you have in place or those server roles. For example if you have two Hub Transport servers in a site, upgrade them one at a time.
Exchange 2010 Service Pack 3 Step by Step
The upgrades steps are very straightforward and easy to follow. Extract the SP3 files to a folder and run Setup.exe. When the splash dialog appears click Install Microsoft Exchange Server upgrade.
You’ll need to click through the usual introduction and license agreement.
Next the Readiness Checks will be performed. Any errors will prevent you from proceeding. Warnings will not prevent you from proceeding, but you should pay attention to them anyway as they are often important.
Remember, if you’re upgrading CAS Array or DAG members refer to the guidance above.
Click Upgrade when you’re ready to proceed.
The actual installation time will vary depending on the server roles installed, and whether you’re upgrading from a very recent or much older Service Pack level of Exchange.
When the installation has all completed successfully click the Finish button.
Each of my test lab servers took between 20 and 30 minutes to upgrade, but your performance will no doubt vary.
About Paul Cunningham
Its probably also worth mentioning for those who also have the UM role installed alongside the CAS & MBX role, that language packs, other than the default US one, will need to be uninstalled prior to the installation of SP3, as has been the case with previous Service Packs.
Great article once again. Keep the tweets going!
Hi
Can you check Exchange version after installation SP3 ? is it the same 14.3.123.4
@Remigiusz that is the correct version. 14 is the version of exchange. .3 the service pack. and 123.4 the build number.
Thanks Paul for this post. Wil be upgrading all cas and dag servers coming friday so this post was good timing.
Do you recommend waiting for official statements of support from your backup/antivirus/filtering vendors for a service pack or is it fairly safe to treat it like a normal rollup and just do your own tests?
How much do you like your job?
I tend to look for clear statements of support from the vendors of any critical integrated products.
Ok, just haven’t had to do an exchange sp for years so wasn’t sure how they effect compatibility. Will probably take months to get official support from backup exec so better plan to install UR6 for now.
Hi Paul excellent as always
just one little thing you probably need to mention(especially where hlb are involved)
a service pack will “wipe” custom settings you maybe have(static ports and such)
so be sure to backup any custom stuff you have set…
Good catch, I’ll add a note.
Thanks Paul.
May be setup.com /PrepareAD ?
I had a number of issues getting this installed, only some of which were scary. Coming from a 2010 RTM I had a bunch of hotfixes that were needed, but that was straightforward enough.
However, also hit on a number of issues that also hit SP2 upgraders earlier on; stuff like admin tools role is not current, open files error (that was the backup software service, had to disable that one) and another failure with the mailbox role upgrade. The solution I could find (thank heavens there is Google) was to do registry changes to bring stuff like the admin role version number into sync with the rest before rerunning the upgrade.
Probably due to remnants of an abortive SP1 install once upon a long ago. But with some regedit elbow grease I got SP3 installed, but a painless process it wasn’t.
I just installed this on a server: took over 6 hours. I think the issue MAY have been that the DC is Server 2012, but the Exchange server was a SP2 install running in Hyper -V. It took over two hours per some sections. There was not an individual hang, just a cluster of hanging. Working great now, though.
I am preparing to inplement a transition to Exchange 2010 SP2 with UR5v2 slipstreamed (currently have Ex2003SP2). Would you suggest I stay with my plan to install using EX2010SP2 executable or use Exchange 2010 SP3 executable? Any advice is appreciated. Thanks.
Firstly, I don’t recommend you include the UR when you install Exchange 2010 SP2. I recommend applying it after 2010 SP2 has been installed. The reason is that I have almost always had installs fail when I try to include a UR using that method.
Secondly, SP3 is very new at this time and not “proven” yet. Frankly, given Microsoft’s recent history of quality issues with updates I would recommend you stick with SP2 and UR5v2 for now *unless* something in UR6 or SP3 explicitly fixes an urgent bug or problem you’re experiencing (which I guess is unlikely since you’re still on 2003).
Thank-you for this expert advice. I will stick with my original plan, with the one change of NOT slipstreaming the UR. I will apply it after the install with elevated rights. And, again thanks for your advice.
Is it possible somehow to reinstall SP3 on 2010 ?
I have following situation..
I have installed SP3 on Exchange server 2010 (on machine A) and after it ono other machine successfuly installed exchange 2013. Domain controller is on the machine B (same as Exchange 2013)
For some reason I have restored snapshot for machine B .. and now when I run Exchange 2013 install again it says that Exchange on machine A is not SP3 (but it is) .. somehow it seems like domain controller thinks its still without SP3…
Is it possible somehow to tell new Exchange2013 that Echange2010 has SP3 ?
Thanks,
Marko
Marko, you’ve done two unsupported things:
1) You’ve installed Exchange 2013 RTM into an Exchange 2010 organization. Co-existence is not supported yet, until Exchange 2013 CU1 is released.
2) You’ve used a snapshot to rollback an Exchange server. This is quite simply unsupported and as you’ve discovered causes technical problems.
You’ve also installed Exchange on a DC, which is not unsupported but is certainly not best practice, and I personally recommend against ever doing it because it only ever causes me problems.
To answer your question, can you reinstall Exchange 2010 SP3? Yes, you can try. If it doesn’t work you can try and get assistance for whatever specific error it blocks you with.
Thanks Paul for such fast respone..
I know I have a bit strange neetwork configuration and I plan to migrate DC to a new machine one day. but what bothers me is that before I recovered snapshot for machine B it all worked well..
First I upgraded 2010 to SP3… and after that installed 2013 to an new machine (same one that DC is on)
and everything worked..
after restoring snapshot on machine B (where 2013 and DC were) .. and trying to install 2013 again.. it says that 2010 doesnt have SP3.. but it does..
When I try to run Setup.com from SP3 agan it doesnt offer any kind of reinstall but just a options to remove some server roles..
What really interests me is on which criteria 2013 check if 2010 has a SP3 or not… is it written somewhere in DC ? .. if so.. is it possible to manually edit DC so that it reflects a real state.. or some kind of command on 2010 server that will recover DC entries..
Thanks,
Marko
So not only did you use a snapshot to roll back an Exchange server, it was also the only DC in your environment? Sounds like you’ve also rolled back your AD to the a state prior to when the 2010 server was upgraded to SP3. Does that sound likely?
Would you suggest installing SP3 just because it’s there or not installing it unless it addresses a specific issue? A little background is we’re going to be migrating to new Exchange 2010 hardware soon. For a time we will have 2 Exchange 2010 systems in existence and I know they need to be the same patch level. I think going right from Exchange 2010 RTM to SP3 on the new box is going to be a easier than RTM to SP2 to RU6 (which is the patch level the existing box is at) however I don’t know of any specific issues we are having that is addressed in SP3. Suggestions?
I think I may have answered my own question with further research. It seems that if we want to install Exchange 2010 on Server 2012, I have to upgrade to SP3. Would you agree?
There’s a few points to be made here:
- staying up to date with service packs is generally a good thing. There’s a balance though between applying an update “because it’s there” and being so overly cautious that you lag way behind on updates.
- SP3 is new and not “proven” yet though I haven’t personally encountered problems with it. SP2 UR5-v2 is probably the latest “proven” version.
- Generally speaking servers in a site should be at the same level (obviously they can’t always be, eg while you’re working on upgrading them they will be out of step for a period of time) and the internet-facing one(s) are supposed to be the highest version.
- yes, Exchange 2010 SP3 is the supported version for running on Windows 2012
So, if you’re wanting to put the new server on WS2012 then I would recommend upgrading the existing server to SP3 first, with the caveat that it is still fairly new and not “proven”.
Exchange 2010 SP3 schema update failed
Organization Preparation FAILED
The following error was generated when “$error.Clear();
initialize-ExchangeUniversalGroups -DomainController $RoleDomainControll
er -ActiveDirectorySplitPermissions $RoleActiveDirectorySplitPermissions
” was run: “The well-known object entry with the GUID “BBBBBBBBEEEEEEEEEEE”, which is on the “CN=Microsoft Exchange,CN=Services,CN=Configuratio
n,DC=XXXXX,DC=YYYYY” container object’s otherWellKnownObjects attribute, refers to
a group “CN=Organization Management,OU=Microsoft Exchange Security Groups,DC=XXXX,DC=YYYY” of the wrong group type. Either delete the well-known object entry, or
promote the target object to “Universal, SecurityEnabled”.”.
That error suggests that your “Organization Management” is not a Universal group. Have you checked that?
Thanks!!! will promote “Organization Management” to universal group.
Hmmmm…strange issue… I’ve just tried installing Exchange 2010 SP3 and I get the exact same screens as your demo install in this article – right up to the part where its doing the upgrade and showing progress…..it goes all fine until it gets to the Copy Exchange Files and fails…..cannot copy exchangeserver.msi … and an error code of 2 ….and then craps out…
The file is there in the sp3 folder…..
not sure whats going on….any ideas?
I’ve just noticed that ALL of the Exchange services are now marked ‘Disabled’ and it looks like I’ll have to reset them back to auto before I can re-run the patch again as it will not re-run now.
Actually….it would appear that my Exchange 2010 has been uninstalled somehow?!?! My DB files are all still there but Exchange itself is now gone from the server WTF?!?! All the services are still listed as I mentioned before but you cannot start any Exchange services as all the files are gone!
WTF?!?!
I have a ShadowProtect backup which I can put back on to recover it – but I am at a loss as to how the whole bloody install can be gone?
How extremely odd! All the files are there…but its been uninstalled from the system. Just checked the Programs and Features in Control Panel and its gone. All icons on the desktop/start bar/all programs are all gone….
Hmmmmm……not sure whether to attempt o ‘fix’ it or just recover the server…will loose a half days worth of email though….but its a Saturday so it shouldn’t be much…how odd…
If you look at the progress screenshot in the article above you will notice a step is removal of Exchange files. Basically service packs are a “build to build upgrade” meaning a full install of a new build. Failing at the step where yours did can leave the server in a non-functioning state.
You should be able to to a server recovery installation:
http://technet.microsoft.com/en-us/library/dd876880(v=exchg.141).aspx
Cheers Paul – I ended up just recovering the server from the backup I did prior – than heavens for ShadowProtect hey
Now I’m a tad gunshy to try rolling it out again…..might leave it a few weeks I think
You may ran into this. I ran into this problem with SP1 – SP2.
http://support.microsoft.com/kb/2810617
Hi All
Im installing SP3 on our 2 exchange servers this morning in a DAG enviroment, the first finshed in less than a hour and the other has been stuck on Hub Transport role for over an hour.
The server that is stuck is running Forefront for Exchange 2010, not sure if that makes any difference, its also the server that our databases are mounted on.
Any ideas, why its running so slow on hub transport?
Hub Transport is the role that always seems to take the longest. But why are you upgrading a server in a DAG while it has active databases on it? You should be able to move the active databases to a different DAG member while you’re upgrading that server.
Paul,
We have 2 exchange SP1 servers running DAG in different locations, but logically in same domain site. Each server has all three CAS, Transport and mailbox roles.
My questions are 1/ should we install SP3 on two servers at same time or one by one?
2/ Currently we are in SP1, can we install SP3 directly or we have to go SP2 first?
Thanks,
Hello,
We have 2 CAS servers, 2 Mailbox Servers in DAG mode, running Exchange 2010 SP2 Rollup 6.
I’m unable to install SP3 at all on any of these servers.
I have tried to run “setup /prepareAD” on Domain Controller as well and it fails with the same Error:
Some controls aren’t valid.
– Exchange Server cannot be installed in the directory C:\Program Files\Microsoft\Exchange Server\V14, which contains profile files of the current user.
Any suggestions/ideas?
Tom
Hi Paul, great article as always. I’m in the process of upgrading a lab environment and the install is currently well over an hour. It’s spent an 1h20m on “Languages” (after Restoring Services and before Hub Transport). In your screenshot, Languages took about 2m30s.
Would you know what might be wrong?
Thanks
Depends on the power of your lab servers. Mine are relatively new and have lots of power to spare at the moment. But in other years I’ve run my labs on fairly low powered hardware and everything just takes longer.
Well, after waiting for 72 hours, the “Languages” part of the upgrade is still waiting. I think that’s very much broken. In fact I know it is. I rebooted the server and EMC wouldn’t open. I had to revert to a backup of the web.config file. OWA was shot and recreation of the VDs didn’t work. Mail was flowing, but I had no management. I had to build a second Exchange Server into the environment (which I patched to SP3 immediately – which worked) and then moved all the roles, certificates, mailboxes and PFs over. Now however, I’ve got an Exchange box that I can’t remove and if I try and insert the original Exchange 2010 SP1 media, it keeps trying to carry on with the SP3 upgrade. Is there I way I can delete the temp files that have been copied for SP3 and to remove any traces of the proposed upgrade?
Have you tried doing a recovery install of the failed server?
Hi Paul,
Firstly, thank you for your continued help with such issues, it is appreciated. I’ve resolved the issue today by doing the following:
Removed the SetupSave from the registry under Software/Microsoft Exchage
Removed the SetupMSI folder from the Exchange install location (DriveLetter:\Program Files\Microsoft\Exchange Server\v14\
I ran the upgrade again and it installed straightaway, total time, 40 mins
http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/0975bb5e-f270-4e39-abf3-70809cd98161
While upgrading exchange server from 2010 sp2 to 2010 sp3, facing schema related error, tried /prepareAD as well, but it didn’t work.
Hi,
I have tried to upgrade exchange server 2010 sp2 to sp3m, but everytime it is failing as it fails to restart the services like “Exchange Service Host service” and Service ‘tmlisten’ failed to stop due to error:’Cannot stop tmlisten service on computer…
please provide your suggestions…
Stop your Antivirus software
Hi
I have tried doing this upgrade but its gone very wrong!
It was all going well until the setup tried to restart services. It fails there saying it cant start the Exchange Service Host service. Needless to say I have tried a number of different things to get this service started but i cant!
Any clues? Cheers
If you have custom OWA HTTP to HTTPS redirection configured in IIS, SP3 will reset these settings to default.
I can verify this behavior. More so with us we had redirection set at the root of the Default Web Site, then turned off on all of the virtual directories. The upgrade turned the redirection back on all of the virtual directories, essentially breaking all CAS to our array. Led for a panicked few minutes since we didn’t have the redirection on our lab environments, only our production environment. Be careful with this!
Did the upgrade this weekend with no problems. Thanks for this post!
We updated our Exchange 2010 environment (2 CAS/HUB and 2 MBX in DAG) last week to SP3 and it took awhile but the update went very smooth. BUT, I would hold off on installing RU6 for SP2 or SP3 as we ran into quite a known bug that is just now surfacing. I was on the phone with Microsoft last week for hours as they gathered logs. Microsoft is aware of an issue where random users cannot delete certain emails unless they do a hard-delete. They said it is due to the a fix in RU6 but also happens in SP3 since it includes RU6.
http://thoughtsofanidlemind.wordpress.com/2013/02/28/exchange-2010-sp2-ru/
If I had known what I know today, I probably wouldn’t of install SP3 and waited. We decided to install it for the fix that helps the battery drain on EAS devices due to a loop in communication.
I have the following;2008R2 SP1 (Enterprise VM) with Exchange 2010 14.00.0722.000 and an old sbs 2003 sp1 with Exchange 6.5.7638.1. I am trying to retire the SBS 2003 that has the bulk of mailboxes and Public Folders. I was thinking about going from Exchange 2010 (no SP) directly to SP3 and then moving MB’s and PF’s. I could not find any incompatibilities but cannot brick their SBS Exchange. Will the upgrade, which involves an AD Schema modification, cause any issues with the SBS Exchange? Or can anyone think of any other issues?
The only AD schema change is ms-Exch-Calendar-Logging-Quota and the references that you are on AD Schema Exchange version 14733.
I am doing a SBS 2003 to SBS 2011 migration myself and hoping the MS fixes the deletion issue otherwise I plan on going straight to SP3. Did it in a VM test environment and didn’t have any issues.
This is good guide. please keep it up.
I used the guide to update my exchange servers cas/hub and edge successfully. However, on mailbox role, am stucked at language files task. i keep getting the following error:
Summary: 6 item(s). 0 succeeded, 1 failed.
Elapsed time: 00:21:44
Language Files
Failed
Error:
Installing product E:\Tools\EXSP3\ko\ServerLanguagePack.msi failed. Another installation is already in progress. Complete that installation before proceeding with this install. Error code is 1618.
Another installation is already in progress. Complete that installation before proceeding with this install
Click here for help… http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.3.123.3&e=ms.exch.err.Ex96BCEE&l=0&cl=cp
Warning:
An unexpected error has occurred and a Watson dump is being generated: Installing product E:\Tools\EXSP3\ko\ServerLanguagePack.msi failed. Another installation is already in progress. Complete that installation before proceeding with this install. Error code is 1618. It was running the command ‘Install-MsiPackage -PackagePath ‘E:\Tools\EXSP3\ko\ServerLanguagePack.msi’ -LogFile ‘C:\ExchangeSetupLogs\Install.ko.Server.20130323-135815.msilog’ -Features ‘AdminTools’,'Mailbox’,'ClientAccess’,'Gateway’,'Bridgehead’,'UnifiedMessaging’,'ServerLanguagePack’ -PropertyValues ‘LOGVERBOSE=1 TARGETDIR=”E:\Exchange Files””.
Elapsed Time: 00:21:44.
There are no installations going on all the other exchange servers and I have restarted the mailbox server several times but getting the same error.
Please help out!!!
KoKo,
Did you get this figured out? I have the exact same error with Exch 2010 SP3 install.
I’m confused. You say run /PrepareAD under Applying the Schema Update section, what about /PrepareSchema?
Shouldn’t we do it in this order?
Setup /PrepareSchema
Setup /PrepareDomain
/PrepareSchema – adds the attributes to the schema
/PrepareAD – does the above if not already done, then does a bunch of stuff like adding AD groups, security permissions, etc etc
/PrepareDomain – only needed if there are additional domains to update.
So for a single-domain AD Forest you only need to use /PrepareAD unless you have a specific need to separate out the tasks (eg different teams doing them).
A client of mine is looking at doing an SP3 upgrade in a multi-server environment. Would it be possible to stagger the installs where the CAS server would be on SP3 for a couple days before the 2 mailbox servers are upgraded, and then do the Edge Transport a couple days after that? I assume the environments would function, considering upgrading your CAS servers won’t “bring exchange down” until everything is SP3, but I was seeing if there were any significant problems that would, or any functionality that would clearly limited.
Thanks!
Yes you can roll it out over a period of days. Start with the internet-facing site/servers first.
I have :
Exchange Server 2010 Standard
Version : Version 14.1 (Build 218.15)
and i want to install SP3 , What should I care during installation ???
many thanks
Leo
I have the same version as you and I am also curious seems like several people have had issues which worries me.
I just went through that (no SP to SP3). I did backup EVERYTHING (SYS STATE, EXCHANGE hub/cas/mb & Edge) to be safe, but the wizard did everything. It checks your environment and will make suggestions if there are deficiencies. Follow the suggestions for role order if you have a complex environment. I sweated a lot but it came through minimal problems. I did have issues with back pressure but I am not sure if that was an SP3 thing or the fact that SP3 left 1.5GB in C:\temp on the edge server. Deleting the temp folder to get more disk space remedied the back pressure issue.
During my upgrade the checks kept failing noting a bunch of cscript processes having files open. When I checked for the processes, the processes were not listed. I found the answer here
http://www.jaapwesselius.com/2013/03/08/upgrade-to-exchange-2010-sp3-fails-with-cscript-error/
Stopping the System Center Managment Service allowed the upgrade to proceed.
I have taken over a very old installation of Exchange 2010 with no service packs at all (!) and I’m thinking about installing SP3 on it. It is the only server in the organization, with all roles installed.
Are there any recommendations how to do this (like installing SP1 first, then SP2 then SP3) or I can just install SP3 straight away? Taking a full backup first, of course.
Thank you!
Each SP is cumulative, so you can just go straight to the latest one.
You should expect a potentially lengthy upgrade though as the DB schema has changed since the RTM version.
Thanks, Paul! That’s reassuring! I’m slightly worried because the company has about 180GB of e-mails and if the installation fails I may have to go for plan B. Currently this means re-storing from backup, but I hope to come up with something better.
Just looked again at all comments above and found that others (like Robert Martin) went through this already, so I take that it’s possible to go from no SP to SP3.
Obviously backup, backup and backup again!
I’ll plan for this for next maintenance window…
Hi Paul, thanks for all the great info you provide!
How long would you give SP3 to mature before applying to production servers?
Well it isn’t a wine that gets better as it ages.
I recommend you simply do your research in the TechNet forums and on the Exchange team blog (look at the comments on the post that announced SP3) for any common issues people are seeing. Investigate whether any of your third party products (backups, AV, Blackberry etc) are incompatible. Apply to a test environment if possible. Deploy to prod with a good roll back plan ready just in case.
Hi,
Are there any issues with upgrading in Environments utilizing Lync 2010 especially on the schema update end, as well as Microsoft Dynamics CRM 2011 Email Router.
For Lync, I believe you may need to redo the Exchange parts of the integration, as the SP3 install will wipe out customizations.
Schema-wise, no issues that I’m aware of.
CRM, no idea. You will need to check to see if the Dynamics team has published any notes or warnings about compatibility issues.
Hi Paul S and Paul C,
Slager, have you upgraded to Exchange 2010 SP3 yet?
If so, did you experience any issues with either Exchange or Lync?
We are running Exchange 2010 SP2 and Lync 2010 in our environment.
We only have one Exchange server and it is running the following roles:
Mailbox, ClientAccess, UnifiedMessaging, HubTransport
I am a bit worried about the Lync part. What kind of customization’s will it wipe when we upgrade to SP3?
Hi,
Thanks for all the comments here. I have found it all very interesting.
Not sure if anyone can help me with an odd problem I have with my installation?
I have 2 multi role Exchange 2010 servers in a DAG. I updated these to SP3 without issue. I then installed Exchange 2013 which also went through without issue. All seems fine, imported certificates, can send and receive mail from Exchange 2013.
The strange problem I have is when trying to add the new Exchange 2013 server to my existing Exchange 2010 DAG. From the ECP in DAG’s, it shows the two Exchange 2010 servers. I then try to add the Exchange 2013 server but it says; ERROR. Server ‘SVR08-EX01′ must be running Exchange Server 2010. ???
If you go to servers my versions are: 14.3 (Build 123.4) and 15.0 (Build 620.29) I believe there are correct. Not sure what to check next!
Any help or pointers would be much appreciated.
DAGs are version-specific. An Exchange 2013 server can’t be a member of an Exchange 2010 DAG.
Right.
I wondered if that was the case as mailbox moves are working fine.
Time to install another 2013 box then!
Thanks for the information.
Great work Paul.
Question, II Have an environment with one exchange server with CAS, HUB and Mailbox with no services pack and the server is win 2008 R2 standard running out of memory – I need to install a window server 2012 with Exchange server 2010 sp3 in the same environment but migrating all users to the new environment and decommissioning the old server . What do I do in this scenario ?
Create new environment and use the move mailbox wizard. Leave the old server online until all clients connect so their Outlook profile gets redirect automatically. Then follow standard procedure to remove old exchange server.
Excellent Work Paul,
I am planning to upgrade Ex 2007 (RTM) single server to Exchange 2010 single server with latest updates. Kindly help me with your expert advise. We have Ex 2010 SP1 STD license.
My plan is as below:
1) Upgrade existing Ex 2007 to Ex 2007 SP2 to support co-existence.
2) Install Ex 2010 server and then follow the migration steps.
Question:
1) If i use Ex 2010 SP2 setup to Install, will it support Ex 2010 SP1 product key which we have, or I shall install only Ex 2010 SP1 and then install UR 8 for Exchange 2010 SP 1 (KB2787763)
2) As mentioned in the comments above. it is not recommended to use Ex2010 SP3 as it not proven yet..
Can i use Ex 2010 SP2 Setup for Installation rather than installing Ex 2010 SP 1 and then UR for SP1?
3) Minimum we need Ex 2007 SP2 for co-existence. right? or we must install Ex 2007 SP3 before introducing 1st Ex 2010 server. Ex 2007 update order will be as below:
a) SP1 > SP2 > SP3
b) SP1 > SP3
which one i shall follow
4) Last question how much downtime we need, during Co-existence.. i.e.:while changing / configuring CAS. HT roles.
Appreciate your quick response.. my project is starting in 3 days..
Thanks once again..
1. Service packs do not affect product keys, you will be fine. Install your SP2 setup.
2. If you are worried about Exchange 2010 SP3 then only install up to SP2 UR5-v2. The SP2 UR6 includes the issues that are with SP3.
3. I believe you only need to be up to 2007 SP2 to migrate
4. If planned correctly, you shouldn’t have downtime. During co-existence, the clients gets redirected to the new server automatically. I did mine from 2003 to 2010 and did it over a weekend. Depends on size of environment of course.
This might help to: http://www.petenetlive.com/KB/Article/0000236.htm
Thanks a lot Rick
Dear Rick.
I will do the installation using Exchange 2010 SP2 setup in below order:
Plan A:
1) CAS
2) HT
3) MBX
4) UR5-v2 for all above 3 roles.
Plan B:
1) CAS
2) UR5-v2
3) HT
4) MBX.
I believe Plan A is correct, Plan B is just a thought. whats your opinion?
Thanks Dude
Hafeez
Great summary, but as the Comments have outlined its not as straight forward as it should be.
I’m actually in flight with my rollout of E2010 SP3 as I write this… We have no multi-role servers so our install order is 7 CAS, 2 HUB, 3 Edge, and 4 Mailbox servers last. We’ve had SP3 installed and tested in our Lab for over a month now without issue, so I would support your 20-30 min estimate. Unfortunately that time does not account for Production User Load and Traffic, not to mention Lab database sizes are significantly smaller typically.
That said, from Midnight until 5am the CAS, HUB and Edge were completed successfully. I’d estimate them at 45-60 minutes each server. Caught a few winks before starting on the Mailbox servers at 9am this morning, and that’s where my headaches started.
After the 3 Retry on the Setup’s Readiness Checks in little less than 2 hours, I’m in the process of Rebooting the server entirely to start with a clean slate. This is only the first of 4 Mailbox servers on a DAG with 20 databases. Fortunately for the DAG configuration, availability is not an issue right now so I don’t have 10000 users breathing down my neck right now. I anticipated possible complications, so I gave myself a significant 36 hour change window. Did not expect 2+ hours on one mailbox server after the CAS, HUB and Edge servers went relatively smooth. Hopefully this is an isolated problem child with this server, and not indicative of the remaining rollout on the mail servers.
Will post updates and anything I discover…
After the reboot, it didn’t take the 4th try to fail Readiness Checks to PSKILL two background services that were running. Once those processes were killed the Readiness breezed right through this time, and not onto the actual Upgrade. Now into the next problem area of “Stopping Services”…
Stopping services failed after 20 minutes on WinMgmt service stuck in Stopping state. Used Process Explorer to identify the PID and manually killed the process ID overall. Reran Setup and it breezed through Stopping Services and into the meat of the SP3 Upgrade. 3 hours in on first Mailbox Server and making Progress at last!
Finished at last 3.5 hours later!!! Hopefully the remaining 3 mailbox servers go a bit smoother.
Ok, after 24 hours into a 42 hour Maintenance Window in which 12 of 16 Exchange 2010 Servers were brought up to SP3 from SP2 RU4v2 in 6 hours, and its taken me a little more than 12 hours to do the Mailbox Server, I am DONE! Here is a wrap up on lessons learned, and I apologize in advance for the haphazard writing and grammer…
On Mail Server 1, we didn’t run into any issue with the Discovery Search Mailbox, but it cropped up on Server 2.
There were two solutions out there for this issue related to E2010 RTM and SP1, but nothing I could find related to SP3:
1) Manually Disable/Enable/Permission the DiscoverySearchMailbox, or
2) Delete the DiscoverySearchMailbox altogether and recreate it again later.
We used Option 1 to complete Server 2 successfully, but the same issue cropped up yet again when we moved onto Server 3, so we exercised Option 2, deleting the DiscoverySearchMailbox altogether. My recommendation should you run into this error (which requires a Reboot before you can Retry again) just Deleting the mailbox and recreate afterward. That said here are the steps we took.
Pre-Install:
Open Exchange Powershell to run: StartDAGserverMaintenance.ps1
Disable TrendMicro Services
Disable SMEX Services
Disable WinMgmt Service
Reboot the Server
When the server comes back online, Enable and Start WinMgmt Service. *
* Not sure why, but SP3 has a lot of trouble Stopping all the WinMgmt associated PID’s during the Upgrade Phase, which causes the Stopped Services to take a long time only to timeout after 20 minutes. Restarting the Server with this service Disabled prevents a number of the WinMgmt subsystems from starting, thus is no longer a problem during the Upgrade phase.
Have PSTOOLS from SysInternals installed to use PSKILL in an Administrative Command Prompt. During the Readiness Check phase of the installation, it may stop because a service is running that it was not programmed to stop itself. Use PSKILL to kill the service(s).
Using Setup GUI, run as Administrator
Steps are fairly straight forward as outline in the article above, and they were just like that when we installed in our Lab for Testing. However, like I noted previous, things go wonky between Lab and Production implementation.
What goes on under the hood, SP3 Installation seems to have little resiliency for processes that take a little longer than usual to Stop or for Stopping (Even if with a Prompt to Confirm to Kill/End) a Process its unfamiliar with leading to A LOT of frustration and wasted time.
During the Readiness Check at least there is a Retry option when some process is found running that SP3 is unfamiliar with under the *Role Prerequisites section, and there will be thus the PSKILL at the ready.
During the Upgrade/Progress there is no Retry option, and ANY failure detected requires you to start the Upgrade all over again, and in some case Reboot before Retrying like in the DiscoverySearchMailbox issue noted above.
The most critical of stages in the Upgrade Phase to watch are the Stopping Services and *Role Upgrade stages. Unfortunately on two of my servers, Exchange services were slower to stop causing the GUI to timed out and restart the SP3 installation to proceed. This is also where we ran into the WinMgmt timing out after 20 minutes. Rather annoying.
When one server is complete, enable the Disabled Services above and Reboot.
Check Server and Service Health before running StopDAGserverMaintenance.ps1
Move onto the next server and repeat as above.
After your last server is done, Rebalance and Call it a Day… Have a good Single-Malt calling my name.
No matter how easy your CAS, HUB and Edge servers are, the Mailbox servers are going to take 2-3x longer because of this fiasco. Pad your Change Window with some extra time – just in case.
Unfortunately for me, I didn’t encounter any of these nuances in the Lab prior to Production Implementation; otherwise, I would have documented and prepared accordingly to make Production as smooth as can be.
Fortunately for you, I took the time to summarize my Production Implementation headaches and findings to hopefully save another poor soul some aggravation. Staying this current is the price we pay for being on the bleeding edge.
“Disable TrendMicro Services
Disable SMEX Services”
I’ve had a similar experience with SMEX and Exclaimer services.
Thanks for sharing your experience with the upgrade
Ok, there was a bit of unforeseen fallout that we have attributed to SP3 installation that I would like to share.
If you are still using OWA, SP3 appears to Change, Disable or Reset the Forms Authentication requiring it to be reconfigured in a specific way. Reference Article for configuration of Exchange-related Virtual Directories:
http://technet.microsoft.com/en-us/library/gg247612(v=exchg.141)aspx
Be sure to thoroughly test OWA access after installing SP3 and verify Forms Authentication if applicable to your environment. The strangest part of our testing was that some OWA users worked while many others did not, and since this is a Global setting that should effect all OWA users, I can only surmise that the “working” client had a Cached page, credential or certificate that would have eventually expired leading to the same issue.
Hi.
First I must say that this guide is excellent and it is perfect to follow in our attempts to upgrade 4 CAS- and HUB servers an then 4 mailboxservers.
I first did a /prepareAD and then setup /m:upgrade /Hosting because we have a hosted exchange with about 100 customer sites.
The problem we encounter when trying to upgrade the CAS/HUB servers, version 14.02.0247.005, was that it failed in the end of HUB transport role upgrade with the following error.
Configuring Microsoft Exchange Server
Language Files COMPLETED
Restoring Services COMPLETED
Languages COMPLETED
Hub Transport Server Role FAILED
The following error was generated when “$error.Clear();
if ( ($server -eq $null) -and ($RoleIsDatacenter -ne $true) )
{
Update-RmsSharedIdentity -ServerName $RoleNetBIOSName
}
” was run: “Object ‘servage.local/Users/FederatedEmail.4c1f4d8b-8179-414
8-93bf-00a95fa1e042′ in organization : the linked object of property ‘RMSCompute
rAccounts’ is ‘servage.local/Microsoft Exchange Hosted Organizations/Servage/com
puters/CAS04′, which doesn’t exist in the same organization with the object. Pro
perty Name: RMSComputerAccounts”.
After searching for info on Google the tip was to delete this user with ADSI edit and rerun the upgrade and it would succeed. The user could be added after the upgrade job was finished. But when I did a rerun it failed at on the same spot but with a different errormessage.
The Exchange Server setup operation didn’t complete. More details can be found
in ExchangeSetup.log located in the :\ExchangeSetupLogs folder.
Exchange Server setup encountered an error.
Configuring Microsoft Exchange Server
Language Files COMPLETED
Restoring Services COMPLETED
Languages COMPLETED
Hub Transport Server Role FAILED
The following error was generated when “$error.Clear();
if ( ($server -eq $null) -and ($RoleIsDatacenter -ne $true) )
{
Update-RmsSharedIdentity -ServerName $RoleNetBIOSName
}
” was run: “RMS Shared Identity user FederatedEmail.4c1f4d8b-8179-4148-9
3bf-00a95fa1e042 not found.”.
The Exchange Server setup operation didn’t complete. More details can be found
in ExchangeSetup.log located in the :\ExchangeSetupLogs folder.
Exchange Server setup encountered an error.
I’m stuck in a loop here, because if I add the user manualy or make a prepareAD (user could be corrupt) again it fails again, so any help would be appreciated.
Thanks
Hi all!
i have E2010 RTM, 2 domains (domain.local and comp.domain.local) and 2 servers:
-CAS/HUB in domain.local
- MB in comp.domain.local
all users are in comp.domain.local
All domains and forest level = 2003!
What is step-by-step upgrade for my Exchange??
The step by step process is explained in the article. Is there something specific you’re not sure about?
Paul, what about
setup /PrepareSchema
setup /PrepareDomain
setup /PrepareAllDomains
??
i need run this command if i have forest with 2 domains ???
You can use /PrepareAD for the forest root (which automatically does /prepareschema and /preparedomain for the local domain).
Then you can use /preparedomain or /preparealldomains for any additional domains in your forest.
Dear All,
I am close to Exchange 2010 SP1 to SP3 upgrade.
Have few confusions.Please help me to clear the confusions prior to start.
I am confused with http://support.microsoft.com/kb/2668686 and http://support.microsoft.com/kb/2810617
On my Exchange servers all powershell execution policies are undefined except “LocalMachine” set to “RemoteSigned”.
Which KB article should I follow as I have Exchange 2010 SP1.
Also if upgrade fails, what will be the workaround ?
Do I need to use RecoverServer Switch OR any twik is there to restart SP3 installation ?
Thanks in Advance..
Regards
Mahesh
I have a question to all that have upgraded to Exchange 2010 SP3.. What are you using to backup your DAG’s, I use Backupexec 2010 R3 and apparently Exchange 2010 SP3 is not supported yet?
I have been told from PS people that the database schema upgrade only occurs if you are upgrading directly from Exchange 2010 RTM.
Hi Brenton,
I have confirmed with product group that there was no schema update in SP2 or SP3. SP1 was the last one. The release notes mentions it because someone could go from RTM directly to SP3 and they would see the upgrade happen when the DB is mounted.
Best Regards,
Johnny
Correct. Unfortunately the initial version of the release notes was quite vague about that. Glad to see it got updated.
“If you’re upgrading the Mailbox server from the release to manufacturing (RTM) version of Exchange 2010 to Exchange 2010 SP3, the database upgrade process could take an additional 30 minutes or longer per database. You can track the progress of the database upgrade process by examining event 1185 in the Application event log on the server you’re upgrading.
If you’re upgrading from Exchange 2010 SP2 or SP1 to Exchange 2010 SP3, the upgrade process takes less time. You also won’t see any 1185 events in the event log.”
For a multirole environment what is your thought on applying SP3? Does the same process in this blog still apply?
multi-role servers (M/C/H)
2 DC’s
Active/Passive
HLB
Thanks,
Jack
Yes. Start with internet-facing servers first, and combine the CAS Array/DAG preparation and steps together for the multi-role servers.
• Yesterday I have upgraded Exchange 2010 service pack 3 for my customer successfully.
Prior to update I had searched MS online documentation and numerous blogs and posts for Exchange Service Pack upgrade so as to avoid or minimize failures.
Based on my experience and findings, it is not painless activity but we can try as much as possible to make it painless by implementing proper prerequisites and proper plan. Just thought to share all those prerequisites and planning we had might help to Exchange Administrators.
• Precautionary Measures Prior to Deploy Exchange 2010 Service Packs 3:
o Important and appropriate latest Windows Server patches must be deployed.
Do not update KB2506143 (Not supported for exchange 2007 & 2010 and might create issue)
o Exchange BPA needs to be run and address any critical issue reported by him.
o Check\search MS online KB articles for known issues with Rollup update \ Service packs.
For Ex. http://support.microsoft.com/kb/2810617 in case of Exchange 2010 SP2 RU6 and SP3
o Download ExPerfWiz.ps1 from http://experfwiz.codeplex.com/ and Execute Performance data collection Script for Exchange 2010 server to identify any performance bottle necks related to Memory, Disk Sub System etc. You can get more info about command syntax from above site.
o If you are upgrading from RU to newer RU or newer Service Pack, check if Previous RU setup (.msp \ .msi) files must be resided in %Systemroot%\Installer folder or upgrade will fail. Check article. http://support.microsoft.com/kb/2784788
o Ensure that account to be used must be domain account. Account must be member of Exchange Organization Management and must be local administrator on exchange server. If the same account is used to update AD schema, then it also must be member of Domain Admins, Schema Admins and Enterprise Admins Group Membership.
o Service Pack \Rollup update sequence must be as below.
CAS
HUB
UM if you have
MBX
Edge Transport
• Activities to be done on Exchange Server to be upgraded prior to Deploy Service Pack:
o Ensure that you have Latest AD and Exchange Server System State Backup with all Mailbox Databases full backup
o If you prepared AD schema manually, then force replication to all domain controllers in forest.
o Uninstall \ Remove any Interim Updates (IU) provided by Microsoft to address specific issues as Service Packs are cumulative and contains all fixes since last service pack.
o Microsoft has published 2 articles related to PowerShell Execution Policy as below.
http://support.microsoft.com/kb/2668686 – Correct One
http://support.microsoft.com/kb/2810617 – Don’t understand why MS published this confusing article.
First one is the correct and set your exchange server powershellexecutipolicy accordingly.
o Check Exchange Server IPv6 status in network card properties and enable it if not enabled already.
o Disable certificate Revocation Check on Exchange Servers. Check below link.
http://eightwone.com/2009/12/13/speedup-exchange-2010-rollup-1-setup/
o Stop and Disable Antivirus Software services completely.
o Stop and Disable Backup Software Services Completely.
o Stop and Disable SCOM services if any. Also on SCOM server put Exchange Server in Maintenance Mode.
o Stop and Disable SCCM agent services if any.
o Stop and Disable any 3rd party software processes / services.
o In case of Hub Transport Server, disable windows Firewall for all profiles but Firewall service must be running. Remove server from NLB. Also remove server from Send Connectors.
o In case of Mailbox Server, Firewall must be enabled and running. To avoid any disruption, add a custom inbound and outbound firewall rules which allows all programs and allports through all firewall profiles. Check below Link.
http://technet.microsoft.com/en-us/library/bb691354(v=exchg.141).aspx
Also put server in Maintenance Mode with scripts provided by Microsoft or as guided in below link.
http://exchangeserverpro.com/how-to-install-updates-on-exchange-server-2010-database-availability-groups/
o Make sure that File Share Witness (FSW) is online if it is other than HUB Transport Server.
o Copy Exchange Service pack Binaries in compressed format on Exchange Servers and Extract it with some extraction software to avoid file alteration during transit or copy process.
o Lastly reboot server to be upgraded and then start Exchange setup from extracted dump.
o You must run Setup with “elevated Command Prompt” or “Run as Administrator”.
After obeying above Hugh checklist I got success to deploy Exchange 2010 Service Pack3 over SP1 in production environment without ant failure.
Good Luck
Thanks Paul for this great article, it’ll be really helpful when i’m gonna upgrade our Exchange 2010 servers next week.
Thanks Mahesh for the additional tips.
Has anyone an idea about how long the upgrade of the databases would take ?
5 Databases each around 180GB.
Regards Misha