Written by What a week, last week. Last week I experienced one of the strangest problems I have ever encountered and I ended up using Microsoft Premier Support. I have worked day and night for the past week so forgive me if I forget to mention something I tried during that week. Let me take you through the case.
I had an engagement at a customer that wanted a solution for deploying Windows 10 to some specific models. In this case it was the HP Elitebook 755 G3 and the HP Elitebook 820 G3 models. As I always do, I used Microsoft Deployment Toolkit to build the image with Office 2016 Pro Plus and all the available patches (I also included Visual C++ and .NET). I used the latest Windows 10 ISO available from Microsoft, the 1511 version. After successfully building the image we tried to deploy the image to the machines using Microsoft Deployment Toolkit. Everything worked like a charm except for the activation process of Office.
The customer wanted to use MAK keys for Windows and Office, since they did not have the KMS infrastructure ready and they were used to use MAK. As I said to the customer: “It does not matter to me if it is MAK or KMS, both will work just fine.”
So in the task sequence we added the following for activation wrapped inside a little script that did some other stuff also (nothing related to this issue):
- cscript c:\windows\slmgr.vbs /ipk #####-#####-#####-#####
(customers MAK key for Windows 10) - Cscript c:\windows\slmgr.vbs /ato
- Cscript “c:\program files(x86)\Microsoft Office\Office16\ospp.vbs” /unpkey:#####
(the last 5 digits of the KMS client key) - Cscript “c:\program files(x86)\Microsoft Office\Office16\ospp.vbs” /inpkey:#####-#####-#####-#####
(customers MAK key for Office 2016) -
Cscript “c:\program files(x86)\Microsoft Office\Office16\ospp.vbs” /act
This normally works just fine in every other installation I have ever done, but when we looked at activation status for Office 2016 it said OOB_GRACE and the KMS client key was still present. Our first thoughts were of course that we added something wrong to our task sequence or one of our scripts, so we used a fair amount of time trying to pinpoint what was causing the issue. We then tried to deploy the same process to other machines than these two models and it worked flawless. After some troubleshooting and a lot of deployments to different models, we pinpointed it to be only on these two specific Elitebook models. I know what you are thinking right now. Yeah, it must be the drivers for these models. We had the same thought, so we tried to update the drivers. The first drivers we used was from HP and was a readymade SCCM driver package. We compared these drivers to the ones that where available from web and we added newer drivers to our deployment share. After adding newer drivers, we tried to re-deploy to the Elitebook models, but it still failed. Then we tried to deploy the Elitebook without drivers except for the NIC driver, still no cigar.
We started to do deep analysis of the problem looking at the event log, using Wireshark and procmon. After a lot of troubleshooting the only interesting part we found was that the Software Protection terminated unexpectedly many times. After a lot of frustration and pulling our hair, we built a clean Windows image without Office. We then deployed and installed Office manually just to make sure that the customization we used in our Office setup was not the issue. The problem still persisted. We then came up with the idea that we would add the MAK key directly into the Office Customization wizard to make sure that the KMS client key was gone. We built a new image with the Office MAK embedded into the installation, then we deployed it. It still did not work.
We (me and the technical person at the customer (you know who you are 🙂 )) found out that we would take one computer each and install it manually with Windows from a memory stick, install Office and the patch it up. After that, try the activation. We did that and I was able to activate, but he was not (even more strange). And we used exactly the same model. We discussed what we did during the installation, but could not find any obvious difference during our installation. The deadline in the project was approaching and we had used a lot of time trying to pinpoint the problem and find a solution without luck, so we decided to create a support case at Microsoft to get some help.
I then came in contact with a great and helpful person at Microsoft support that worked with this case the rest of the time. After explaining the problem and showing that it was impossible for us to remove the KMS client key or activate using MAK. The Microsoft support technician insisted that we started from scratch and he pointed out that maybe some of our customization could be the root cause of the problem. I agreed (but still certain that I did nothing wrong since it worked on all other models), so we started by installing Windows clean from Microsoft ISO files. We of course used the memory sticks we created during our testing of the clean install we did the night before.
Me and the support technician at Microsoft created a list of test scenarios to eliminate problems to find out if it was a Microsoft issue or third party. We created the following test plan:
-
Install Windows, office then activate (verify clean install works)
- If activation worked, then update the installation to verify that it remains activated (eliminate patches)
- Install Windows, Office, update and then activate
-
Install Windows, Office, Update, HP drivers, then activate
We went through the first 2 scenarios and it worked fine. During scenario 3 I noticed that the 1511 upgrade of Windows appeared as an update. 1511 takes some time before it appears in Windows update and that is why I probably missed it in scenario 2. I reinstalled the machine in scenario 3 with a new memory stick containing the 1511 version of Windows (this should not be a difference right?). Right before I was done and was about to start the activation process I had the support technician on the phone and we both agreed that this would probably work since drivers normally does not affect the activation process of Office. I started typing the commands and when I typed the /dstatus to view the result I saw that it actually did not get activated and I experienced the same as I did when using our deployment solution.
Finally! – Some progress. We now had a working scenario and a not working scenario.
(Sorry for the bad screenshot, it is from a movie I recorded from my Phone.)
We decided that it was probably the drivers, so we created new testing scenarios:
- Install Windows 1511, Drivers(from the sccm driver package), Windows updates and Office
- Install Windows 1511, Office, Windows updates, drivers(from the sccm driver package)
- Install Windows 1511, Windows updates, drivers(newest available), Office
-
Install Windows 1511, Office, Windows updates, drivers(newest available)
I went through scenario 4 and while it was installing I decided that I should really try scenario 1 and 2 with the 1511 ISO just to be 100% sure. So I worked on that in parallel and discovered that all that I needed to trigger this fault was to install Windows 1511 and Office 2016. I immediately called Microsoft support again, and he verified my discovery. So this issues are not related at all to drivers (could be those that comes from Windows update of course) and it is a purely Microsoft problem. This problem only occurs if you have HP Elitebook 820 G3 and HP Elitebook 755 G3 and you try to use MAK activation, either by adding the key with Office Customization Wizard tool or by using the script to remove the KMS client key and then adding the MAK key. The support technician told me that this issue will be further forwarded into Microsoft and that the outcome will probably become a hotfix and a KB article (WOHO!!! My first – I will update this post with the article number (if or when it is present)) This issue is not a top priority for Microsoft since the impact is so small. I mean, you have to have these 2 models (as far as we know – could of course be even more new HP models) and you need to use MAK activation. Another scenario that we tried that I have not mentioned, is that we tried to install Office 2013 instead of 2016 on Windows 1511. This also fails.
In this project we ended up using KMS server as a workaround and it worked very well. Another workaround could be to deploy RTM version of Windows, install Office, activate and then upgrade to 1511.
If you remember, we (me and the customer) installed two machines manually and experienced different results. This was of the fact that he installed the 1511 update and I did not. Thanks to all that contributed in finding this out – And when I say all I mean the customers personell, Microsoft support and colleagues.
My colleague Christian Knarvik also found this KB that shares some similarities https://support.microsoft.com/en-us/kb/3170450
I hope that this blogpost helps someone facing the same issue, since I could not find any information about this problem when I was facing it. Sharing is everything. 🙂