For some time, I had been getting random account lockouts.  All the usual suspects had been interrogated, scheduled tasks, drive mappings, credential manager, etc.  As the problem started to annoy me more and more, I took a deep dive determined to find the root cause of the issue.

When I would get locked out, the AD domain controller would show that it was an Exchange server actually showing the bad password attempts.  The exchange server in question would change, it wasn’t any one in particular and they all sat behind a load balancer.

The first step was to check these exchange servers and make sure I didn’t have any scheduled tasks or the like running on those servers.  No such luck.

I also tried using troubleshooting tools from Ingo Gegenwarth (https://ingogegenwarth.wordpress.com/2015/02/14/troubleshooting-exchange-with-logparseriis-logs-1/)  to examine the IIS log files on the exchange servers to see if an Exchange activesync device was causing the bad password attempts, but that investigation turned up nothing as well.

To be extra sure, I disabled Exchange ActiveSync and Skype for Business on my mobile phone for a few days and saw that the issue persisted.  At this point I was sure that something else was causing the bad password attempts, maybe the phone could still be a contributing factor, but even with it disabled, it still occurred.

I started to notice specific behavior though, for example, rebooting my PC would cause it to happen more frequently, probably as the system was starting up.  Using the Microsoft Account Lockout tool (https://www.microsoft.com/en-us/download/details.aspx?id=15201) I could see the domain controllers that were getting the bad password attempts, sometimes it caused enough that it caused my account to get locked out.

Seeing the exact time of the lockouts on the domain controllers, I then started examining my system to look for events happening at the same time.  Bingo!  I found a smoking gun, no bullet yet, but definitely smoke.  At the same time the bad password got logged on the DC, I had a 4648 Logon event on my PC showing an Audit Success and it showed the Exchange server where the bad password or lock occurred listed in the Target Server information, just as the DC was showing as well!

Now, I received another clue!  It shows the process ID: 0x15c4.  Since this is in hexadecimal, I had to convert this to its decimal equivalent: 5572.  The process ID or PID is important here because there are many svchost.exe process that run on a Windows computer.   SVCHOST.EXE is a process that acts as a container for many services of similar types.  You can read up more about it here: https://en.wikipedia.org/wiki/Svchost.exe

Using Microsoft Process Explorer (https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer) from the Sysinternal’s tools, I took a look at svchost.exe PID 5572 I found this:

As you can see, that particular svchost process had a few services running within it.

CDPUserSvc_14f82f
MessagingService_14f82f
Sync Host_14f82f
Contact Data_14f82f
User Data Storage_14f82f
User Data Access_14f82f

This was something I hadn’t noticed on my system previously.  They are part of this UnistackSvcGroup.  What is the UniStackSvcGroup?  Googling only increased the mystery level, lots of talk, but very little concrete information.  As an example: https://answers.microsoft.com/en-us/windows/forum/windows_10-other_settings-winpc/7-services-on-win10-tied-to-mysterious/970ffeff-c338-48d3-b250-8152b289011d

And https://superuser.com/questions/1115769/what-is-the-cdpusersvc-service/1118109

Finally, https://social.technet.microsoft.com/Forums/en-US/c165a54a-4a69-441c-94a7-b5712b54385d/what-is-the-cdpusersvc-for-?forum=win10itprogeneral provided some of the best info showing me that these services were doing something with mail, calendaring, etc.

Also, some talk about “Project Rome” and “Universal Glass”, pretty interesting stuff.

As a FYI, the characters after the underscore will change it seems after a reboot, so everyone’s system will show something different for those last few characters.

So, what to do next…I wanted to confirm that these services were the cause of the issue I had been seeing, so I put together a quick script to keep these services stopped as disabling them could lead to possible system issues according to some of those posts:

I ran the following in a PowerShell prompt:

$a=0;while ($a -ne 1) {$date=get-date;stop-service *_14f82f -Force;write-host “Run time:” $date -ForegroundColor Cyan;get-service *_14f82f;sleep 30}

This checks the state of the services every 30 seconds and stops them if they were running.  Using this script I was able to see my system had no bad password attempts, except at times when one of the services tried to start up on its own. (The screenshot below shows where the last few characters changed to 84532)

Now it was time to see what was happening when these services were starting up to cause the bad password.  I ran Wireshark to capture packets and then manually started the “Sync Host_14f82f” service.  Sure enough, a bad password was tried, confirmed via the Lockout tool and in my systems security log with a 4648 event showing the Exchange server it occurred on.  Looking at the Wireshark capture at the time of the lockout showed my laptop talking to the Exchange load balancer.  Setting up a display filter in Wireshark allowed me to see more clearly the issue.

My system was trying to communicate using IMAP on port 143 to the Exchange load balancer.  WHY?  I wasn’t using any IMAP software, but rather Outlook connected to Exchange.  Looking further I see reference to “Windows Mobile” and ultimately a TLS Fatal Handshake Failure.  Clearly, my system was trying to communicate via IMAP and it was failing due to the bad password.  What on my system could be using IMAP when these mysterious services were started?

Then it hit me…..I had setup Windows Mail that comes with Windows 10 about a year and a half prior to test IMAP when I was setting up a new F5 load balancer for our Exchange environment.  I started up Windows mail and sure enough, it presented me with the following:

“The search is over, you were with me all the while” (Maybe those who grew up in the ‘80’s get the reference). At least I figured it out and I was a survivor.  OK, I’ll stop now!  Haha 😊

In summary, if you or someone complains that they are having account lockout issues, bad password attempts and it looks like an Exchange server is causing the lockout, check to see if on any of their devices are any forgotten about email clients with an old password configured.

But, in this case, the really interesting thing was that I was never actively running the Windows Mail client!  These UniStackSvc services were kicking it off in the background, most likely checking for new mails, etc. from time to time, completely unknown to me.

Hopefully this helps someone experiencing a similar issue.