Pulling my hair out on this one - Account keeps getting locked out.

cruwl

So one of our upper management users account keeps getting locked out and we cannot for the life of us track it down.

user is a laptop user, Does not use roaming profiles. Issue follows user from machine to machine, we have recerated her profile several times. user is not typing her PSWD to lock her self out. she gets locked out several times a day.

Our network takes 5 bad PSWDs to lock you out. Some times all 5 happen in the span of 20-30 minutes, some times an hour or 2, average 1.5-5 minutes between bad PSWD attempts. when she goes home the bad PSWDs stop. We have had her change her PSWD several times on her laptop to try and sync it with what ever application it might be, no change.

I use MS account unlock tool and can see the account gets locked out on the same 2 DCs each time. We installed the DLL and reg key on her laptop to log whats causing it, only the debug log was never created. This makes me think its not something on her system other wise it should have been logged.

viewing the netlog.log from accountlockouttool.exe on the primary FSMO role holder shows the auth attempt was passed to it from the other DC. This DC is not just a DC, Its a SQL box, DNS server, DHCP server, has FS and IIS roles installed as well. (I was not the one who set this up)

any thoughts on tracking this down?
My thought for tomorrow was to turn AD DS off on this DC and see if the Auth request passes to another DC, and maybe logs more info.

Oh Also the Security log on the DC doesnt show source machine, its just blank, this again kinda makes me think its the DC its self.

rsutton

cruwl wrote: »

Issue follows user from machine to machine

How many computers does the user have? Do you have a terminal server of any type? Check that the user does not have a mapped drive on any of their computers, or terminal sessions.

Essendon

This is a strange one indeed. Usually the security log shows the source machine and you have the culprit. Is the log blank on both DC's? Maybe the culprit is a mobile device, if she has one? Hung remote session?You've done a lot of troubleshooting, it may be time to create her a new account? Log it with Microsoft while you do further investigation. Whats the event log ID?

ptilsen

cruwl wrote:

This DC is not just a DC, Its a SQL box, DNS server, DHCP server, has FS and IIS roles installed as well. (I was not the one who set this up)

Bad practices alert! I know you didn't set it up, but if you have the power to stop it, now is the time.

Anyway, I suspect either an application using the IIS site or SQL database, or something she consistently does. It would be worth doing a packet trace while she is working and analyzing it later. It might be time consuming, but it is probably the best way to get to the bottom of this.

Mobile Device @ Essendon is a good call, too. If user goes home that could be it just as easily. If you have Exchange and wifi that mobile devices go on, an ActiveSync configuration could absolutely do this.

Bundiman

My guess would be a IOS or andriod device that is trying to connect with and old password. Maybe a VPN connection from home that is trying to authenticate. Do you have a AAA server check those logs.

kj0

Had a user this morning. iPhone of course. Connected to the wireless. She had changed her password and the Device was still trying to connect. Just said not to use their phone. Their isn't a point for us to connect them to the wireless as they just get Filtered internet.

astorrs

This doesn't help you narrow down the source of the problem, but the practice of "5 wrong passwords and then lockout the account" is pretty dated. Most security best practices these days in Enterprise deployments suggest ~15 wrong passwords before locking, then automatically unlock the account after 15-30 mins or so, etc.

https://benchmarks.cisecurity.org/tools2/windows/CIS_Windows_Server_2008_Benchmark_v1.2.0.pdf

petedude

Lame guess on my part, but could this be some sort of cached password/mismatched Kerberos issue?

thronetm

Check her Credential Manager. To be honest, on her Laptop I would just remove all entries from there and then test..

blargoe

You should still be able to track this down. Keep following the trail. If your PDC says the source is the other DC/App Server, look at the logs on that server. Maybe the user was authenticating against that server initially, and you will see the "real" source IP, or maybe this server really is the source, and maybe you will find the offending process that generated the invalid logon (like a connection to the SQL Server).

Look for the Microsoft utlity EventCombMT and use that to filter through the Security Logs on your DC's for this guy's user ID. From there, you will either find the "real" source IP (in which case, you should run the tool against THAT computer's logs), or will find another clue, such as a logon type or a process.

I have seen smartphones be a culprit as well. We had one guy for whom someone on the support desk set up his AD credentials on his Blackberry for connecting to the Wireless network. He never used it and never knew he had this set up. After his regularly scheduled password change occured, every morning when he reached the parking lot, his phone would lock out his AD account. Support desk chased that one down for a week... why was his account locking out before he even started working on his computer?

lsud00d

I've seen this issue because of a password change and old password trying to authenticate:

in an email client
on a phone (connecting to email or wifi)
in an mstsc session

Does the person use remote desktop at all?

cruwl

Here is an example of the security event:

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 2/13/2013 10:59:04 AM
Event ID: 4776
Task Category: Credential Validation
Level: Information
Keywords: Audit Failure
User: N/A
Computer: DomainController
Description:
The computer attempted to validate the credentials for an account.

Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Logon Account: USERAccount
Source Workstation:
Error Code: 0xc000006a


As you can see the source workstation is blank.

You all had great Ideas.
she has an Iphone, we Verified the wifi is off, issue still occurs.
Also issue will stop happening if she leaves the office, goes home VPNs in and the issue starts up again.

The user only has 1 laptop, but when she forgets it she uses a loaner.

I will try the credential manager, and check for mapped network drives. I'm sure she has mapped drives just not sure if she set them up or they are GPO mapped.

biggene

cruwl wrote: »

she has an Iphone, we Verified the wifi is off, issue still occurs.

Does she also get her corporate email on her IPhone? If she does and there has been a recent password change, you will need to go into the Outlook account set up on her IPhone and change to the new password there also.

Hope I was of some help.

Gene

crrussell3

My guess is she has one of the following going on:

1. Scheduled task or service set to run as her with old credentials
2. Application or website that requires AD authentication to work where she saved her old credentials.

blargoe

The service information should be available in any corresponding 4771 events.

You might want to check the ERRORLOG file on the SQL Server, and any logs on any other services that might be running on the 2nd domain controller. We had a Wireless Authentication agent running on one of ours at a different job that would generate these bad password attempts, for example. That narrowed it down to it coming from a wireless device, and our network admin was able to take it from there and determine the culprit.

cruwl

Ok an update:
credential manager on the laptop was empty.
all of her network drives are mapped by GPOs.

I can now replicate a bad PSWD.
Had her go to our report website and try to view a report. As soon as she hits view report a bad PSWD attempt is made.
I cleared cookies, temp files, forms, passwords ect out of IE.
Still occurs when she attempts the report.

started looking through security logs on the report server, I only see successful kerberos events for her account, no failed attempts.

I turned AD DS off on the DC mentioned earlier.

Also outlook started prompting for user name and PSWD, Had her manually reset her PSWD via CTRL+ALT+Del on the laptop. she also updated the iphone to use the new PSWD. Verified the PSWD is updated on all DCs except the one with AD DS turned off.


Continuing to monitor her account....

crrussell3

Does the report server require an ODBC connection which may contain her saved old password?

cruwl

just checked and it doesnt have any user DSNs, it has 2 system DSNs, neither are configured with her account.
The SQL DSN is set to use integrated windows Auth.

RKDus

Are you 100% sure that event viewer is not showing the source of the lockout? Sometimes they show the server name and sometimes just the IP. Also if one DC has identified another as the source then check the logs on that DC, you have to kind of follow the trail with these ones.

Does the user ever use RDP? Disconnected sessions can sometimes cause lockouts if the user changes their password.

jayc71

Phone. 90% of the time this happens to me, it's a phone someone configured to check their email.

biggene

Cruwl,

Did you ever get this issue resolved?

cruwl

Nope not yet. We had maintenance this last weekend so all the windows boxes got patched and rebooted, we were praying this fixed it. So far I have not seen the users account get locked out yet. But as I look this morning I think It might have been locked out last night.

MrSharp

Um, i know this is not getting to the root cause of the issue for those curious; but wouldn;t a simple workaround be to make a small modification to the user's login name?

cruwl

It looks like we were lucky and after the maintenance weekend the issue stopped occurring, As i Look at the users account now there hasn't been any bad PSWDs since the 25th of Feb.

I'm not sure if that would have corrected it or not but I would think it would have as well.

lsud00d

What maintenance did you do? Were any of the DC's rebooted?

cruwl

WSUS updates, and rebooted every windows box on the network which included all the DCs.

anderssose

Hi, we have the same problem with users using wireless devices such as iPads, iPhones & androids. We have setup a rule on the radius servers that blacklists the users on the radius server when she reaches 9 bad passwords which is 1 less than the Default Domain Policy to lock an account. Inspite of this some users keeps getting their account locked and when we check the security logs on the DC's the line Source Workstation is always empty AND the code is 0000000x6a indicating bad password.

zose

Cruwl, have you got any more info about the root cause to the lockouts?