Incident Response tips

What a day. I had my first taste of incident response today and it didn't go so well. I received a call that an internal user was receiving emails from another internal user, only the user wasn't sending those emails. At first, they thought the email address was being spoofed. I ran some traces on O365 and looking at the header, I could tell the email was definitely not spoofed. The account was compromised. I was able to tell from the Originating IP, that the email was coming from a remote location in the US. I reset the user's password immediately, and logged off all remote sessions for this user ID. We don't allow mail forwarding, so I quickly looked at mailbox rules.The attacker created some mailbox rules to hide the responses from the user they were sending emails to, in a different folder. At this point, it's clear what happened. The password was changed, so I can continue with my investigation. I had the pc off the network and looked at for keyloggers,malware, etc. All clean. I asked the user if they've given out their password to anyone or logged into webmail or any other computer. Their answer was no. I asked if the password was the same or close to any pw that is being used for other services, answer was no.
I checked for failed login attempts and 1 day a few weeks ago, there were about 30 failed attempts from a single IP in China, but there was never a successful attempt from that IP.
I then reviewed o365 logs for this user to see if there are any abnormal logins. Other than the ones from this incident, I could not find a single one. I went back months, and the user had changed their password at the end of last month. I ran a message trace to see if any emails contained hyperlinks that could be phishing links. I couldn't find any.
I feel like I might be missing something very basic, but can't put my finger on it. I have no answer for this, other than the answer I've been saying for a long time, we need multi-factor auth to help combat this. However, that doesn't give me the answer I want, which is how did they get the password in the first place???
My assumption is if it was brute forced or some other kind of password attack, I should see a lot of failed attempts and then a successful attempt.
I'm taking the L on this one.
I checked for failed login attempts and 1 day a few weeks ago, there were about 30 failed attempts from a single IP in China, but there was never a successful attempt from that IP.
I then reviewed o365 logs for this user to see if there are any abnormal logins. Other than the ones from this incident, I could not find a single one. I went back months, and the user had changed their password at the end of last month. I ran a message trace to see if any emails contained hyperlinks that could be phishing links. I couldn't find any.
I feel like I might be missing something very basic, but can't put my finger on it. I have no answer for this, other than the answer I've been saying for a long time, we need multi-factor auth to help combat this. However, that doesn't give me the answer I want, which is how did they get the password in the first place???
My assumption is if it was brute forced or some other kind of password attack, I should see a lot of failed attempts and then a successful attempt.
I'm taking the L on this one.
Comments
There have been numerous credential harvesting attacks targeting o365 and other externally exposed email portals, sometimes originating from other infected legitimate business contacts. Those emails can sometimes appear as an o365 document share request that prompts you to sign in first but the user enters their credential into the phishing site. Minutes later the attacker attempts to sign in their o365 account and succeeds if you don't have 2FA enabled.
We've actually had to block entire domains of two of our largest business partners due to outbreaks described above until they implemented 2FA because keeping up with the infected accounts sending us phishing links was an exercise in futility.
2020: GCIP | GCIA
2021: GRID | GDSA | Pentest+
2022: GMON | GDAT
2023: GREM | GCWN | GSE
WGU BS IT-NA | SANS Grad Cert: PT&EH | SANS Grad Cert: ICS Security | SANS Grad Cert: Cyber Defense Ops
@paul78 I did check the mail logs. Nothing outbound looks suspicious. The rules that were created just moved the emails into a subfolder where the user wouldn't check. The attacker was logged into OWA from a remote computer, responding to their victim, while the actual user was working. The initial login happened in the early morning hours before the employee comes to work. I also believe they were phished, just cannot prove it.
Lack of 2FA on email portals, VPN and SSO is a huge and unacceptable problem right now that could have prevented this issue. At least you have some ammo now to show a critical need in this area.
Best of luck to you sir!
2020: GCIP | GCIA
2021: GRID | GDSA | Pentest+
2022: GMON | GDAT
2023: GREM | GCWN | GSE
WGU BS IT-NA | SANS Grad Cert: PT&EH | SANS Grad Cert: ICS Security | SANS Grad Cert: Cyber Defense Ops
Thanks for the info
Did you check the system logs? If they weren't a sophisticated group they probably just flat out cleared the log. If otherwise, there are ways to remove specific logs from a specific day/time. Can you narrow down your time window anymore?
PHP
Kotlin
Intro to Discrete Math
Programming Languages
Work stuff
2FA, unless it is fully seamless (i.e. machine/user certs), is typically a pain for users as it slows things down and the work performance drops down a bit -- business hates it. But that's the price one has to pay for "ubiquity of access" that comes with the cloud territory. I hate cloud if you ask me...
And yeah, as others have said, you need this moron's browser history. Use a Nirsofer's Browser History View tool, for example, a free tool that does miracles. Also, check their browsing history on your firewall, which may not reveal all the URLs (unless you do MitM on it, like your name suggests). If they deleted their browser history view, which is typical for users when they feel threatened by a subsequent investigation that may expose their wrongdoings, try to collect it from shadow copies. But cleaning up the history is a tell by itself: if they did it -- it increases the chance that they did the offense.
EDR won't help against phishing. Phishing is a huge problem and not many things help against it, the most successful one I've seen is the anti-phishing module of Checkpoint's Sandblast endpoint security software. Educating users through security awareness program should be there, but it's barely helpful. Users are just being dumb and in a large enough enterprise someone will click, someone will enter credentials, etc.
What I've found to be helpful on user education part is making sure everything works via SSO and you communicate to the users like every month that under no circumstances any company software asks for user's credentials besides the initial windows logon screen. This makes them suspicious if something asks for their creds.
I think this is the most common situation, Passwords used on multiple systems. Just have to obtain one and apply to all other accounts users have. Face it the average user is a complete moron, hackers don't have to be rocket scientists, just a tad smarter than the average user.
Have you investigated created accounts in the past 30, 60, 90 days? An attacker will, more than likely, attempt to create an account for persistence (to avoid losing access when they are discovered, and the admin changes the password to the account they compromised).
Did the user have admin access to any resources? If so, you should investigate those resources for any abnormal logins from that user's accounts, or any other account that doesn't seem right.
Long term solution, and you're on the right track, is 2FA. Good luck!
I think that 2FA can give a false sense of security, convincing end users to use a weak/dictionary password.
Until you can get management to agree on 2FA, what you can implement in the short term is increased password strength rules, reuse/change policy.
It will take time for management to agree, set policy, select vendor, set up 2FA architecture, educate users and distribute tokens. (unless already in place, but waiting for teams to throw the switch) There are a lot of resources at the hackers disposal that make cracking simple passwords a trivial task. Increased password length+strength will make the passwords "resistant" to brute forcing, but not eliminate the threat. I don't know the O365 setup, but willing to gamble that there is a hash of the password somewhere on the system.
Problem is, if you have users on the real internet with untrusted devices, you cant control their level of access on their own PC and if they have functional anti-malware solution. No malware can lead to viruses like a keylogger or proxy installed which would lead to zero failed login attempts. Users on untrusted devices do things like creating a spreadsheet called passwords.xls making easy work for midnight script kiddies.
Lastly, don't always blame the user! Once upon a time, I used IE11 (I accept all responsibilities for my past decisions) and used the default "blank" Bing startup page. One day I get a popup message telling me to dial an 866 number to fix my PC as soon as I opened IE and I'm like holy [email protected], I'm getting a drive-by from King Bill. I'm not running as admin and I killed the process without clicking anywhere on the box. Lo-and-behold, 3 days later at about 3:30am I get a text message from Microsoft that somebody has logged into live.com with my legacy Skype credentials and was trying to change contact info. However I am partly to blame as my Skype account had an incredibly weak password, however when MS took control of Skype, the accounts were merged and I never looked back to change the old credentials since I was already using the new.
Yup. I wish I had a dollar for every user that lied to me when something happened on their machine. I get it, no one wants to believe they caused an issue and in IT we should probably start the conversation with something like "this happens all the time so don't worry" or "you aren't in any trouble and the issue is contained, but we need to ascertain how this occurred in order to combat it in the future...you're assistance will greatly enhance our security for future issues". Really no different than questioning a suspect, they'll deny for hours of any wrong doing, but eventually the truth will come out.
PHP
Kotlin
Intro to Discrete Math
Programming Languages
Work stuff
Then there's the lady who argues she didn't click. When knew she clicked but didn't see any proxy hits or artifacts on the machine. She had no mobile device so we were a bit puzzled. Upon investigating it turned out that "well, I thought it looked suspicious so I forwarded it to my home PC and opened it there". (insert head bang here). Lesson learned: we now monitor for phishing test emails being forwarded.
It really is the same as when people fall for a scam like the IRS ones, fake car sales or the Nigerian Prince. Lots of things like that will go unreported because they don't want to appear to be stupid. Einstein put it best, "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid". Having been on both sides of the house I can attest to how IT people come off when trying to figure out what occurred. End users are dumb, we come off in a holier than though or down right hostile manner and then wonder why we aren't given the truth.
The biggest point to make is that we learn from these things and then turn it into education for the user (along with ourselves). Users see being honest gets the whole thing tied up faster, we learn about tactics to combat it in the future (such as checking for forwarding like one commenter pointed out) and the company becomes safer as a whole.
PHP
Kotlin
Intro to Discrete Math
Programming Languages
Work stuff
@gespenstern. Thanks for the tip! I'm definitely going to check out Nirsofer's Browser History View, in case I missed something. I did check the firewall logs and yes, we do TLS Decrypt. I didn't see much but I found some URL Categories not being logged, which I'm annoyed with. Plan to deal with that tomorrow.
@Sylabicuma, I did check all incoming emails since the last password change. I found emails that were for sure phishing attempts but they were quarantined and not released by the user. I also found a few questionable ones that were delivered to the mailbox. I need to look into those more. I have not required all users to reset their passwords, but I've been monitoring logins closely, looking for risky logins. As for domain accounts being created, we have that logged, so no issues there. We don't allow local admin rights
I'm looking into a credential theft feature on Palo Alto, that will check if your users are using their AD password on any external resources. Requires a read-only domain controller but seems like a good feature.
What's interesting about this user is they NEVER use Outlook Web Access, which I was able to confirm. They only check their email on the company device. The person did tell me a portion of what their old password was, and I can tell you, it was NOT very secure. I could easily say that's it, your pw wasn't secure, but I doubt anyone could guess the right password on the first try. I would think I would have saw some bad login attempts followed by a successful one, but that wasn't the case. During my email checks, I did see the user received an email about changing her LinkedIn password, so that could possibly be interesting.
As far as EDR, we use Sophos w Intercept X. They will be adding some EDR functionality soon