Infosec Hunt

SaSkiller

Hi all.

I've seen a few infosec related "hunt" positions advertised (indeed my current position was advertised as such... isn't) but I'm having difficulty finding data that explains what a hunt team is, what they do, and how they do it. Anyone seen one of these in action or know of any documents that explain this? best i've seen yet is the comments section of a SANS blog post.

ITSpectre

I found this... I hope this helps you

http://itpeernetwork.intel.com/cyber-security-hunter-teams-are-the-next-advancement-in-network-defense/

the_Grinch

Hunt Teams are exactly what they sound like: teams of security professionals who prowl the network looking for compromise. The prevailing theory is that most companies are compromised and just don't know it yet. Members of the hunt team will begin looking for signs of a compromise and then go about fixing it. To rattle off the numbers (and I'm probably slightly off) a company didn't detect a compromise for an average of around 200 days. Also, when they did become aware of the compromise, it was because of a third party reaching out and telling them (be it the government, a payment processor or company infected thru them).

Thus we have to operate under the assumption that we have been compromised and now have to find them. As we find them we can plug the holes and develop policies to allow us to catch them earlier. The end game here is detection and remediation while limited the effect an attack has. A win nowadays is getting compromised and being able to say "they only got X number of Y".

SAN SEC511 is a course that covers hunt teaming extremely well and teaches you the tricks of the trade. Continuous Monitoring is the big thing with hunting. You have to know what to look for and setup a means to detect it. As an example, a systems log is suddenly cleared for an unknown reason. Now depending on your monitoring tools you might have a two scenarios:

Scenario A: you have no monitoring or weak monitoring in place thus you miss it and are compromised. If the person is good, they're doing everything low and slow (like BBQ). They're mapping your network and finding out exactly what's out there. From there they'll start establishing beach heads, areas where they can come in when needed in the event their initial point is taken out. They'll establish a persistent presence and will blend into your everyday operation. After some time, they'll find their point of exfiltration and begin getting whatever it is they're after out.

Scenario B: you have monitoring. You're team is watching and while they might miss the initial compromise, they catch the clean up from the entry point (logs are cleared, new software is installed, weird program running in the background, etc). Now since you have the monitoring in place you are able to go back through the logs and note that in the past eight months the server logs have never been cleared. Uh-oh. Thus you begin to isolate, investigate and remediate. You should be declaring an incident, informing your management team (providing updates on a regular basis) and following your incident response plan. It will probably be months before you can say "we have a clean bill of health", but the point is you limited the exposure. Maybe you catch it in the beginning and they get nothing, but even if you get it in the middle that is still a victory.

I've seen cases where a compromise was caught within the first 24 hours and the perp didn't get a chance to get what he was after out. I consider that to be pretty successful. Now if you don't learn from what lead to the infiltration you have no business remaining in your position, but if you do learn and update your processes you'll be even better prepared. Most of the people who are compromising companies have a very specific playbook and if you can get them off script at even just one step of the process it throws them off big time.

In the end, it's a big game and you need to play to win. Having a hunt team or being a member means you are looking at the overall security posture of your organization. It means that you have a comprehensive security program that deals with end users, policies, procedures and the technical aspects that make up the posture you are wanting to achieve. The process has to be treated as if it were alive, continually evolving so not to be sedentary.

Probably a lot more than you needed, but I was bored at lunch.

JDMurray

Does anyone have a good methodology for threat hunting? You can't just randomly poke around a network's syslogs and Netflows and expect to find truly suspicious activity with any great efficiency. I am wondering what others use as their threat hunting recommended procedures or best-practices.

the_Grinch

I haven't seen much in the way of a formal methodology for hunting. It seems that every team customizes it for their environment since everyone has different tools and different risks. NIST 800-137 seems to be the best place to start when it comes to establishing effective continuous monitoring, which should assist in your hunting.

YFZblu

JDMurray wrote: »

Does anyone have a good methodology for threat hunting? You can't just randomly poke around a network's syslogs and Netflows and expect to find truly suspicious activity with any great efficiency. I am wondering what others use as their threat hunting recommended procedures or best-practices.

The Rob Lee's put together a nice introductory paper which places interested parties on the correct path as it relates to what hunting is - it can be found here, in the SANS Reading Room. They use the Hunting Maturity Model (HMM) as a reference to help identify how mature an organization's hunting efforts are, and to illustrate what further maturity would look like.

The HMM framework was developed by David Bianco, who managed the Hunt Team for Mandiant at one time. More on HMM here.

the_Grinch

Needed a Masters Thesis, might design a framework for hunting!

SaSkiller

Good to see all the discussion here. As expected, there does seem to be ideas everywhere of what a hunt team does, but no consensus on how it's done.

wes allen

I think the sec511 course has some good info, as does the NSA Spotting the Adversary paper. Jessica Payne posts some good IR stuff on her twitter and is working on a MVA video, I think. Otherwise, think of it as running IR, maybe look at some of the powershell frameworks for DFIR, like Kansa - check your firewall logs for long lived sessions or other weirdness (easier said then done of most networks). Run a tool to check all the startup items, then dig into the ones at the long end of the tail.

LionelTeo

JDMurray wrote: »

Does anyone have a good methodology for threat hunting? You can't just randomly poke around a network's syslogs and Netflows and expect to find truly suspicious activity with any great efficiency. I am wondering what others use as their threat hunting recommended procedures or best-practices.

Something along the lines of irregular account usage, also download and analyze malware/exploits and understand their behavior, get an idea how it will show up in your logs and craft a search criteria for it.

JDMurray

Is there any differentiation between (pseudo) real-time hunting using Netflows and packet caps and sometime-after-the-fact event hunting using logged events? Having a SIEM look for account lockout events in syslog messages isn't what I think of as "threat hunting."

To me, "hunting" is a human looking for things happening now--either evidence of active intrusions (e.g., scanning or failed logins happening now) or remaining evidence of past intrusions (e.g., suspicious files or connections left established between hosts).

I'm wondering where people draw the line between "threat hunting" and business-as-usual security monitoring.


There are some good resources posted to this thread. I'll see what I can dig up to help the_Grinch decided on his thesis topic.

LionelTeo

The hunting that I mentioned here is to really to dig the logs yourself to check for anomalies. Ideally you would have real time rules in place that will monitor for this. If the real time rules is not in place yet, or may not be perfect to cover everything, a good hunt will eventually led to the creation/refinement of a rule.

Profiling the characteristics of malware/exploits will also help in the hunt. If a malware had a characteristics behavior change, or notice a characteristics that your currently detection won't picked up, this would result in the hunt to look for that characteristics present logs. If it does not have lots of false positives, then you can refine/add your current rules to pick up the characteristics on glass.

the_Grinch

For me it is pseudo real-time that is the actual hunting. That being said once we find it we look to generate some sort of "signature" and then run it on our past data. The adversaries that are really good might be else where so that "signature" is key to locate them on the network. We retain all of our collected Netflow and logs so I can go back to two years ago and run a query.

JDMurray wrote: »

Is there any differentiation between (pseudo) real-time hunting using Netflows and packet caps and sometime-after-the-fact event hunting using logged events? Having a SIEM look for account lockout events in syslog messages isn't what I think of as "threat hunting."

To me, "hunting" is a human looking for things happening now--either evidence of active intrusions (e.g., scanning or failed logins happening now) or remaining evidence of past intrusions (e.g., suspicious files or connections left established between hosts).

I'm wondering where people draw the line between "threat hunting" and business-as-usual security monitoring.


There are some good resources posted to this thread. I'll see what I can dig up to help the_Grinch decided on his thesis topic.

zxbane

I know it was mentioned above but I would highly recommend the SANS SEC 511 GMON course. I don't have a huge interest in this area of IA personally but this course focused on this very concept and provided a wealth of information for those interested.

wes allen

A little bit thin on content still, but a great starting point:

ThreatHunting Home

wastedtime

To me hunting is finding the anomalies in a computer network that can lead to identifying loss or potential loss of the confidentiality, availability, or integrity of them without solely relying on traditional ([NH]I[PD]S, Antivirus, vulnerability scanners, Firewall) means of detection.


Hunting is not automation, monitoring, or threat intelligence. Automated alerts is what monitoring looks at. It's also one of the things successful hunt teams produce. Monitoring is for the most part automated where hunting is not. The hunting team is trying to develop methods to find these threats so it can be automated. Threat intelligence along with high value targets help steer hunting activities.


For effective hunting you need to know your network and have access to information when needed. The best way I have seen to achieve that is to involve all areas of IT in the hunting process. Your network engineers know the network, your system engineers know the systems, etc. The more involved they are in the process the better the hunt team can function. That being said the hunt team is not comprised of engineers but is comprised of analyst. Not saying one is better then the other but it’s two completely different methods of thinking.


Hunting is a very data driven activity. Logs, intelligence data, collected files, and network traffic captures all go into information that needs to be utilized by hunting team. Knowledge of Data Science is needed to help utilize the information collected and one of the areas that has typically been a weak area in hunting.


As far as “realtime" vs “historic" hunting in most cases it’s typically over more recent data (last hour-last month) but not realtime. Realtime is a misnomer in my opinion. The only type of realtime in computer security i see is active prevention such as firewalls or IPSs that are actively denying stuff. Everything else is near-realtime or historic.


A day in the life of a hunting team is hard to pin down as their environment and information received will heavily change what they do. Think hunting as every day you have to do one or more science projects. These science projects are based on a logical hypothesis of your network or something on the network work. Then following a loose implementation of the scientific method till you have proven or disproven your hypothesis. If you prove your hypothesis it may be looked at as something that can be automated and or documented for monitoring.

UnixGuy

Ohhh so that's what cyber hunting is! I had a recruiter last week asking me if I did cyber hunting as part of my role....no I don't do it in my current role.