Whats it like working in a SOC? Boring?

CyberCop123

Just wondering what your view was?

Part of me thinks it sounds boring as you're just staring at network traffic all day everyday with little variation.

Has anyone worked in a SOC? What was your experience?

johndoee

I feel that you should be more specific. Asking what it is like working in a SOC is like saying what is it like working at the Pentagon.

Everybody has a specific job in a SOC. It's not a one size fits all. A title would give you more relevant, applicable, and specific answers.

CyberCop123

Good point!

Specifically as a threat hunter and analyst.

MalwareMike

I have worked in a SOC as a firewall guy and a threat analyst. Here's my review of the positions.

Firewall guy: a customer calls in, says he cant connect to something...you hop into the IDS (ours were Linux based) and start troubleshooting. I would mostly use tcpdump, and vi to edit firewall rules. I dreaded coming to work in this role.

Threat analyst: I would watch our alert page and review when customers would get alerts from their IDS/IPS's. Say company A had an IPS alert pop up...I would look at the attack, review the pcap (if-needed) to see if the alert was triggered by an actual attack or if it was a negative, and then tune the alert if needed. This job was cool at first but then I became a glorified alert closer. Once you get used to the customers and what their "normal" traffic is, its a lot easier to eye ball an alert and pretty much know if the alert is valid or not.

BUT...you have to start somewhere in the security world and most people start in the SOC. Full disclaimer, the SOC kind of killed my drive for security for about 4 years. I let the media and all these cool stories on the internet make me believe infosec is something its not. I left after a year and became a network engineer...but now I'm back into security full swing and will be transitioning into a security role soon...just not a SOC type role.

CyberCop123

Thanks Mike...

I've never worked in a SOC before. I'm more from an incident response and digital forensics background.

I may have an opportunity to join as a threat and hunting manager within a SOC.

Responsibilities / skills include:

- knowledge of SIEM tools (which I don't have)
- some basic malware stuff
- forensics
- knowledge of OS and Networks
- scripting knowledge
- some pentesting knowledge (I have no commercial experience but have oscp)

It's a new team and I would be overseeing 4-5 people doing threat hunting stuff.

CyberCop123

Oh and my main concerns were:

Would it be just staring at network logs all day long in a dark room day after day

Where would the job lead on to

MalwareMike

CyberCop,

That sounds a lot more interesting and challenging. Best of luck!

MalwareMike

CyberCop123 wrote: »

Oh and my main concerns were:

Would it be just staring at network logs all day long in a dark room day after day

Where would the job lead on to

Being able to read network logs is a valuable skill but it will wear on you if you do it most of the day.

mikey88

MalwareMike wrote: »

Full disclaimer, the SOC kind of killed my drive for security for about 4 years. I let the media and all these cool stories on the internet make me believe infosec is something its not. I left after a year and became a network engineer...but now I'm back into security full swing and will be transitioning into a security role soon...just not a SOC type role.

I agree. While security can be a fun and engaging role, a wrong position can drain the life out of you as well. Figure out what you like doing the most and strive to get to that goal.

LionelTeo

As someone who with 8 years+ experience working in 3 different SOC including MSSP and non-MSSP SOC, I believe I can provide a reasonable answer/overview regarding the experience of working in a SOC.

SOC work is largely dependent on the maturity of the SOC itself. For SOC that is far from being mature, a majority of the workload will be handling escalations and creation of tickets. SOC work can be so mundane and boring that someone came up with the joke defining SOC as "Sleeping On Chair".

However, very mature SOC is very challenging in nature as it requires high-end technical skills such as network, host-based, and memory forensics, reverse malware engineering. A good understanding of web applications attacks and vulnerability exploitation is also necessary. Daily work compromising analysing malware and pcap samples, extracting additional IOCs, performing exposure checks as accordingly. Secondary work will be doing threathunting to identify visibility gaps, create contents and drive projects to better engage/secure the business. Some other activities include understanding the common behaviours/characteristics across multiple malware samples and hunting/driving the content based on the behaviours that we had identified. Other scopes of work also include attempting to trace a particular red team activity that had occurred through the network to understand the visibility gap of the activity and drive the necessary countermeasures. In addition, there is an also a hotline where we handled escalations reported by users.

After reading this, I hope this can give you an idea of the different type of SOC work that you can expect. If you want to work in a high-end SOC environment, you had to work really hard and have very good analytical skills to get into one.

CyberCop123

Just looking at images of security operations centres and it's really not an appealing environment for me. Tons of screens, searching for issues, etc...

the job ive been offered is nearly double my current salary and a chance to build my own team of 4-5 people so it's a great opportunity but not one I think is quite right

[img][/img]

LionelTeo

I personally feel that SOC startup is really a good experience to go for but you should definitely have your own preference for your next career hop. SOC startup is for you to define the gold standard for the SOC you want to run, but it has lots of challenges building up the skillsets and the monitoring requirement in place. I had worked on a startup where it starts from being immature to a very mature state to high end technical hands-on work.

CyberCop123

LionelTeo wrote: »

I personally feel that SOC startup is really a good experience to go for but you should definitely have your own preference for your next career hop. SOC startup is for you to define the gold standard for the SOC you want to run, but it has lots of challenges building up the skillsets and the monitoring requirement in place. I had worked on a startup where it starts from being immature to a very mature state to high end technical hands-on work.

I don't think it's quite a startup.

They have a Global SOC in place, they have analysts I assume doing all the alerts, and responding to things as they pop up.

My role will be to set up and lead a new team of threat hunters. Looking for issues before they occur, doing some simple pen tests, looking at digital forensics if something happens, some simple/basic malware triaging. That kind of thing.

geo8

SOC work in a mature well established and managed company can be fun and a learning opportunity. But what happens if it is for the BIG 4 like PWC etc. or a managed service provide MSP? Pure hell?
Can anyone please elaborate from personal experience?

eddiezoozoo

I'm newly recruited as a SOC analyst in a Big 4 firm. At the onset it seems like a great learning curve, but it's been what a month or so already and I'm bored of closing these alerts. It just seems like god knows how long I can do this for? Especially with the messed up time schedule!! Any other career path someone can suggest in the Cyber security field?

JDMurray

So, you have made a huge accomplishment just getting your foot into the door of a SOC in a big company. You should congratulate yourself for that, because now you can start planning on how you will pivot to your next career opportunity within the same organization. It is likely that your SOC team works with other internal security teams (e.g., IR, DF, TI, VM, AppSec, CloudSec, Insider Threat, etc.) for triage/analysis/containment/mitigation/remediation of security incidents. Any of those teams is a potential pivot target for you.

Your most important thing now is to learn the SOC's policies, procedures, tools, and methods inside and out. Understand the systems, networks, data, and business processes of your company to become the best (i.e., thorough, knowledgeable, and helpful) SOC analyst on your team. This will not only impress your managers, but also impress the managers of other teams as well. Your reputation as a solver of problems and as a subject matter expert will make you very a desirable target for internal hiring and promotion. As a SOC analyst, everyone in your org is your customer, so always be smiling and helpful and glad to tackle the toughest assignments!

TechGromit

MalwareMike said:

I have worked in a SOC as a firewall guy and a threat analyst. Here's my review of the positions.

Firewall guy: a customer calls in, says he cant connect to something...you hop into the IDS (ours were Linux based) and start troubleshooting. I would mostly use tcpdump, and vi to edit firewall rules. I dreaded coming to work in this role.

You can just edit the firewall rules on the fly?  For me to make modifications to firewall rules I have to get a request by the system owner, signed off by there manager. Then corporate security must review and approve the request, then it goes to my division, Nuclear Operations Firewall group to review/approve, then I can either stage the policy, or review the staged policy and push/apply changes. I do both functions, stage or review/push, just not both on the same request.

Generally it takes 2 weeks to get any firewall changes done, but often it's 6 to 8 weeks. Emergency requests take a day or two, but have to be approved by upper management. We can look at the logs and recommend changes, but not submit the requests. 

TechGromit

CyberCop123 said:

Part of me thinks it sounds boring as you're just staring at network traffic all day everyday with little variation.

Entry level SOC jobs are going to be shift work, so low man in seniority might get grave shift with weekends. I know a co-worker who applied to work at our SOC, they selected her, but when she found out it was shift work, she turned the job offer down. .

JDMurray

TechGromit said:

I know a co-worker who applied to work at our SOC, they selected her, but when she found out it was shift work, she turned the job offer down. .

It is both the fault of the interviewer(s) and the candidate this unsuitability was not identified earlier in the hiring process. During the first-round interview, the interviewers should automatically provide specific details that are mandatory to the job, such as the need for flexibility to work multiple shifts (e.g., 1st, 2nd, 3rd), rotating shifts (e.g., occasionally one or two days on the weekend), extended shifts (e.g., overtime for incident command duties, or attending mandatory 1st shift meetings by analysts working on 2nd or 3rd shift), etc. The interview teams should never assume details beyond the normal M-F 8am-5pm standard workday are implicitly known or understood by the candidate. In addition, it is also the responsibility of the candidate to mention limitations that would prevent him/her from complying with possible mandatory rules (e.g., the candidate is never available to work on a Saturday or Sunday or both). To do so wastes the time and effort of both parties on either side of the hiring table.

E Double U

I have worked in two Security Operations Centers in my career. 

2012: My first security role was in the SOC of a telco. I was responsible for making firewall and proxy changes (Check Point, Cisco ASA, Blue Coat) plus some checking of SIEM events (mostly false positives). This lasted for less than a year, but I enjoyed it because I was learning a lot and I liked the people I worked with. Indeed this role was not the 'sexier' side of what people think when they hear cybersecurity.

2016 - 2019: My second SOC experience was within a financial institution. I joined as a tier II analyst so I had tier I analysts that looked at all of the incoming incidents and I picked up whatever they escalated to me. We started to grow very fast beyond incident response and started doing change management, hunting, threat intelligence, vulnerability management, forensics, use case building/tuning, and more. It was a fast-paced, dynamic environment. Lots of colloboration with all of the other security teams (red team, crypto, risk management, consultants). We performed purple team exercises, created attack trees, ...you name it. I even started a new network security function within the team that focus on perimeter security so anything related to DDoS, IDS, firewall, and web filters were my domain. Did lots of work with external vendors during this period including lots of travel for conferences which I really enjoyed. On top of all that technical stuff we started working Agile/Dev Ops and adopted the Scrum framework. That did not change the content, but the way of working so we started having daily standups, refinements, sprint reviews/demos, and sprint planning. Technologies included Splunk, SIEM (ArcSight, QRadar), Cisco Sourcefire NIDS, Akamai, Zscaler, Service Now, DLP (RSA, Symantec), Tufin, Azure, AWS, and more. 

Not all SOCs have the same level of responsibility or maturity. Each place is different which is determined by the needs of the organization. I have had two different SOC experiences and neither were boring. Also, I never had to do shift work. The first SOC had a follow the sun model with different locations working during EMEA and APAC hours. The second SOC had on-call for the tier III analysts. 

E Double U

TechGromit said:

MalwareMike said:

I have worked in a SOC as a firewall guy and a threat analyst. Here's my review of the positions.

Firewall guy: a customer calls in, says he cant connect to something...you hop into the IDS (ours were Linux based) and start troubleshooting. I would mostly use tcpdump, and vi to edit firewall rules. I dreaded coming to work in this role.

You can just edit the firewall rules on the fly?  For me to make modifications to firewall rules I have to get a request by the system owner, signed off by there manager. Then corporate security must review and approve the request, then it goes to my division, Nuclear Operations Firewall group to review/approve, then I can either stage the policy, or review the staged policy and push/apply changes. I do both functions, stage or review/push, just not both on the same request.

Generally it takes 2 weeks to get any firewall changes done, but often it's 6 to 8 weeks. Emergency requests take a day or two, but have to be approved by upper management. We can look at the logs and recommend changes, but not submit the requests. 

My experience is similar to yours. The first SOC I worked in had very strict procedures surrounding changes. One of my colleagues made a firewall change without an approved change request that resulted in a large outage for the customer. That analyst and our manager were both fired over that. 

I was only able to make impromptu firewall changes when I worked in an info sec team at a regional bank, but then again it had to be for resolving an issue which led to me documenting an emergency change request. 

UnixGuy

Boring is very subjective and changes with time. Just reading what @E Double U typed, I thought damn I can't see myself do this again, but it some point I did all of that and it was awesome.

Most jobs you will get learning opportunities so you can pick things and learn, the time I spent in a SOC was so valuable, makes everything else easier in comparison

E Double U

UnixGuy said:

Boring is very subjective and changes with time. Just reading what @E Double U typed, I thought damn I can't see myself do this again, but it some point I did all of that and it was awesome.

Most jobs you will get learning opportunities so you can pick things and learn, the time I spent in a SOC was so valuable, makes everything else easier in comparison

I will definitely never do it again because that would be backwards motion, but it definitely was a fun ride.

JDMurray

The structures and duties of a SOC are intertwined with the organization and people that it protects. Just as every org and org chart is different, every SOC is different . Therefore, whenever you compare SOCs also compare their parent organization to get a proper perspective on what the SOC is and does.

TechGromit

E Double U said:

My experience is similar to yours. The first SOC I worked in had very strict procedures surrounding changes. One of my colleagues made a firewall change without an approved change request that resulted in a large outage for the customer. That analyst and our manager were both fired over that. 

A few months ago I finished a project were I had a lot more latitude in making firewall changes. We were deploying  two new firewalls (one a Cisco firepower and the other an Cisco ASA - same hardware but different IOS loads) and we did most of our testing and changes in the lab environment, at least as close as we could get to a production environment. We did some fine tuning on the fly once the Firewalls were deployed in production, but we had an open work order that allowed us to make changes, until we officially completed the mod turned over to operations, the system wasn't considered "live" yet. This is the only exception to this firewall change policies. 

E Double U

TechGromit said:

E Double U said:

My experience is similar to yours. The first SOC I worked in had very strict procedures surrounding changes. One of my colleagues made a firewall change without an approved change request that resulted in a large outage for the customer. That analyst and our manager were both fired over that. 

A few months ago I finished a project were I had a lot more latitude in making firewall changes. We were deploying  two new firewalls (one a Cisco firepower and the other an Cisco ASA - same hardware but different IOS loads) and we did most of our testing and changes in the lab environment, at least as close as we could get to a production environment. We did some fine tuning on the fly once the Firewalls were deployed in production, but we had an open work order that allowed us to make changes, until we officially completed the mod turned over to operations, the system wasn't considered "live" yet. This is the only exception to this firewall change policies. 

I used to use my former employer's DR site Cisco ASA as my test environment when preparing for the CCNP Security. I would use the GUI to create my changes, but not actually push them. The CISO did not like that :-)

JDMurray

The previous posts provide an excellent example of the differences in SOCs between organizations. Where I work, a SOC is only for threat detection and working security incidents. Firewall implementation, testing, and maintenance is handled by the network engineering and operations teams only. A SOC that also did this type of work would be refereed to as an NSOC.