Passed CISSP 10/24 - disappointing as expected

ITHokie

Finished the exam in just over 3 hours (first attempt). I was on a roll, so I only took one 10 minute break. All told, I spent about 2.5 weeks of solid study. As for material:

- Infosec Institute videos: employer provided, not very comprehensive. I wouldn't really recommend them unless you are confident with the material or plan to watch them a few times.

- Conrad's 11th hour

- Transcender practice exams

- AIO/CBK for reference

I'll preface my thoughts on the relevancy of this certification by conceding that it is marketable, it is respected, it's not intended to be (primarily) a technical exam, and the sheer breadth of information and length of the exam are a challenge.

A couple of years ago, we discussed the merits of the cert here http://www.techexams.net/forums/isc-sscp-cissp/80918-read-article-lets-discuss.html

Now that I've taken (and passed) the exam, I wanted to revisit my thoughts on it. So not much has changed for me. There were probably more references to technical references than I was expecting, but even then, it seemed like they were often used as distractions rather than information relevant to answering the question. As I've heard many times, there were many questions where 2 of the answers could be eliminated right off the bat. However, the other two were often very similar or were vague or misleading - meaning I had to be careful not to misconstrue what concept the answer was trying to express.

I see why many people find it such a challenge. Unlike many tests, it's difficult to really know how you did until you see the results. I would imagine that good test takers do well with this exam, but those who have a hard time with the "pick the best of these crappy options" format will struggle. It seems to me that the root cause of the perception that this exam is so difficult is the intentionally obtuse question structure and sheer length of the exam.

The actual content did not see very difficult to me. I probably missed a handful of questions because I did not memorize the exact step of every single method/model in each domain ("What is the first thing you should after step x?"). I probably had a 50/50 chance at those. Other than that, I felt pretty solid about my answers.

So what does this mean in the context Information Security? I'll start with this snip from a the linked thread:

ITHokie wrote:

Broad theory certs should have the type of influence in the industry that Security+ has, not CISSP, which has come to be seen as some sort of gold standard for security practitioners. I realize that DoD and HR have led the charge in creating this misperception, but the result is that there a lot of folks and even companies out there in security roles that know little about the technologies behind the assets they are trying to protect.

For some roles, that is perfectly fine. For many roles, such as mine and most of the people I work with, that's not fine. The fact of the matter is that CISSP gives one an awareness of a broad range of basic security concepts. It aids in developing the terminology to communicate on these subjects. It can even help integrate disparate security concepts. But that's a very low bar for most technical positions.

As has been said before, we need good security officers, good policy types, good auditors and good management in the security arena. I'm not in any way diminishing that. But what we really need is DO-ers that can learn and analyze and protect and defend.

Here is a quick background to give some idea of where I'm coming from. My employer required me to pick up CISSP. I am in professional services for a large security vendor. My customers are all in the federal space, one of which is a branch of the US Armed Forces. Even though DoD has specified that multiple certifications (including GCIH which I already had) qualify for the top Information Assurance technical level (III), this customer is requiring CISSP in upcoming contracts because it gives them the warm fuzzies.

Fine, that's their perogative. I knocked it out. But it's scary because they believe that CISSP = skilled engineers, analysts, architects or whatever the case may be. I have seen the same nonsense in other industries. But since I'm in the business of protecting data sensitive to US interests and have seen successful attacks against US assets, it is very concerning to see this nonsense in the public sector. It has created bad incentives for everyone involved.

The battlefield is literally moving into cyberspace. It would help if the obsession with information assurance, compliance, auditing, policy etc was re-calibrated to allow for more emphasis on the threat monitoring, detecting, analyzing, incident response loop with red teaming/pen testing thrown in. These things require serious technical knowledge and skills.

ISC2 is not the problem - perception is. I think as more and more large scale attacks are successful, perception will change. I'd like to see communities like this leading the charge on changing that perception.

TeKniques

Congratulations on the pass. Nice post - I pretty much agree with what you said. I think the main issue here is that there aren't many ways to 'validate' someone as an information security expert and CISSP may be the best way to do that. We all know there's more to it then that, but it does present the question: What would be a better way to do it? I kind of balk at just saying 'experience' as that can sometimes be a nice head fake too.

Perhaps the CISSP is the best indicator, but changing the way the test works and the material presented would help? I've heard that the auditing of candidates isn't as well as it should be, but that's just anecdotal stuff. Interested in hearing some responses.

Congrats again on passing the exam!

Cyberscum

Anyone that is in the security field and has been for sometime knows that the CISSP amounts to what is basically an entry level cert for a security career (professional). It is a baseline certificate that says I know just enough info about basic security that I can operate in a security role. The gov loves to baseline everything so they made baseline security certs and a requirement for people to follow. The prob is no one has made anything better in the general security realm.

...But I have a good feeling that someone somewhere is making a great deal of money off these requirements for 8570 etc. And this person has close ties to government.

datacomboss

You can say this about most certs that don't involve hands-on demonstration like CCIE, OSCP, RHSA/RHCE or VCAP.

Cyberscum

datacomboss wrote: »

You can say this about most certs that don't involve hands-on demonstration like CCIE, OSCP, RHSA/RHCE or VCAP.

Oh now your just hurting feelings...

broli720

How does CCIE not involve a hands on demonstration of the exam topics? There is a lab portion to the exam...

broli720

And I agree that we do need doers, but I'm not convinced that will yield the results you're looking for. The war in cyberspace is being fulled by money. Being able to convince people to spend money is what we need in security. Show them where to focus their funds to yield the best return on investment. Managing a program, staffing an organization, building high performance teams; all these components is what wins the contracts, creates new markets, and allow "doers" to be told what to "do".

Life isn't a charity and companies and governments aren't out there to save the whales. We need more people with the technical foundation that can sell services and win contracts. You show them the value and cost savings and they will give you all the money you need to higher as many "doers" as you want.

CISSP is about keeping the big picture in mind when it comes to the business of security and in my opinion, it sounds like you missed that because your mindset is still stuck in weeds doing...

ITHokie

TeKniques wrote:

Congratulations on the pass. Nice post - I pretty much agree with what you said. I think the main issue here is that there aren't many ways to 'validate' someone as an information security expert and CISSP may be the best way to do that. We all know there's more to it then that, but it does present the question: What would be a better way to do it? I kind of balk at just saying 'experience' as that can sometimes be a nice head fake too.

Perhaps the CISSP is the best indicator, but changing the way the test works and the material presented would help? I've heard that the auditing of candidates isn't as well as it should be, but that's just anecdotal stuff. Interested in hearing some responses.

Congrats again on passing the exam!

Thanks! I think the way forward is to let CISSP be the entry level cert it is. It should not be tied to having expertise in any particular area of Infosec.

Cyberscum wrote:

Anyone that is in the security field and has been for sometime knows that the CISSP amounts to what is basically an entry level cert for a security career (professional). It is a baseline certificate that says I know just enough info about basic security that I can operate in a security role. The gov loves to baseline everything so they made baseline security certs and a requirement for people to follow. The prob is no one has made anything better in the general security realm.

...But I have a good feeling that someone somewhere is making a great deal of money off these requirements for 8570 etc. And this person has close ties to government.

Yeah, I totally agree on someone making a lot of money and the entry level nature of the cert. Most don’t seem to see it that way. In terms of baseline, CISSP has no business being technical level 3. IAM level 3 is fine. But the larger issue is that DoD organizations really believe that CISSP is the gold standard. Its value is wildly overestimated while other credentials have next to no value.

datacomboss wrote:

You can say this about most certs that don't involve hands-on demonstration like CCIE, OSCP, RHSA/RHCE or VCAP.

Perhaps at the most basic level, but there is no comparison in degree. Besides, this really isn’t about the lack of hands on. It’s about how basic CISSP knowledge is vs how deep it is perceived to be.

broli720 wrote:

And I agree that we do need doers, but I'm not convinced that will yield the results you're looking for. The war in cyberspace is being fulled by money. Being able to convince people to spend money is what we need in security. Show them where to focus their funds to yield the best return on investment. Managing a program, staffing an organization, building high performance teams; all these components is what wins the contracts, creates new markets, and allow "doers" to be told what to "do".

Life isn't a charity and companies and governments aren't out there to save the whales. We need more people with the technical foundation that can sell services and win contracts. You show them the value and cost savings and they will give you all the money you need to higher as many "doers" as you want.

CISSP is about keeping the big picture in mind when it comes to the business of security and in my opinion, it sounds like you missed that because your mindset is still stuck in weeds doing...

Actually, you got it backwards. Sure convincing some companies (not the Federal space where I work) to spend money is a challenge. This is why penetration testing is very valuable – shows how well security programs and controls are actually working.

But companies like Target and Home Depot and the like have the “management”, “high performance teams” (yeah, what are they doing, exactly?) and “staffing”. They were PCI compliant. Where did it get them? Target actually had a system in place that detected the attack, but they were clueless in dealing with it. How did it work out for them?

Showing ROI on security is a sketchy thing. It’s not nearly as easy as you think. It’s easy to make up some numbers, but predicting the future is not a precise endeavor. How do you think Target feels about its ROI now? Those civil suits are going to be extremely expensive.

You’re right, companies aren’t out there to save whales. But again, you’ve got it backwards. They should spending less money large security programs that can show meaningless ROI on paper and more on people that have the requisite skills to protect information systems. We need both, but it’s out of balance at the moment.

TeKniques

datacomboss wrote: »

You can say this about most certs that don't involve hands-on demonstration like CCIE, OSCP, RHSA/RHCE or VCAP.

Yes, but these are all 'technical' certifications. The CISSP is not a technical certification so it's comparing apples to oranges. What should be the hands-on demonstration for the CISSP exam? Construct a security policy? Write a business case for an information security awareness program? Write up a response to a case study?

broli720 wrote: »

And I agree that we do need doers, but I'm not convinced that will yield the results you're looking for. The war in cyberspace is being fulled by money. Being able to convince people to spend money is what we need in security. Show them where to focus their funds to yield the best return on investment. Managing a program, staffing an organization, building high performance teams; all these components is what wins the contracts, creates new markets, and allow "doers" to be told what to "do".

I really think this is a good point and accurate of what I see in my day-to-day work. Doers are definitely really important, but for the majority of them they can't make good business cases. Most just want to fly out of the gates guns blazing to do the work ... we do really need these types of professionals, but security in general does not only involve the technical aspect.

ITHokie

TeKniques wrote:

I really think this is a good point and accurate of what I see in my day-to-day work. Doers are definitely really important, but for the majority of them they can't make good business cases. Most just want to fly out of the gates guns blazing to do the work ... we do really need these types of professionals, but security in general does not only involve the technical aspect.

It's not a case of either or. It's not management or Incident Handlers, for examply. Both are needed, but as I said, the balance needs a re-calibration. It's not terribly shocking because the landscape is changing so quickly.

broli720

ITHokie wrote: »

Actually, you got it backwards. Sure convincing some companies (not the Federal space where I work) to spend money is a challenge. This is why penetration testing is very valuable – shows how well security programs and controls are actually working.

But companies like Target and Home Depot and the like have the “management”, “high performance teams” (yeah, what are they doing, exactly?) and “staffing”. They were PCI compliant. Where did it get them? Target actually had a system in place that detected the attack, but they were clueless in dealing with it. How did it work out for them?

Showing ROI on security is a sketchy thing. It’s not nearly as easy as you think. It’s easy to make up some numbers, but predicting the future is not a precise endeavor. How do you think Target feels about its ROI now? Those civil suits are going to be extremely expensive.

You’re right, companies aren’t out there to save whales. But again, you’ve got it backwards. They should spending less money large security programs that can show meaningless ROI on paper and more on people that have the requisite skills to protect information systems. We need both, but it’s out of balance at the moment.

I guess we'll have to agree to disagree. The technical people don't win contracts. The technical people won't establish the business need in our landscape. Business development and proposals win work. The technical staff hired deliver on said promises in proposals. Both are needed, but the monetary side will always drive demand in the security space and management is in charge of that. Even the decisions you make at work require management approval and that C-level exec is not an SME.

Security is a support function which only exists to drive tactical and strategic organizational goals. When more people really approach things from that standpoint, that's when you'll see the paradigm shift.

broli720

TeKniques wrote: »

Yes, but these are all 'technical' certifications. The CISSP is not a technical certification so it's comparing apples to oranges. What should be the hands-on demonstration for the CISSP exam? Construct a security policy? Write a business case for an information security awareness program? Write up a response to a case study?



I really think this is a good point and accurate of what I see in my day-to-day work. Doers are definitely really important, but for the majority of them they can't make good business cases. Most just want to fly out of the gates guns blazing to do the work ... we do really need these types of professionals, but security in general does not only involve the technical aspect.

I definitely agree with your last statement. The goal of security is not really the tools, appliances, or programs; it's about awareness. Awareness changes behavior and that change in behavior drives everything else.

Mitechniq

The intent of the CISSP certification is not meant to be too technical more like the 'PMP' of IT security, now it is true that for many years there has been an emphasis by the DOD to make this the defacto standard but as 'policies' are written and changed as the new 8140 replaces the 8570. You will see the evolution of security within the DOD and place closer attention to job specific certs and training. Compared to all other domains, cyberspace and its security are still at its infancy. It is a very exciting and sometimes frustrating time for all of us that have input in how are future in cyber will be established...

ITHokie

broli720 wrote:

I guess we'll have to agree to disagree. The technical people don't win contracts. The technical people won't establish the business need in our landscape. Business development and proposals win work. The technical staff hired deliver on said promises in proposals. Both are needed, but the monetary side will always drive demand in the security space and management is in charge of that. Even the decisions you make at work require management approval and that C-level exec is not an SME.

I have some thoughts on winning contracts (especially in the federal space), but that really is outside the scope of this discussion. If you want to start a thread on that, I'd be happy to discuss. Re-calibrating emphasis on technical skills can be accomplished under the status quo for writing proposals and competing for contracts.

broli720 wrote:

Security is a support function which only exists to drive tactical and strategic organizational goals. When more people really approach things from that standpoint, that's when you'll see the paradigm shift.

Sounds nice. This is how security is currently approached and it has failed miserably. Semantically, it's a true statement but it's not terribly useful. It doesn't translate well to organizations like DoD, FBI, CIA, etc. And there is a problem in degree, even for commercial companies. Help Desk and threat analysis are support functions. Worst case scenario for not having a competent help desk means long wait times to get your Outlook client fixed. Worst case for not having competent analysis is theft of state or trade secrets, loss of industry reputation, or even bankruptcy.

Oh, by the way, the unclassified EOP (read: White House) network was just compromised. There is a new one every day. This one appears to have been state sponsored. Question - how do you propose a non-doer focused program could have stopped or mitigated this attack?

broli720

We do have technical and administrative controls. Both are important for the successful execution of any cyber program. In the White House breach (if there was one), we don't know what was lacking. I'm just cautioning you on not losing site of that.

The doers are necessary, heck I'm one of them. But as I make my transition into management, I've found that the political climate is even more treacherous than the trenches with higher stakes and stress. Management never really produces anything, but they do assume the majority of the risk, responsibility, and provide strategic direction. Because of that, they are the ones with the final say in the decisions.

The approach I described probably is failing, but I never said it was perfect; just the realities of the era and world we live in and I don't see it changing anytime in the near future. It's our duty to adapt and evolve if we want to be successful.

Great discussion by the way and I hope no one takes offense to my contrasting views.

datacomboss

broli720 wrote: »

How does CCIE not involve a hands on demonstration of the exam topics? There is a lab portion to the exam...

I guess you don't know anything about the OCSP, RHSA/RHCE or VCAP certifications.

Those were examples of certs that DO include hands-on.

broli720

datacomboss wrote: »

I guess you don't know anything about the OCSP, RHSA/RHCE or VCAP certifications.

Those were examples of certs that DO include hands-on.

I don't think you understood what I was saying. The CCIE does include a hands on lab which is why I was asking the question. Don't really know how that relates to the other certs you listed there.

datacomboss

broli720 wrote: »

I don't think you understood what I was saying. The CCIE does include a hands on lab which is why I was asking the question. Don't really know how that relates to the other certs you listed there.

YES SIR, WE ARE IN AGREEMENT! My point is all of the above, including the CCIE have hands-on labs/demonstrations.

And my apologies to you security pros for getting the offensive security cert acronym wrong, it's "OSCP" and not "OCSP". lol

broli720

I'm still confused lol

ITHokie

broli720 wrote: »

In the White House breach (if there was one), we don't know what was lacking.

That there was a breach is not in question. We know the unclassified White House network was compromised, Russia was likely the source, and actual extent of the compromise has not been reported (if it is even known). What administrative controls do you think might have been able to prevent or detect/mitigate this attack from occurring? It's ok to speculate - think of it as a thought experiment.

broli720 wrote: »

Both are important for the successful execution of any cyber program.

I agree. We need folks who are skilled at designing, operating and maintaining both types of controls. I've been consistent in saying that both are needed. The problem is that, right now, the balance is out of whack, especially with the monitoring/detecting/analysis/response loop that gives organizations the ability to detect and respond to attacks that bypass controls.

broli720 wrote: »

The doers are necessary, heck I'm one of them. But as I make my transition into management, I've found that the political climate is even more treacherous than the trenches with higher stakes and stress. Management never really produces anything, but they do assume the majority of the risk, responsibility, and provide strategic direction. Because of that, they are the ones with the final say in the decisions.

Yes. Rather than helping management see the big security picture, political dynamics are a distraction from what is good for the organization. They cloud the issue making it even more difficult to make decision for people that have limited security knowledge to begin with. It takes some guts to put organizational needs above political wins and CYAing. I see this all the time. My current boss has that I character, but most don't. It's enlightening to watch him operate with crowd pleasers.

broli720 wrote: »

The approach I described probably is failing, but I never said it was perfect; just the realities of the era and world we live in and I don't see it changing anytime in the near future. It's our duty to adapt and evolve if we want to be successful.

I certainly would not accuse anyone of claiming that it is perfect. But "it is what it is" just isn't good enough. I challenge the notion that it's our job to adapt to a failed approach. It's our job to evangelize the risk of the status quo, recommend new ways of thinking, and demonstrate the type of competence and expertise that improve overall security posture. To me, CISSP represents the status quo.

ITHokie

Very good discussion, BTW.

JDMurray

Many people do not understand the limitations of the testing methods used for most certification exams. Far too much is often expected from testing that is only single or multiple choice selections. It is a similar situation with people who strive to achieve a black belt in a martial art and, after succeeding, are disappointed that they still can't levitate, become invisible, or kill with a touch of their finger. The disappointment is blamed on the martial art, but truly comes from not understanding the limitations of what they were setting out to achieve.

If you compare IT certifications to Bloom's Taxonomy, you see that the most basic cert exams only test the lowest level of rote Knowledge and memorization. More detailed exams will also test Comprehension of concepts for giving correct answers and in solving puzzle problems. This is where most certifications stop. A few certs test the next level, Application, where Knowledge and Comprehension are used for actual, hands on problem solving. Bloom's highest levels (Analysis, Evaluation, and Synthesis) are found in much more complex demonstrations of competency, such as research projects, Masters theses, and Ph.D. dissertations.

ITHokie

JDMurray wrote: »

Many people do not understand the limitations of the testing methods used for most certification exams.

While I understand that this may be a common problem, it certainly isn't here. I don't blame the test method. My disappointment with CISSP is primarily a function of the disparity between the knowledge required to pass the exam and the perception of knowledge required to pass the exam.

JDMurray

ITHokie wrote: »

... the disparity between the knowledge required to pass the exam and the perception of knowledge required to pass the exam.

This is true of most modern certification exams, including most certs by Cisco and Microsoft and certainly the CEH. Consider this disparity to be a cross between "cert marketing" and "what you should know for your own professional good."

ITHokie

JDMurray wrote: »

This is true of most modern certification exams, including most certs by Cisco and Microsoft and certainly the CEH. Consider this disparity to be a cross between "cert marketing" and "what you should know for your own professional good."

While it is technically true that many mainstream exams have this issue, the disparity WRT CISSP is dramatically worse. I don't think they are comparable when considering the degree. Note that I don't include dumping exams as part of this calculus.

I would agree that marketing has a lot to do with it.

JDMurray

If an educational institution, or cert vendor, uses a point system for exams, there will be a limited number of complex items that can appear on an exam. Therefore, an exam alone will never be an adequate determination of the student's understanding of all the material covered. To make a full coverage exam would mean reducing the amount of material covered by the certification, or require passing multiple exams to get a single cert (a la Microsoft).

I am remembering back to many courses I took in college where the exams really weren't as challenging as learning the material. Students were always asking, "There is so much information here? What will be on the exam?" We knew the exams were limited in length, so to cover all the course material meant you might only see one exam item per topic, and that was rarely the case. The best solution was to test on a certain percentage of the material and have projects and assignments act as supplemental testing for the rest.

IMHO, there just too much information being covered by modern, single-exam IT certifications.

philz1982

ITHokie wrote: »

Fine, that's their perogative. I knocked it out. But it's scary because they believe that CISSP = skilled engineers, analysts, architects or whatever the case may be. I have seen the same nonsense in other industries. But since I'm in the business of protecting data sensitive to US interests and have seen successful attacks against US assets, it is very concerning to see this nonsense in the public sector. It has created bad incentives for everyone involved.

The battlefield is literally moving into cyberspace. It would help if the obsession with information assurance, compliance, auditing, policy etc was re-calibrated to allow for more emphasis on the threat monitoring, detecting, analyzing, incident response loop with red teaming/pen testing thrown in. These things require serious technical knowledge and skills.

ISC2 is not the problem - perception is. I think as more and more large scale attacks are successful, perception will change. I'd like to see communities like this leading the charge on changing that perception.

My take on the last part of your post is this. Noone is going to come out of the CISSP being an "expert" on cyber security. Even the premier certs like OSCE will still leave you with gaps. The thing is Cyber is ever changing. What CISSP gives you, is a broad base of knowledge so that when you begin to research on a topic you have a fundamental understanding. Imagine if you never had any, and I mean ZERO, experience with the SDLC or CPU architecture. All you know is, C&A or maybe Audit. You go on an industrial site and they are worried about the security of their embedded compute. At least with the CISSP you will understand what a Kernel is, what a processor is, how memory works, ect. That way when you begin to study protocols, and Layer 1 through 4 security your not totally lost.

I think CISSP shows you can study, you have a broad knowledge base, and you can understand the business ramification of security. When you have 50 different controls and a budget for 3 controls how do you apply the controls, how to you classify and develop your approach? That is what the CISSP prepares you for.

-Phil

goatama

I think what a lot of people here are forgetting is that, in order to obtain the actual CISSP (not just an Associate status), you're supposed to have at least five years verifiable experience in at least two of the ten domains on the exam (or four years, with a one year waiver for having a degree or qualifying cert). This requires you to have your manager or someone sign off on your experience, and you also have to be sponsored by another CISSP who, ostensibly, is supposed to validate that you're not just a test dumper. Unlike other certifications that just require you to pass a test, this is, again, *supposed* to ensure that you actually know what you're doing in the field.

Are there ways to game the system? Absolutely. I worked with someone who had her CISSP and when I asked her to give me her local IP address had no clue how to find it. But these steps are there to at least make an effort to ensure the successful candidates are knowledgeable and competent in the realm of InfoSec. I agree that some people (especially those in the upper management realm) put way too much stock in this cert, but there's a method to the madness.

colemic

ITHokie wrote: »

Sounds nice. This is how security is currently approached and it has failed miserably.

I very much disagree. Security to most organizations is treated as a bolt-on function, whose weight slows the business down, instead of seeing it as a critical function/security system for the fast shiney sports car. There's a big, big, difference in that perspective, and truly incorporating security from the ground up, and viewing it as an integral part of the team. As broil720 said, when organizations really begin to shift that direction, you will see a remarkable change in the business landscape.

ITHokie

JDMurray wrote: »

IMHO, there just too much information being covered by modern, single-exam IT certifications.

Yes, I completely agree.

ITHokie

philz1982 wrote: »

What CISSP gives you, is a broad base of knowledge so that when you begin to research on a topic you have a fundamental understanding. Imagine if you never had any, and I mean ZERO, experience with the SDLC or CPU architecture. All you know is, C&A or maybe Audit. You go on an industrial site and they are worried about the security of their embedded compute. At least with the CISSP you will understand what a Kernel is, what a processor is, how memory works, ect. That way when you begin to study protocols, and Layer 1 through 4 security your not totally lost.

This is a low bar, and there are plenty of people out there that have this ability with or without CISSP. Again, if CISSP was perceived by the industry as an entry level baseline of security knowledge, I wouldn't find it problematic. However, the level of knowledge required to obtain CISSP is going to do little to prevent and deal with the types of attacks that I've been referring to.