Question for other Penetration Testers

bermovick

So, at the beginning of the year I managed to get myself into a penetration testing position.  Doing red-team work is something I'd always thought had the potential to be lots of fun.  I knew there would be quite a bit of paperwork, but still... I figured the thrill of popping a box / domain would more than make up for it  (and it does... when it happens).  Similarly for running social engineering engagements and having someone just give you their password over the phone, or following the link in a phishing email and entering credentials into a phony website I've made.

BUT

A fair amount of the time I feel kindof like a fraud I guess?  Most of the external engagements are verifying what the vulnerability scanner already detected -- SSL vulnerabilities (SSL, TLS1, Bar Mitzvah, Drown, Poodle, etc).  Or some type of information disclosure (private IPs in html responses, etc).  Blindly guessing passwords on external facing sites which has never had any success - that kind of stuff.

On the internal engagements, it feels like it's 90% opening responder & ntlmrelayx and hoping some hashes come over the wire that I can forward AND have sufficient credentials to get a foothold.  Yeah, occasionally I get something else -- a java deserialization or today when I tried to exploit Dameware v12 with shellcode I made with msfvenom (which didn't work - whether that's me or it was patched I still don't know haha),

All in all it kinda feels a bit cookie-cutter or like a 'mass produced' version of pentesting.  Maybe that's how it is nowadays?  I'm hoping other pentesters might be able to chime in and give me a better idea if I've made an unrealistic version of what this type of job can be in my head, or if there's variation in this type of work.  It feels a bit silly now that I've typed it out -- why wouldn't there be variation?  But I'm still curious the extent of that variation, I guess.

paul78

There's really nothing surprising about what you are experiencing. I run a pentest company, although I don't consider myself a real pentester since that's not my background. 

There are actually a lot of companies that provide cookie-cutter pentesting. The work you described is typical of the bigger companies that provide pentest services or security companies that provide a menu of security services. As a business, pentesting doesn't really scale well so many security companies tend to assembly line their testing.

On the flip side - most companies don't really care that much about security and just go through the motions so pentesting is just a checkbox and they either hire the cheapest company (which is always one of the bigger cookie-cutter security firms) or they find a company that they know does cookie-cutter pentesting so that they get a clean report.

In my experience, the boutique pentest companies like ours - usually less than 20-50 people in size are the ones that typically do the interesting work and custom offensive work. Of course, all the big companies have  their A teams too - but I don't know how they provide those services.

I don't want to imply just because our work tends to produce more interesting findings because of our TTPs, that we are "better" than the cookie-cutter pentest companies. Our services are just different. There is a place for cookie-cutter pentesting, it's just not what we do or what we enjoy.

The CISO's that have a mature and well-funded security program tend to practice test diversity when it comes to pentesting. So if there is cookie-cutter testing with one pentest provider, it doesn't mean that they aren't employing one or more of the other boutiques for additional pentest engagements.

tedjames

Out of curiosity, what defines a "cookie cutter" penetration test? Just running scans and a few manual tools with no real attempt to exploit? Following the same procedure for each test without deviation (a la assembly line)?

paul78

tedjames said:

Out of curiosity, what defines a "cookie cutter" penetration test? Just running scans and a few manual tools with no real attempt to exploit? Following the same procedure for each test without deviation (a la assembly line)?

I'm sure different people have different thoughts and beliefs about this topic. But for me, it's partly what you stated. Running some tools and removing false positives. Perhaps a standard way of demonstrating an exploit - for example - with XSS flaws, it's pretty straight-forward to demonstrate the vuln - same with injection flaws.

I also consider "cookie-cutter" to be testing based on minimal reconnaissance of the target. For example - scope being limited just to known networks and running nmap to find open ports and then just running metasploit, etc. Nothing necessarily wrong with the approach but it's a very minimalist approach to pentesting which could be fine for some companies.

JDMurray

Part of the cookie cutter is automation. There are just some things that are too impractical to do manually (e.g., scanning, fuzzing, password guessing, etc.). Another part is templates that form a baseline of the pentests to be performed for a specific environment. A CISO wants to know what is (and isn't) being performed for their money. And as @paul78 pointed out, more than one pentester company can be used to test an org. You don't want to hire two teams that perform 90% the same tests. You need them to be using very different cookie cutters (aka pentesting strategies).

This discussion dovetails into "threat hunting." Many people assume that threat hunting is the unstructured searching of a network for active "bad actors." You are cruising through a network in the Millenium Falcon looking for Imperial ships to destroy. It is actually a much more structured, automated, and unglamorous process of looking for residual IOA/IOC's in host file systems and their logs. And let's not forget the documentation, documentation, documentation...

shochan

COOKIEEEEEESSSSS!

https://www.youtube.com/watch?v=M60iFi5dd_E

tedjames

Excellent discussion! I agree that there are some things you want to try in every test, especially automated scanning. I've known plenty of others, mostly auditors, who just wanted to check off an item on their list stating that testing was performed. In my agency's case, we have 50+ apps, and it would be easy to go off scope, but it's important to stay on target. Otherwise, you'll never finish.

Re: using multiple teams, it's always a good idea, if you can afford it, to get more than one opinion, even if one of those opinions comes from your own organization. But also, I find it necessary to verify a finding using a different tool, just to be sure.

Re: documentation, amen to that! If you can't show proof, it doesn't exist.

paul78

tedjames said:

.... some things you want to try in every test, especially automated scanning. I've known plenty of others, mostly auditors, who just wanted to check off an item on their list stating that testing was performed....

Just a quick comment about automated scanning... There are plenty of auditors and assessors that confuse vulnerability management which is typically performed using automated scanners with penetration testing. And there are many organizations that will also pass-off a bug bounty program as penetration testing.

bigdogz

@paul78
I agree. I think there has to be some education to the difference and the process and scope to be achieved.

scasc

Just my 2 pence - you may be interested in a security researcher type of position which gives you the freedom to identity flaws through various techniques. With pen testing - as far as I’ve seen having worked and spoken to pen testers over the years - it can be a very frustrating job due to the restrictions, time scales, scope, engagement rules etc. It sounds cool from the outside but I know many testers looking to get out - for their own reasons. However there are ppl who love this too so cannot discount this. 

Most the time all I’ve seen are testers running tools and if they can prove an exploit exists they stop there. I guess it’s different for different places and countries etc.