Defending during a PenTest

MitM

When you hire an external company to perform a pentest, should your staff be performing defensive measures during it?

For instance, if you notice they were able to create an account and elevate it to domain admin, should you be removing it from domain admins during the test?  My opinion is no, as it skews the results of the test. If it were a real attack and you happened to be off that day or stuck in a meeting, you wouldn't be there to remove from domain admins.

I may be wrong, but that seems more like a Red Team vs Blue Team exercise, more than a pentest

paul78

It totally depends on what you are trying to accomplish during a pentest. When we conduct a pentest, we usually will work that out with the client ahead of time. Our preference is to actually ask our clients to employ their normal counter-measures because we test to OWASP and SANS recommendations for defences.

Not all pentest firms are can test with countermeaures in place so it also depends on the company that you are using.

One other reason for not disabling or throttling back countermeasures is because most companies can't really identify if it's our analysts or a real adversary.

iBrokeIT

Entirely up your company and what you are trying to accomplish.  We get a yearly pentest.  Director and above are the only ones that know the exact date because they want to see how we will respond and test our detection capabilities.

tedjames

paul78 said:

One other reason for not disabling or throttling back countermeasures is because most companies can't really identify if it's our analysts or a real adversary.

Unless you give the client your source IPs... Then they'll know for sure that you are the ones doing the attacking.

JDMurray

Pentests are overt, scheduled, and part of security auditing. Pentests are used to improve network security by finding (possibly) exploitable vulnerabilities. The Blue Team (i.e., SOC) will be informed of the pentester's activity, and ignore the activity they see from the pentester's source IPs. You do not actually "defend" against a pentest any more than you would actively defend against a vulnerability scan.

In comparison, a Red Team's activities are private, unscheduled, and, if detected, are expected to be regarded as an actual attack (incident). The purpose of a Red Team is to make the Blue Team better. The Blue Team will actively detect and defend against Red Team activity because it is an actual, unannounced attack from an unknown source. In Red Team exercises, it is the performance of the Blue Team that is being measured rather than the security of the network. The lessons learned from the exercise will be used to improve the Blue Team.

veritas_libertas

It really depends upon what your company is trying to achieve. I've seen pentests where the targets are ignored by defenders intentionally and I've seen pentests where the defenders are expected to try and defend the targets.

MitM

JDMurray said:

Pentests are overt, scheduled, and part of security auditing. Pentests are used to improve network security by finding (possibly) exploitable vulnerabilities. The Blue Team (i.e., SOC) will be informed of the pentester's activity, and ignore the activity they see from the pentester's source IPs. You do not actually "defend" against a pentest any more than you would actively defend against a vulnerability scan.

In comparison, a Red Team's activities are private, unscheduled, and, if detected, are expected to be regarded as an actual attack (incident). The purpose of a Red Team is to make the Blue Team better. The Blue Team will actively detect and defend against Red Team activity because it is an actual, unannounced attack from an unknown source. In Red Team exercises, it is the performance of the Blue Team that is being measured rather than the security of the network. The lessons learned from the exercise will be used to improve the Blue Team.

This is exactly how I am viewing it.

I appreciate ALL the replies, very insightful. 

paul78

tedjames said:

paul78 said:

One other reason for not disabling or throttling back countermeasures is because most companies can't really identify if it's our analysts or a real adversary.

Unless you give the client your source IPs... Then they'll know for sure that you are the ones doing the attacking.

We haven't found that to be effective. Because once we gain a foothold, our targets typically can't tell it's us.

To be honest, it really depends on the scope of the engagement - a lot of pen testing scopes these days are routine to the point of being useless. The routine testing still serve a purpose but if that's all an organization does - it's usually just a checkbox that they are seeking. Some of our clients only want those types of pentest so we do it. We usually try to encourage them to increase scope - heck - we don't even increase the fee. Usually what ends up happening is that we end up as their secondary pen test company.

iBrokeIT

paul78 said:

We haven't found that to be effective. Because once we gain a foothold, our targets typically can't tell it's us.

To be honest, it really depends on the scope of the engagement - a lot of pen testing scopes these days are routine to the point of being useless. The routine testing still serve a purpose but if that's all an organization does - it's usually just a checkbox that they are seeking. Some of our clients only want those types of pentest so we do it. We usually try to encourage them to increase scope - heck - we don't even increase the fee. Usually what ends up happening is that we end up as their secondary pen test company.

Lack of detection should be a critical finding.  IF you detect them, yes whitelist and allow them to continue.  Unless you have the budget for both red teaming AND a pentest why would you not want to test and validate that your detective controls are working as part of a pentest?  What is the ratio of red team engagements vs pentests that your company performs?

JDMurray

There are many types of pentesting. Remember that auditing using pentests are a big part of compliance these days. Pentesting can be watered-down to be just another set of checkboxes on a (SOX, HIPAA, PCI-DSS, etc.) auditor's list.

paul78

iBrokeIT said:

paul78 said:

We haven't found that to be effective. Because once we gain a foothold, our targets typically can't tell it's us.

To be honest, it really depends on the scope of the engagement - a lot of pen testing scopes these days are routine to the point of being useless. The routine testing still serve a purpose but if that's all an organization does - it's usually just a checkbox that they are seeking. Some of our clients only want those types of pentest so we do it. We usually try to encourage them to increase scope - heck - we don't even increase the fee. Usually what ends up happening is that we end up as their secondary pen test company.

Lack of detection should be a critical finding.  IF you detect them, yes whitelist and allow them to continue. 

I don't ask our clients to whitelist us. It's just our style, but if they can detect and stop us, we just move on to other vectors.

iBrokeIT said:

Unless you have the budget for both red teaming AND a pentest why would you not want to test and validate that your detective controls are working as part of a pentest? 

It all depends on what the client is trying to accomplish. Sometimes it's just a webapp pentest in a test environment.

iBrokeIT said:

What is the ratio of red team engagements vs pentests that your company performs?

I guess it depends on your definition. Our style of pentesting and ttp's is somewhat in-between a broad coverage pentest and a red-team engagement. I don't call it red-teaming because we do remote attacks and not physical attacks. Our goal is to breach our targets so if we detect or perceive a weakness, we focus most of our efforts during the engagement on that weakness. We do a smal percentage of webapp assessments and pentests which are intended for audit purposes.

Most companies don't really like our type of pentest approach because of the high compromise rates so we end up as a secondary provider meaning that they use a pentest company to generate their required audit or regulatory obligations and then they use us for the tests that they won't disclose to a regulator or customer. The big problem with our business model is that very few companies can afford multiple pentest services. So this year, we probably will focus more on the routine pentest. We lost a few clients because they opted for pentests which they are more comfortable of the outcome.

As @jdmurray succinctly put it best - "there are many types of pentesting". They all serve very different purposes. It's all in how it's scoped.