What Is Red Teaming?

If you are reading this, then you have certainly heard of both red teaming and penetration testing before. You likely have a good idea of what they both are, but could you explain what the difference is? In many ways, they are quite similar, and as a result, people (salespeople and security practitioners alike) confuse the two, leading to situations such as the far too common "network penetration test sold as red teaming". The point of this module is to offer an understanding of what red teaming is, why it is important, and how a red team engagement is run.

A Crash Course in (Cyber) Risk Management

To understand what red teaming is and how it differs from penetration testing, we will first need to define both. Perhaps the best way to do that is to think in terms of goals, i.e., what are we trying to achieve with a penetration test/red team engagement? To answer that, let us first take a step back and discuss risk, as well as the broader goal of cyber risk management.

Define: Risk

The National Institute of Standards and Technology (NIST) defines:

• Vulnerability as a "weakness in an information system, system security procedures, internal controls, or implementation that could be exploited or triggered by a threat source." (NIST SP 800-53 Rev. 5, p. 423)
• Impact as "the effect on organizational [operations, assets, or individuals] of a loss of confidentiality, integrity, or availability of information or a system." (NIST SP 800-53 Rev. 5, p. 404)
• Threat as "any circumstance or event with the potential to adversely impact organizational [operations, assets, or individuals] via unauthorized access, destruction, disclosure, modification of information, and/or denial of service." (NIST SP 800-53 Rev. 5, p. 422).
• Threat actor as "an individual or a group posing a threat." (NIST SP 800-150, p. 31).
• Risk as "a measure of the extent to which an entity is threatened by a potential circumstance or event, and typically is a function of: (i) the adverse impact, or magnitude of harm, that would arise if the circumstance or event occurs; and (ii) the likelihood of occurrence." (NIST SP 800-53 Rev. 5, p. 414).

Imagine we run an online business. Our reputation would suffer if an attacker exploited a SQL injection in our web portal and stole our data. In this example:

• The vulnerability would be the SQL injection in our web portal.
• The impact would be the damage to our reputation that our data being stolen would have.
• The threat would be an attacker gaining unauthorized access to our systems.
• The threat actor would be the attacker.
• The risk would be a measure of the likelihood that the threat is realized, and the severity of the impact that would follow.

A useful mnemonic is:

Risk = Likelihood x Impact

Define: Likelihood

How exactly does one calculate the likelihood of a threat occurring? There are many factors that should be considered when doing so, and oftentimes, answers are subjective. Consider this: the SQL injection is one vulnerability that a threat actor could exploit in order to achieve their goal, but it is by no means the only way. A threat actor with enough motivation could break into our office and access the physical server containing our data. For sure, this is not as likely to happen, however it is still a possibility. To calculate likelihood, all realistically possible ways of realizing a threat should be considered, as likelihood depends on both the nature of the threat and the vulnerabilities available to exploit it. We can expand the previous mnemonic to account for this:

Risk = Likelihood x Impact
 = (Threat x Vulnerabilities) x Impact

These two mnemonics, though useful, are not meant to be taken literally (as in literal multiplication). Think of it more along the lines of risk scaling proportionally to these inputs, e.g., a target with more vulnerabilities is more likely to be exploited.

Define: Risk Management

Risk management, as defined by NIST, is "the program and supporting processes to manage risk to [operations, assets, and individuals]" (NIST SP 800-53 Rev. 5, p. 415). There are many frameworks that companies may use when establishing their own (cyber) risk management programs, but generally they can be broken down into the following steps:

1. Plan: The initial work of deciding what risk management approach the organization will take.
2. Identify: Identify cybersecurity risks that affect the organization.
3. Assess: Assess identified risks against criteria decided upon in the planning stage.
4. Respond: Plan and execute responses to the assessed risks (e.g., accept, mitigate, remediate, transfer).
5. Monitor: Monitor risks over time, and respond accordingly if triggering events occur (e.g., an employee fell victim to a phishing attack; respond by isolating their workstation).

Besides the planning stage, these steps are all meant to be done regularly, as risks posed to an organization constantly change.

Tying It All Together

With a basic understanding of cyber risk management, we can discuss how exactly penetration testing and red teaming fit in.

Define: Penetration Testing

Penetration testing is defined by NIST as "a specialized type of assessment conducted on systems or individual system components to identify vulnerabilities that could be exploited by adversaries." (NIST SP 800-53 Rev. 5, p. 93). In other words, the goal of a penetration test is to identify exploitable vulnerabilities.

If you think about penetration testing in terms of the cyber risk management steps from before:

• Identify is performed partially, as a penetration test identifies exploitable vulnerabilities, which feed the organization's identification of risks.
• Assess is performed partially, as penetration testers typically rate the severity of identified vulnerabilities using a scoring system like CVSS. An organization can then take that input into consideration when assessing risks against their own criteria.
• Respond is only touched indirectly, as penetration testers do not themselves plan or carry out risk responses. That said, the remediation recommendations they provide are typically used by organizations as inputs when performing this step.
• Monitor is untouched, as it is an ongoing organizational function that a test may inform but does not perform.

Define: Red Teaming

As defined by NIST, red team exercises "simulate attempts by adversaries to compromise mission and business functions and provide a comprehensive assessment of the security and privacy posture of systems and organizations. [...] The results from red team exercises can be used by organizations to improve security and privacy awareness and training and to assess control effectiveness." (NIST SP 800-53 Rev. 5, p. 93). In other words, the goal of red teaming is not to identify exploitable vulnerabilities, but to emulate a realistic threat actor in order to train and measure the effectiveness of the people, processes, and technology used to defend an environment.

Red teaming does not aim to perform any of the cyber risk management steps directly. Instead, it measures how effectively an organization is able to perform them:

• Respond and monitor are directly tested: when a simulated adversary moves through the environment, does the organization detect the activity, and do its planned responses hold up?
• Identify and assess are indirectly tested: if the red team succeeds along a given path, it implies the organization failed to identify or properly weigh that risk beforehand. The engagement reveals these gaps as a byproduct, rather than evaluating the organization's risk-identification process directly.

In summary, from the perspective of an organization, penetration testing answers "what could an attacker exploit?" and red teaming answers "if we were attacked, would we notice, and could we respond effectively?"

Questions

For each of the following multiple-choice questions, select the single best answer and enter its number (1, 2, 3, or 4) into the answer boxes below.

Question 1

A company has spent three years and a large budget building a SOC, deploying EDR, and hiring incident responders. Leadership now wants to know where all of this actually works against a real attacker. Which type of engagement best fits this goal, and why?

1. A red team engagement, because it measures whether their people, processes, and technology can detect and respond to a realistic attack
2. A penetration test, because it will enumerate every exploitable vulnerability in their systems
3. A red team engagement, because its primary goal is to produce a CVSS-scored list of vulnerabilities
4. Either one, since both share the identical goal of identifying exploitable vulnerabilities

Question 2

An attacker exploits a misconfigured permission in a company's cloud storage and leaks its customers' personal data, resulting in severe reputational damage. Mapping this to the NIST risk vocabulary, which statement is correct?

1. The misconfigured permission is the threat, and the attacker is the vulnerability
2. The misconfigured permission is the vulnerability, and the reputation damage is the impact
3. The reputational damage is the vulnerability, and the data leak is the impact
4. The data leak is the threat actor, and the attacker is the risk

Question 3

If an organization's primary goal is to directly identify as many exploitable vulnerabilities as possible to feed into their identify risk management process, which assessment type is most appropriate?

1. A continuous red team engagement
2. A penetration test
3. A purple team engagement
4. A zero-knowledge threat simulation