Adversary emulation: Measure your ability to detect attackers

DXC Labs | Security

Read more insights from DXC Labs.

Most well-known offensive security assessments today are heavily focused on pre-compromise. Organizations use “red teams” to evaluate their assets with vulnerability assessments and penetration tests, simulating a sophisticated attack as realistically as possible. It is a defensive exercise, as well as a way to get a view of the exploitable attack surface of the organization and a method to identify gaps in defense.

The tests, however, provide actionable data only on how to remediate vulnerabilities in a certain subset of assets. Consequently, the tester can spend only a limited amount of time executing post-compromise actions, such as data exfiltration and credential dumping, meaning that the exploration of the defensive capability is limited in scope, breadth and depth.

What’s more, few companies measure or assess the effectiveness of their expensive cybersecurity controls to safeguard against attackers that have already gained access. This activity is often more relevant and related to the actual goal of an attack. After all, the objective of an attacker is generally not to exploit a vulnerability or successfully social engineer a password, but rather to steal data and information or disrupt business-critical services.

Vulnerability management, penetration testing and red teaming are key exercises, but it’s critical to consider adversary emulation at both technical and behavioral levels to ensure highly effective post-compromise resilience. 

While adversary emulation is a generic term in the security community, with no standard execution method, DXC Technology recommends beginning by differentiating between pre- and post-compromise phases of attacks and focuses on what an organization wants to measure to get the best results.

Pre-compromise and post-compromise

There are many models that can be used to visualize a cyber attack. Usually these are high level, because cyberattacks differ greatly in detail. One example of such a model is the cyber kill chain. It consists of seven phases representing different steps adversaries typically take when attacking IT systems (Figure 1). The first four (recon, weaponize, deliver and exploit) are steps taken during the actual intrusion, such as gathering information, discovering vulnerabilities and building malicious payloads. The other three focus on post-exploitation activity. This includes achieving persistence on the victim host, escalating privileges, establishing a command and control (C2) channel and actions on objectives. These actions vary based on the attacker’s goals and can include data exfiltration or the disruption of services.

Figure 1: The cyber kill chain

Pre-compromise incorporates all the activity and actions the adversary takes before initially compromising an enterprise environment. Historically, companies invested in tooling that provided detection and prevention capabilities in this part of the kill chain. And this is where organizations put the most focus when it comes to offensive testing as we know it today.

Vulnerability scanning and penetration testing concentrate on finding a way to gain a foothold in the network. Sometimes findings of a penetration test are accompanied with a proof of concept to demonstrate the implications of a vulnerability. But a real cyber attack goes much deeper than this.

This is where the post-compromise phase starts. Attackers always have an objective, such as stealing sensitive information or important credentials or disrupting services. The real purpose of the attack is revealed only after the initial compromise. Today, the cybersecurity community and vendors are shifting their attention toward the latter stages of the kill chain to cover this post-compromise gap. There are several reasons. 

First, the amount of useful observable activity is considerably higher post-compromise. Adversaries spend substantially more time in the post-compromise phase compared to pre-compromise, so the chance to catch them during the post-compromise stages is much greater.

Another reason is that no matter how secure the perimeter is, it can never be guaranteed to be perfectly secure. Consider zero-day exploits, phishing, unpatched systems or insider threats. From this perspective, the value of an “assumed compromised” cyber defense posture is self-evident.

The internet is constantly being scanned for low-hanging fruit and attacked via automated scripts and tools. All of these events create a lot of white noise for security operations center (SOC) analysts and might hide actual targeted attacks. While it is seemingly obvious that it would make more sense to have SOC analysts look only for pre- and post-compromise activities and behavior in the internal network, it is easier said than done.

Once inside, adversaries will hide themselves in the noise and complexity of the enterprise they are attacking. Adversaries often prefer “living off the land,” using company-deployed system administration tools and masking their activities among normal network traffic. It is challenging to detect or prevent these kinds of activities. To help, MITRE started working in 2010 on the ATT&CK model, which includes threat tactics and techniques that have been observed from millions of attacks on enterprise networks.

ATT&CK framework

In 2013 MITRE released the ATT&CK framework, which classifies adversarial post-compromise activity. It is built on public threat intelligence and threat reports, so it is derived empirically from observations of adversary activity. MITRE’s research proves that automated measuring of endpoint data or telemetry can be used to distinguish post-compromise behavior from typical system noise. And it also provides the necessary input to begin assessing and measuring the effectiveness of deployed security controls in preventing or detecting adversarial behavior. This type of exercise is commonly called adversary simulation or emulation (Figure 2).

Figure 2. Adversary emulation in the cyber kill chain

In general, ATT&CK is divided into tactics and techniques. Tactics represent certain objectives an adversary might want to achieve, such as credential access, lateral movement or exfiltration. In total, there are 11 different tactics, which are then divided into techniques.

Techniques represent the various ways to achieve a tactic’s objectives. Examples for this would be credential dumping, lateral movement via remote desktop protocol (RDP) or exfiltration via ICMP. As of July 2019, the ATT&CK Matrix contained 244 techniques. 

In addition to classification, the ATT&CK framework provides technical information about individual techniques. This includes technical descriptions or mitigation and detection advice.

The framework also includes information about threat groups or advanced persistent threats (APTs). Each group contains a description and additional information, such as incentives or aliases. More importantly, each group contains a mapping to the techniques being used in the wild.

ATT&CK tactics and techniques can be combined and visualized in the ATT&CK matrix. This is a powerful feature of the ATT&CK matrix — visualization. The classification results in a high-level overview that can be used to give an immediate indication of techniques that had been executed in an environment, but not detected or prevented. This information is valuable for reporting, measurement and remediation.

Ideally the assessments should be complemented with adversary emulation using a tool such as MITRE Corp.’s Adversarial Tactics, Techniques and Common Knowledge (ATT&CK) model. This provides a structured approach to improving defensive measures related to red team techniques.

The objective of the first approach is to figure out whether adversary techniques are logged either on the system or in a log management solution, or whether the techniques are detected or prevented by other security solutions in the environment. Improvement and remediation can then be done on a per-technique basis. This is a basic evaluation that can be easily automated and executed repeatedly.

Today, there are good open source initiatives that can help execute single adversarial techniques in a rapid and structured manner. Caldera and Atomic Red Team are two important examples of open source solutions for executing adversary techniques in your environment.

Some commercial niche players are gaining a foothold in this market as well. The number of techniques and integration possibilities with SIEM and EDR tooling are the main differentiators.

The single technique execution approach should be deployed to build a proper logging and detection foundation, but this assessment alone should not be the end goal. To measure the full mission effectiveness of your SOC, computer incident response team (CIRT) and other cyber defense capabilities, you cannot exclusively rely on this form of execution of MITRE ATT&CK techniques.

Attack scenario evaluation

A real cyberattack incorporates multiple chained techniques; therefore, adversary emulation must simulate realistic behavior that either resembles or is inspired by attacks that have been observed in the wild. MITRE provides good examples of such attack scenarios, such as APT1.

APT1 uses 22 techniques, including pre-attack compromise of third-party infrastructure, domain registration hijacking and Dynamic DNS, as well as enterprise-level attacks employing account discovery, automated collection, credential dumping, masquerading, “pass the hash” and many more.

This scenario-based approach helps train cyber defense teams to react and respond quickly and appropriately to malicious behavior in the environment and measures the effectiveness of the SOC, its use cases and relationships with wider teams and processes.

The exercise is based on cooperation. Ideally, the offensive (red) team and the defensive (blue) team are in contact while doing the exercise. This way, the defensive team can learn and improve immediately, based on results. The red team can explain, in detail, what exactly has been executed and when. As a result, the blue team is not just confronted with an extensive due diligence list but can also improve in a practical way. This leads to co-development of the enterprise’s cyber resilience.

These approaches depend on the organization’s maturity. An organization still developing its post-compromise defense strategy might not yet be able to react to complex adversary behavior. It makes more sense to start building defenses for individual techniques and then to improve and fine-tune from there.

Adversary emulation on the technique level can result in continuous evaluation and measurement of these defense mechanisms, as most of these technique evaluations can be automated. Similar to vulnerability management, the organization will be able to identify the low-hanging fruit — techniques that should definitely be prevented or detected. 

Once an organization shows good results at the technique level, the assessment should move to a red team-blue team (purple team) approach. That is not to say that assessments on the technique level should never be done again. Techniques evolve over time and, like vulnerability management, continuous assessment makes sense.

Measure and improve

Employing the ATT&CK framework has another benefit — measurement and reporting capabilities. In adversary emulation, together with ATT&CK, it is easy to show improvement over time. In its simplest form, a green matrix is a good sign and a red matrix is a bad sign.

Adversary emulation and MITRE ATT&CK, in general, enable offensive and defensive parts of the organization to collaborate more closely. Attacks can be translated to techniques, which can be easily translated to appropriate remediation issues for the defensive side. Thus, the offensive side is no longer the team that comes in and creates a list full of technical debt. Instead, both sides need to work together, exchange information to improve the overall defensive capabilities and improve their offensive knowledge.

The ATT&CK framework also simplifies the prioritization of actions that follow an adversary emulation. Investments in securing the environment can be tailored to specific adversary profiles and ATT&CK techniques.

A common language

The MITRE ATT&CK framework can and should be an enabler to build bridges among red, blue and IT teams through the common language and clear reference model that can be used by all teams.

A post-compromise reference model such as MITRE ATT&CK is required to assess and measure detection and response in an effective and structured manner. To withstand a realistic attack, it is important to prepare for the whole bandwidth of such an attack. This includes vulnerability management, penetration testing, red teaming and more. 

However, it is critical to enrich these with adversary emulation, a structured exercise that concentrates on post-compromise resilience. With adversary emulation, an organization can prepare for what attackers might do when they compromise assets. Adversary emulation on the technique level is comparable to vulnerability scanning, in the sense that it can be automated easily and should be done on a recurring basis. It is focused on improving the detection and prevention of individual attacker techniques.

Adversary emulation on the behavior level, on the other hand, is a cooperative exercise based on scenarios, bringing the red and blue teams together while the red team emulates various attacker behaviors. Both teams are in constant exchange during the exercise, so the blue team can effectively improve its detection and response.

MITRE ATT&CK is essentially an ecosystem for the security community, continuously releasing new developments and improvements. This framework, combined with a full range of post-compromise assessments, can help enterprises reach the next level of security.

Contact us to learn more about adversary emulation, ethical hacking and vulnerability management.