Why we need a normalized scale for attacker sophistication

It has been a particularly bad quarter for cyber security; the first attack where a supply chain attack was combined with spreading ransomware was seen in the wild, affecting hundreds if not thousands of customers of the IT-management software Kaseya; The Microsoft Windows Print Spooler service was also attacked, allowing a remote authenticated attacker to execute arbitrary code with system privileges on a vulnerable system; Western Digital revealed that attackers were able to remotely wipe “My Book Live” hard drives that were connected to the internet.
This is just a small selection of incidents in this quarter. Many of the attacks are being described as “sophisticated and coordinated” or “significant and sustained” cyber attacks. That might be the case for these examples, yet, most of the time, cyber-attacks are often not successful due to high levels of sophistication on the side of the attacker but rather because of low levels of sophistication at the victims’ side.

Checkbox security

Attacks can be successful because cyber security standards are seen by organizations as merely a matter of compliance rather than a necessity. Getting hacked is the highest price to pay for checkbox security. Attacks can also be successful due to failing security principles and guidelines or insufficient security levels because of extensive cost cutting for security resources (this was for years the modus operandi at Solarwinds. This appears to have changed after the company was hit by one of the biggest hacks of the decade though). Attackers can force organizations on their knees because of insufficient business continuity plans; for example, if all of your data is encrypted, also in all of your backups, then your BCI planning has been missing out on something. Attacks can be successful because of critical cyber assets not being identified and prioritized: failure to patch systems and software on time, using outdated anti-virus software (as was the case with Equifax), incorrectly configured spam filters, bad regular access reviews, unchecked firewalls rules, weak passwords, fluffy encryption, and low levels of security awareness are a few examples.

None of this needs particular levels of sophistication. Yet failure in any of these tasks can make easy entries for attackers.
Most organizations don’t even know their level of unsophistication. That’s because they don’t have proper controls in place that would adequately monitor their levels of risk exposure. This leaves business executives in the dark about the status of ICT risks.

Incentives to “sophistication.”

So why do we hear so often about “sophisticated” attackers, and why is being successful at attacking an organization almost identical to being “sophisticated”?

One of the reasons is that there are many incentives to make it look that way. For example, companies may not be willing to admit failure because they may face severe consequences in terms of liability or insurance not willing to pay. For individuals, there may be psychological barriers to admitting failure. But usually, more important than individual psychological reasons, it’s the failure culture of an organization that defines how people deal with one of the ultimate human conditions (which failure is). If finding “responsibles” to blame failure on is the organisation’s aim, then people tend to avoid and hide failure, deny it, or blame it to a “sophisticated” attacker. Forensic analysts, too, have an interest in making attackers look more sophisticated than they usually are. The motivation here is that the organizations that help uncover a specific attack may look better. That is how marketing works. You don’t boast about an unsophisticated attacker that you helped uncover.

Journalists tend to dramatize the information they get from the first line of stakeholders, further adding to the trend that any attacker with some kind of impact on their target is “sophisticated.” The same goes in politics; pointing the finger at sophisticated attackers in foreign countries might come in handy on the world stage.

Objective scale for attacker sophistication

This leads to misleading ideas about ICT risks, wrong cost estimates, bad decisions, and fuzzy thoughts about impact and probability for specific attacks. It misguides resources and sends politics astray.
That’s why we need an objective scale for attacker sophistication. “Objective” does not mean that the scale has to be strictly scientific because much of the effects of ICT attacks can only be empirically observed. A scale for attacker sophistication would be somewhat comparable to the Beaufort wind force scale, which relates wind speed to observed conditions at sea or on land. Such observations might go something like this: “exceptionally high waves; small and medium-sized ships might be lost to view behind the waves for a long time; the sea is covered with long white patches of foam; everywhere the edges of the wave crests are blown into foam; visibility affected,” which describes the wind force of a violent storm, or wind force 11 on the Beaufort scale. Like the Beaufort scale, the scale for attacker sophistication does not have to be linear. The main thing is that we need to agree on the type of observations necessary in order to be able to classify attacker sophistication.
What can be the digital equivalent for visibility levels at sea, waveforms and height, or the amount of spray blown from cresting waves?

Adversarial Tactics, Techniques, and Common Knowledge

There is a surprisingly good model available for observations on attackers’ behavior. It was developed in 2013 by the American not-for-profit organization MITRE, which is probably best known for owning the common weakness enumeration (CWE) and common vulnerability enumeration (CVE) databases. The model is called “Adversarial Tactics, Techniques, and Common Knowledge” (short: “ATT&CK”)¹. It is a taxonomy for cyber adversary behavior, reflecting the various phases of an adversary’s attack lifecycle and the platforms they are known to target. Although it has started out of a project to categorize post-compromise adversary tactics against Microsoft Windows, it has grown more generally applicable since then; now, it does not just include Linux, macOS, and mobile devices, but more importantly, it has matured in a systematical way by merging a concrete bottom-up approach of real-world observations, with an abstract top-down analysis of the who, the what, the how, and the why of attackers.
ATT&CK has become a universal tool across many cyber security disciplines to structure and guide threat intelligence, perform testing through red teaming or adversary emulation, and measuring how effective a SOC team is at detecting, analyzing, and responding to intrusions. It is the de facto standard in threat modeling, and it is being used not just in manual assessments but also in automated tools for intrusion detection and attack simulations.

ATT&CK Scoring System

The proposal here in this article is to add functionality to ATT&CK, so that it will finally be possible to give a normalized, unified scale for “attacker sophistication”.

The way it works is comparable to the Common Vulnerability Scoring System (CVSS). CVSS is since 2005 the industry standard for severity ratings of software vulnerabilities². The scale consists of three groups of metrics: a base score, containing a rating for the attack vector, the attack complexity, the privileges required, and whether there is any user input necessary for the attack; a temporal score which looks at exploit code maturity, remediation level and report confidence; and an environmental score, which are context-dependent modifiers to the metric base group. The individual scales have a two to four-step granularity. The National Vulnerability Database provides a calculator for generating the CVSS score.³

CVSS is not perfect, but it gives pretty good guidance. For instance, a recent vulnerability which was discovered as part of the Atlassian public bug bounty program, gave attackers an easy way to gain admin rights on a Confluence server. This vulnerability was rated with a score of “9.8 — critical”. This was completely appropriate scaling, which allowed organisations to treat the vulnerability with the priority it needed. Yet not all organisations were fast enough, because the vulnerability was already being exploited massively in the wild. Many organisations using Confluence were still critically exposed, because they were not patching quickly enough.

Sophistication Calculator

It is reasonably straightforward to build a calculator for attacker sophistication: of the eleven attacker strategy categories (reconnaissance, resource development, initial access, execution, persistence, privilege escalation, defense evasion, credential access
discovery, and lateral movement
collection), all listed techniques and sub techniques would need a simple assessment of sophistication level, e.g. “not defined,” “low,” “medium,” and “high”. This will give around 200 techniques being assessed, from which a total score for attacker sophistication can easily be calculated.

Of course, attack techniques will need some weight factors. For instance, we need to address the fact that defensive evasion is a more important factor for attacker sophistication than credential access.

We also have to take time into account. This must be a factor that calculates the relative level of sophistication compared to the average level of attacker sophistication on an annual basis. This makes sure that a “6.9” on the attacker sophistication scale in 2021 will reflect the same type of sophistication as in 2022. This type of calibration factor should be set globally by an independent committee or an organization such as MITRE. This should work similar to monetary policies to control price inflation, and how inflation rates are set by central banks.

The “attacker sophistication score” will not be defined by just human actors in just a short while. AI algorithms and autonomous AI-bots will increasingly assist attackers. This leads to higher levels of sophistication because attackers will be routinely able to use techniques that are presently seen as technologically advanced.

With an objective attacker sophistication scoring system, we will have a control that allows us to monitor the state of cyber security. That leads to better decisions and will direct resources to where they are most needed.

PS: interested in building an MVP for the attacker sophistication tool? Just drop me a line.

References

(1) https://attack.mitre.org/
(2) https://www.first.org/cvss/
(3) https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator

Updates

24–09–2021: added the CVSS for the recent Atlassian vulnerability