Advanced: continuous security validation and attack surface management.

Why a once-a-year test is not enough

A penetration test is a snapshot. NIST's SP 800-115 notes that annual testing may be enough given its cost and impact, but an organisation's attack surface and the vulnerabilities in it change every day: new servers, new code, new cloud services, new weaknesses disclosed. A test that passed in January says little about the perimeter in June.

That gap has driven a shift to continuous approaches. The standards basis for "continuous" is NIST's SP 800-137 on information security continuous monitoring, which defines it as maintaining ongoing awareness of security, vulnerabilities and threats to support risk decisions, and adds that it is most effective when automated. That is monitoring rather than offensive testing, but it is the principle: keep looking, all the time.

Three related practices have grown up around this idea. They are terms popularised by industry analysts such as Gartner, not formal standards, so treat the labels loosely:

• External attack surface management (EASM): continuously discovering and monitoring everything an organisation exposes to the internet, the attacker's outside view, to find unknown or forgotten "shadow" assets and exposures.
• Breach and attack simulation (BAS): continuously and automatically running known attack techniques against an environment to test whether existing controls detect and block them.
• Continuous automated red teaming (CART): continuous, authorised offensive testing that chains weaknesses toward an objective the way a real attacker would.

Analysts group these under a broader idea of continuous threat exposure management. The point for a security team is simpler: the perimeter and the risk move, so testing and validation should be ongoing, not an annual event.

Attack surface management and adversary emulation

Know what you expose. You cannot defend what you do not know you have. External attack surface management continuously inventories internet-facing assets, domains, servers, cloud buckets, forgotten test sites, and the exposures on them. The most common serious finding is often an asset nobody remembered was there.

Test against real adversary behaviour. The common language for this is MITRE ATT&CK, a free, globally used knowledge base of the tactics and techniques that real attackers use, based on observations. Instead of asking "did we pass a test?", a team asks "which attacker techniques would we detect and stop?" and maps its testing to the ATT&CK matrix to measure coverage.

Automated adversary emulation. MITRE also publishes Caldera, an open-source platform for automated adversary emulation built on ATT&CK, used for red teaming, purple teaming and security validation. Tools like this run chosen techniques automatically and repeatedly, then report which were detected or blocked. Mapping the results to ATT&CK technique identifiers gives a vendor-neutral picture of where defences are strong and where they have gaps.

Red, blue and purple. A red team emulates the attacker; the blue team defends; a purple team exercise brings them together to improve detection and response. Continuous and automated tooling lets these exercises happen far more often than a once-a-year engagement.

The aim of all of this is not more alerts. It is an honest, continuously updated answer to a board-level question: against the attacks that actually happen to organisations like ours, what would we catch, and what would get through?

Prioritising what to fix

Continuous testing finds more than any team can fix at once. The skill is deciding what to fix first, and three complementary scores from the security community help, each measuring something different.

• CVSS (Common Vulnerability Scoring System), from the Forum of Incident Response and Security Teams (FIRST), captures the principal characteristics of a vulnerability as a numerical severity score, from low to critical. The current version is 4.0. Severity is not the same as the chance it will be used against you.
• EPSS (Exploit Prediction Scoring System), also from FIRST, is a data-driven machine-learning model that estimates the probability a published vulnerability will be exploited in the wild in the next 30 days, as a number from 0 to 1, updated daily. It adds real-world likelihood to CVSS's severity.
• The Known Exploited Vulnerabilities (KEV) catalogue, from the US Cybersecurity and Infrastructure Security Agency (CISA), lists vulnerabilities known to be exploited in the wild. CISA advises using it as an input to prioritisation. A landmark 2021 US directive (Binding Operational Directive 22-01) required federal agencies to fix KEV-listed vulnerabilities within set deadlines; it established KEV-driven prioritisation and has since been superseded.

Use them together. A weakness that is severe (high CVSS), likely to be exploited soon (high EPSS) and already being exploited (on the KEV list) is an emergency; a severe one that is never exploited can often wait. Do not treat a single score as the whole answer.

Validate, then fix, then retest. A finding from any tool must be confirmed, because scanners and AI tools alike produce false positives. NIST's distinction holds: a scan shows a weakness might exist; confirming it, fixing it, and retesting that the fix worked is the full loop.

Where this is required

Continuous or regular testing is not just good practice; in several regimes it is required.

• PCI DSS, the card industry security standard, requires penetration testing to a documented, recognised methodology (it names NIST SP 800-115, OWASP and PTES as examples), both internal and external, at least once every 12 months and after significant change. Where network segmentation is used to reduce scope, the segmentation controls must be penetration-tested at least every 12 months, and for service providers at least every 6 months. Exploitable findings must be corrected and retested, and a vulnerability scan does not satisfy the penetration-testing requirement. (Confirm the exact wording against the current standard.)
• The EU Digital Operational Resilience Act (DORA) requires threat-led penetration testing for certain significant financial entities, carried out through frameworks such as the European Central Bank's TIBER-EU. The advanced course on cyber attacks on payment infrastructure covers this in detail.
• CBEST, run by the Bank of England and UK financial authorities, is the UK's equivalent framework for intelligence-led testing of important firms.

The common thread across these regimes: testing by a recognised methodology, on a defined and regular cadence, after significant change, with exploitable findings fixed and retested, and documented so it can be shown to a supervisor.

What continuous validation adds is to make "after significant change" a reality: when change is constant, so must the testing be, within the same authorisation and governance.

Governing continuous and autonomous testing

Doing more testing, more often, and increasingly with autonomous tools, raises the governance bar rather than lowering it.

Authorisation still applies to every run. A continuous or automated tool needs the same written authorisation, defined scope and rules of engagement as a one-off manual test. NIST's control catalogue requires the rules of engagement to be agreed before testing begins, and SP 800-115 asks for an explicit exclude list of systems that must not be touched and criteria for halting. The law does not change because the tool is automated: unauthorised access or interference remains an offence.

Guardrails for autonomous tools:

• Scope enforcement and halting criteria, so a continuous tool cannot drift onto systems it should not touch or cause an outage.
• Least privilege: the OWASP Top 10 for large-language-model applications names "excessive agency" as a risk; give an autonomous tool no more autonomy, permissions or access than the job needs.
• Human validation of findings before action, because false positives are inevitable.
• Protect the data: agree what may be sent to any cloud model, and remember autonomous agents can themselves be targeted or hijacked.
• Logging and auditability of what ran and what it found.

Integrate it into the programme. Continuous validation is one input to ongoing risk management, in the spirit of NIST's continuous monitoring. Feed the results, prioritised with CVSS, EPSS and the KEV catalogue, into vulnerability management and report to management with a risk-based view, not a raw list of alerts.

In Kenya: the Computer Misuse and Cybercrimes Act, 2018 makes authorisation the dividing line; section 18(3)(a) permits the relevant tools and access codes where the act is "intended for the authorised training, testing or protection of a computer system", while unauthorised access or interference is a serious offence, with higher penalties for protected systems such as banking and payments (section 20). The Critical Information Infrastructure regulations of 2024 require operators of critical infrastructure to appoint a security officer, carry out annual risk assessments, keep incident response plans and report to the national coordination bodies, with the National KE-CIRT/CC coordinating national cyber incidents. Take legal advice and have written authorisation before any testing.

Your certificate of completion

Enter your name as you want it to appear, then save the certificate as a PDF. In the print window, choose Save as PDF. A certificate is issued once per completion of the course.

Saolix does not record who takes this course, so it cannot verify these certificates. The certificate confirms completion of a free self-paced course and is not an accredited qualification.