Leaving this course?

Lesson 1 of 12

What application security is answerable for

Application security is frequently equated with testing, and testing is the smallest part of it. This lesson sets the scope, because a programme scoped as "we do penetration tests" will fail in predictable ways.

The three questions a programme has to answer. First, is the code we write free of the defect classes that are known to matter? Second, do we know what is inside the software we did not write, which is most of it? Third, when a vulnerability is published, can we find every place it exists in our estate and fix it before someone else finds it for us? Testing addresses part of the first. The Log4j event in lesson five was overwhelmingly a failure of the third.

Why "shift left" is right but incomplete. Finding defects earlier is cheaper and the argument is sound. But an exclusive focus on the development pipeline misses that most of your attack surface was written by other people, arrived as a dependency, and will need a response long after the pipeline has finished with it. The left is where you prevent your own defects. The right is where you survive everyone else's.

Your software is mostly somebody else's. A modern application is a small amount of your code sitting on a large tree of libraries, frameworks, runtimes, base images and build tools, each of which has its own dependencies. The logging library at the centre of lesson five had been incorporated, in the board's own words, "into thousands of other software packages globally" since its inception in 1999. You did not choose it, you very likely did not know you had it, and that was the whole problem.

What changed in the published consensus. The current application security category list, published for 2025, now includes software supply chain failures as its own high-ranking category rather than treating dependencies as a sub-case of using components with known vulnerabilities. That is an accurate reflection of where incidents come from, and it means a programme organised only around your own developers' output is organised around a minority of the risk.

The organisational problem underneath. Application security sits between two functions that are measured differently. Development is measured on delivery; security is measured on findings. Neither is measured on whether a published vulnerability can be located and remediated across the estate within days, which is the capability that decides outcomes. Somebody has to own that, and in most organisations nobody does — which is why the board that reviewed Log4j recommended, as separate items, that organisations "invest in capabilities to identify vulnerable systems" and "develop the capacity to maintain an accurate information technology (IT) asset and application inventory".

What this course covers in what order. The known defect classes and where they come from; memory safety as the largest single category; the Log4j event as a documented case; the inventory and bill-of-materials problem it exposed; frameworks for building better; who is responsible for the outcome; vulnerability response as a capability; and the regulation now making parts of this mandatory.

Lesson 2 of 12

The current category list

The most widely used application security category list was revised for 2025, and anyone teaching or applying the 2021 edition is working from a superseded document. The changes are substantive rather than cosmetic.

The ten categories, as published for 2025. Broken access control; security misconfiguration; software supply chain failures; cryptographic failures; injection; insecure design; authentication failures; software or data integrity failures; security logging and alerting failures; and mishandling of exceptional conditions.

What the ordering tells you. Broken access control remains first. This is the defect class where the application correctly authenticates a user and then fails to check whether that user is allowed to do the specific thing requested — reading another customer's record by changing an identifier in a request, or reaching an administrative function because the only protection was that the link was not displayed. It is first because it is ubiquitous, it is rarely caught by scanners, and it is a logic failure rather than a pattern a tool can match.

The genuinely new entry. Software supply chain failures now stands as its own category, high in the list. Previously the concern appeared as the narrower problem of using components with known vulnerabilities — that is, a patching failure. Treating it as a supply chain category widens it correctly to include the build process, the sources you pull from, the integrity of what you receive and the question of who can publish into your pipeline. Lessons five to seven are about exactly this.

The entry people will overlook. Mishandling of exceptional conditions. Error paths are where authorisation checks are skipped, where secrets land in logs, where a failure opens rather than closes, and where behaviour is least tested because it is least exercised. It is in the list because defects concentrate wherever attention does not.

How to use a list like this, and how not to. Correctly, it is a vocabulary and a coverage check: can you say, for each category, what in your programme addresses it and what evidence you have? Incorrectly, it becomes a compliance checklist that a scanner report is said to satisfy. No tool covers broken access control or insecure design, because both require knowing what the application is supposed to permit — which is a design question, not a scan.

A caution about currency. This list changes, and the previous edition was widely embedded in policies, contracts and training. If your standards cite a specific edition, they now disagree with the published consensus, and the supply chain category is the gap most likely to matter. That is a document maintenance task, not a security project, but it is the kind that never gets scheduled.

Lesson 3 of 12

Where weaknesses actually come from

Category lists describe classes of problem. A separate published list ranks the specific weaknesses behind real, catalogued vulnerabilities, and reading the two together is more informative than either alone.

What the ranking is derived from. The current list, for 2025, is drawn from the weaknesses behind 39,080 catalogued vulnerability records in that year's dataset — up from 31,770 in the previous year's list. The figures are worth noting because they tell you the denominator, which most people quoting these lists never state.

The top five, with their identifiers. Cross-site scripting (CWE-79) with a score of 60.38; SQL injection (CWE-89) at 28.72; cross-site request forgery (CWE-352) at 13.64; missing authorisation (CWE-862) at 13.28; and out-of-bounds write (CWE-787) at 12.68.

The uncomfortable observation. The two highest-scoring weaknesses are cross-site scripting and SQL injection — defect classes that have been understood, documented and teachable for more than two decades, with well-known structural fixes. Their continued dominance is not a knowledge problem. It is a problem of defaults, framework choices and review practice, and it tells you that the industry's difficulty is in consistent application rather than in understanding.

Note the gap between first and second. The top entry scores more than twice the second. These scores combine prevalence and severity, so a large gap means the weakness is both common and consequential. If you were allocating one unit of effort, the list is telling you where it goes.

Missing authorisation at number four. This is the specific, catalogued weakness underneath the broken access control category from lesson two. It is the same failure seen from the other end: the category list says access control is the biggest class; the weakness list says the specific defect appears in enough real vulnerabilities to rank fourth. When two independently produced lists agree, that is a signal worth acting on.

Out-of-bounds write, which leads to the next lesson. This is a memory safety defect, and it is the only one of the top five that is a property of the language rather than of the code's logic. The others are mistakes a developer can avoid by writing different code; this one is a class of mistake that certain languages permit and others make impossible. That distinction is the subject of lesson four.

How to use this list practically. Not as a list to memorise. Take the top five, and for each ask: which of our applications could contain this, what in our pipeline would detect it, and when did we last confirm that detection works? Three questions, five weaknesses, and the answers will not take long to produce — which is the point.

Lesson 4 of 12

Memory safety

Memory safety is the only defect class for which the published advice from national security agencies is to change the language rather than to improve the code. That is an unusually strong position and it is worth understanding precisely.

What the guidance is. A joint publication from the US cyber security agency, the national security agency and the federal investigation bureau, with international partners, urging senior executives at software manufacturers to prioritise practices that implement memory safe languages, and — specifically — to "create and publish memory safe roadmaps that detail how they will eliminate memory safety vulnerabilities in their products."

The claim at its centre. "Memory safe programming languages (MSLs) can eliminate memory safety vulnerabilities." Not reduce — eliminate, for that class. The guidance names C and C++ as "examples of memory unsafe programming languages that can lead to memory unsafe code and are still among the most widely used languages today."

Why training is explicitly not the answer. This is the most quotable part of the document and the part most likely to be resisted. The guidance acknowledges that manufacturers invest in training programmes, that commercial and university programmes exist, and that professional certificates in secure coding for those languages are available. Then it says: "While training can reduce the number of vulnerabilities a coder might introduce, given how pervasive memory safety defects are, it is almost inevitable that memory safety vulnerabilities will still occur. Even the most experienced developers write bugs that can introduce significant vulnerabilities." Its conclusion is that "training should be a bridge" while more robust technical controls are implemented.

Read that as a general principle and it applies well beyond memory safety: where a defect class is a property of the tool, training the user is mitigation, not a fix.

The honest account of the cost. The guidance does not pretend the transition is free. It notes that targeted efforts exist to make existing code bases in those languages less vulnerable, and that external analysis of one such effort "indicates that there may be non-trivial performance and memory usage costs". It discusses hardening allocators, sandboxing — noting that a subsystem handling untrustworthy data such as network communications or user-generated content "may be a good candidate to isolate" — and potential future hardware mitigations. These are real options with real trade-offs, presented as such.

What a roadmap actually requires. Not a rewrite. A documented position on which components are written in a memory unsafe language, which of those handle untrusted input, which are being migrated and on what timetable, and which are being isolated or hardened instead because migration is impractical. That is a document a board can read and a customer can ask for, which is precisely why the agencies want it published.

The question to ask a supplier. Do you publish a memory safe roadmap? It is a short question with a long answer, and the answer tells you more about a vendor's engineering maturity than any certification.

Lesson 5 of 12

The Log4j event, as reviewed

In July 2022 a statutory board published a review of the December 2021 Log4j event. It is the best-documented dependency failure available, and its findings are more surprising than the folklore.

Where the vulnerability came from. Log4j is an open source logging framework dating from 1999, maintained by a group of volunteers under a foundation project, and incorporated by developers "into thousands of other software packages globally". In July 2013 a member of the open source community requested the addition of a lookup mechanism — the feature that, eight years later, became the vulnerability. Nobody introduced a flaw; a capability was added, and the consequences of that capability were not foreseen.

That is the first lesson, and it generalises: features become vulnerabilities as the environment around them changes. A dependency is not a fixed object you assessed once.

The finding that contradicts the folklore. "At the time of writing, the Board is not aware of any significant Log4j-based attacks on critical infrastructure systems. Somewhat surprisingly, the Board also found that to date, generally speaking, exploitation of Log4j occurred at lower levels than many experts predicted, given the severity of the vulnerability."

This matters for how you communicate about vulnerabilities. The most severe dependency flaw in recent memory was, on the evidence available to a board with subpoena-adjacent access, exploited less than predicted. Severity is not incidence.

Why that conclusion was hard to reach, which is its own finding. "While cybersecurity vendors were able to provide some anecdotal evidence of exploitation, no authoritative source exists to understand exploitation trends across geographies, industries, or ecosystems. Many organizations do not even collect information on specific Log4j exploitation, and reporting is still largely voluntary." Nobody can say with authority how much exploitation occurred, which should make you sceptical of anyone who quotes a figure.

The assessment that should shape planning. "Most importantly, however, the Log4j event is not over. The Board assesses that Log4j is an 'endemic vulnerability' and that vulnerable instances of Log4j will remain in systems for many years to come, perhaps a decade or longer. Significant risk remains."

The most instructive paragraph in the report. The board observed that "many things went 'right'": the foundation had a well-established software development lifecycle with clear roles for vetting, testing and approving code; security researchers followed generally accepted coordinated disclosure steps; the foundation recognised the criticality and acted with urgency. "Yet organizations still struggled to respond to the event, and the hard work of upgrading vulnerable software is far from complete."

Good process upstream and a bad outcome downstream. The failure was not in how the vulnerability was found or disclosed; it was in consumers' inability to act on the disclosure, which is lesson six.

The structural finding about open source. The event "called attention to security risks unique to the thinly-resourced, volunteer-based open source community", which "is not adequately resourced to ensure that code is developed pursuant to industry-recognized secure coding practices and audited by experts." The board also noted such projects generally lack dedicated coordinated disclosure and response teams, and often develop "in the open", where security changes are publicly visible while being made — suggesting advisories "may need to be activated concurrently with any fixes to narrow the window for malicious exploitation prior to the official patch".

And the human cost, which the report records explicitly. The response contributed to professional burnout among defenders that "may, compounded with the generally intense pace of many cybersecurity jobs, have a long-term impact on the availability of cybersecurity talent." That is a finding about sustainability, and it is a reason to build the capability in lesson six rather than to rely on heroics.

Lesson 6 of 12

The problem that made it a crisis

A patch existed within days. The event lasted months and the board expects its tail to last a decade. The gap between those two facts is the subject of this lesson, and it is the most important thing in this course.

The actual problem. Organisations could not determine where they had Log4j. It arrived inside vendor products, inside container images, inside applications built years earlier by people who had left, bundled in archives within archives. You cannot patch what you cannot locate, and locating it was the work.

What organisations did instead, and why it was insufficient. The board records that many used short-term mitigations such as turning off affected systems or deploying web application firewall rules. Its assessment: "while these were helpful in the short term, they did not correct the underlying issues, or were superseded by later guidance that required additional resources." Some measures were more drastic, including removing systems from production, and "had direct impact on mission." Compensating controls bought time at a cost and did not end the problem.

The dependency on suppliers. "Many organizations depended on the responsiveness of their vendors to understand and remediate the vulnerability," and some "still harbor concerns, despite vendors reporting having implemented updates, due to the interdependencies between different applications and components." Your remediation timeline is your suppliers' timeline, and their assurance that they have patched is not something you can verify.

The finding that should temper every tooling conversation. "Industry experts have called for increased automation to help organizations identify vulnerable software and execute Enterprise Risk Management at scale, but they also recognize that cataloging software components at this depth can be prohibitively labor intensive. Solutions to enable these necessary capabilities do not exist."

Read that last sentence again. A statutory board, in 2022, concluded that the solutions to do this properly do not exist. Anyone selling you a complete answer is overstating, and a programme premised on buying one will not deliver.

What the board recommended instead, as two of its nineteen recommendations, under the heading of driving existing best practice for security hygiene: organisations should "invest in capabilities to identify vulnerable systems", and should "develop the capacity to maintain an accurate information technology (IT) asset and application inventory." Alongside those: a documented vulnerability response programme, a documented vulnerability disclosure and handling process, and secure software practices by developers and maintainers.

What this means for your own work, concretely. The question to answer is not "are we exposed to the current vulnerability?" It is: if a critical flaw were published this afternoon in a library we might use, how long would it take us to produce a definitive list of every place it runs? Measure that once, honestly, and you will have the most useful number in your application security programme. For most organisations the honest answer is weeks, and knowing that is the beginning of fixing it.

The partial answers that are worth having. A dependency manifest from every build, stored and searchable. An inventory of running software distinct from an inventory of purchased software. A record of which vendor products are in use and who the contact is. None of these is complete, all of them shorten the search, and together they turn a month into days.

Lesson 7 of 12

Bills of materials, honestly

Software bills of materials are the standard proposed answer to lesson six. This lesson is what they specify, and what a statutory board found when it asked whether they worked.

What the minimum specification requires. A 2021 report defines the minimum elements in three interrelated areas. Data fields: baseline information about each component, being supplier, component name, version of the component, other unique identifiers, dependency relationship, author of the bill of materials data, and timestamp. Automation support: automatic generation and machine readability, with the named data formats being SPDX, CycloneDX and SWID tags. Practices and processes: the operations of requests, generation and use, including frequency, depth, known unknowns, distribution and delivery, access control, and accommodation of mistakes.

Two items in that list deserve attention because they are unusual and honest. "Known unknowns" means the document must be able to say that a component's dependencies are not fully enumerated — an explicit acknowledgement of incompleteness. "Accommodation of mistakes" means the process must expect the data to be wrong sometimes. A specification that builds in its own fallibility is a well-designed specification.

The finding that should be quoted whenever these are discussed. The board reviewing Log4j looked at exactly this question. Bills of materials "theoretically should enable organizations to identify vulnerable software components in their environments." Then: "The Board spoke with representative groups for organizations currently using SBOMs in their environments, and none reported having leveraged them to identify vulnerable deployments of Log4j."

None. Among organisations already using them, during the event they were designed for.

Why, in the board's own diagnosis. "As designed today, SBOMs are limited, for example by variances in field descriptions and a lack of version information about catalogued components, and lack of automation on the consumption end due to these variances. Addressing SBOM standardization gaps would support a faster software supply chain vulnerability response." The board accordingly recommended improving the tooling and adoptability, and separately exploring "a baseline requirement for software transparency for federal government vendors."

What to conclude, which is not "do not bother". The specification's own framing is the right posture: "starting today is better than waiting for perfection." A bill of materials you generate and store for every build is a searchable record you did not previously have, and it shortens lesson six's search even when it is imperfect. What it is not, on the documented evidence, is a solved mechanism for answering a vulnerability question across a supplied estate.

The practical distinction to hold. Producing them for software you build is tractable and worth doing now. Consuming them from suppliers to answer a question across your estate is, as of the evidence in this lesson, not yet a working capability. Plan and budget accordingly, and be sceptical of a procurement requirement that assumes otherwise.

Lesson 8 of 12

A framework you can actually use

If lessons five to seven are about the problem, this one is about the most usable published structure for building software less likely to cause it.

What it is. A secure software development framework, published in version 1.1 in February 2022, describing itself as "a core set of high-level secure software development practices that can be integrated into each SDLC implementation." The phrase "integrated into each" is the design intent: it is deliberately not a methodology, so it can be applied to whatever process you already run.

The four practice groups. Prepare the Organization — ensure people, processes and technology are ready to develop secure software. Protect the Software — protect all components from tampering and unauthorised access. Produce Well-Secured Software — produce software with minimal security vulnerabilities in its releases. Respond to Vulnerabilities — identify residual vulnerabilities and respond appropriately.

Why that structure is better than it looks. Most secure development advice is entirely about the third group, which is where developers are. The first group is organisational readiness, the second is protecting the artefacts and the pipeline — the supply chain concern from lesson two — and the fourth is the response capability that lesson six identified as decisive. Three of the four groups cover things a code-focused programme omits entirely.

How to use it without a two-year project. Take the four groups as four questions. Are our people trained and our roles defined, and does the organisation have a stated security requirement for software? Are our source, build and release artefacts protected from tampering, and do we know who can publish into the pipeline? Do we have practices that address the weakness classes in lesson three? And can we receive, triage, fix and disclose a vulnerability report? Answering those four honestly produces a work list without a maturity assessment.

The group most often absent. Respond to Vulnerabilities, in the specific sense of being able to receive a report from outside. Many organisations have no published route for a researcher to tell them about a flaw, which means the finder's options are to give up, to publish, or to sell. This is lesson ten.

On frameworks generally, stated once. A framework gives you structure, vocabulary and a defensible account of what you do. It does not tell you which of your own systems is most exposed this afternoon. Use the framework for the programme and the measurements in lesson twelve for the priorities — and do not let the assessment exercise consume the budget that should have gone to the work.

The companion worth knowing about. The framework has a companion publication addressing generative AI and the particular development questions it raises, which matters if your organisation is now shipping software that includes a model. That is a different course, but the practice groups are the same four, which is a point in the structure's favour.

Lesson 9 of 12

Who owns the security outcome

The guidance in this lesson is unusual because its audience is not the defender. It is addressed to the people who make software, and it argues that the current distribution of responsibility is wrong.

What it is. Guidance titled "Shifting the Balance of Cybersecurity Risk: Principles and Approaches for Secure by Design Software", published initially in April 2023 and revised in October 2023, produced by the US cyber security agency with seventeen national and international partners. The number of partners matters: this is an international position rather than one country's policy.

The three principles. Take ownership of customer security outcomes. Embrace radical transparency and accountability. Lead from the top.

Read the first one as the argument it is. For decades the implicit model has been that software ships with defects and customers are responsible for compensating: patching promptly, configuring securely, buying additional products to contain the consequences. "Take ownership of customer security outcomes" rejects that allocation. It says the burden belongs with the party that can actually remove the defect, which is the manufacturer. The guidance urges manufacturers to take urgent steps to ship secure-by-design products and to permit only such products to be shipped.

Why the second principle is the sharpest in practice. Radical transparency means publishing things that are commercially uncomfortable: the memory safe roadmap from lesson four, honest advisories, accurate statements about what was affected, and vulnerability classes you have eliminated as against mitigated. It is a commitment that can be verified from outside, which is what makes it a useful procurement question rather than a slogan.

The third principle is the one that determines the other two. "Lead from the top" exists because secure development is a resourcing and incentive question, and both are set above the engineering team. A development organisation measured only on delivery dates will produce software secured to the extent that security does not affect delivery dates. No amount of process changes that, which is why the guidance addresses senior executives directly rather than security teams.

How a buyer should use this. Most readers of this course are purchasers of software far more often than makers of it, and this guidance is therefore leverage. The questions it licenses you to ask: Do you publish a memory safe roadmap? What classes of vulnerability have you eliminated by design rather than mitigated? What is your vulnerability disclosure policy and your track record of accurate advisories? Who at executive level owns security outcomes for this product?

And the honest limitation. It is guidance, not law, and a small customer asking these questions of a large supplier may be ignored. That is changing, which is lesson eleven — the arrival of regulation that makes several of these expectations into obligations with deadlines.

Lesson 10 of 12

Vulnerability response as a capability

Two capabilities decide how a published vulnerability goes for you: being able to find it, which was lesson six, and being able to receive and act on a report about your own software, which is this lesson.

Receiving reports from outside. If a researcher finds a flaw in your product or service, what do they do? If the answer is not a published policy with a route and a commitment to respond, their options reduce to giving up, publishing without warning, or selling. The framework in lesson eight makes this a practice group of its own, and the board reviewing Log4j recommended that organisations have "a documented vulnerability disclosure and handling process" as a separate recommendation from having a response programme. They are different things: one is how you learn, the other is what you then do.

The disclosure window problem, from the Log4j findings. The board noted that open source projects often develop "in the open", where security changes are publicly visible as they are being made. Its observation: advisories and community response "may need to be activated concurrently with any fixes to narrow the window for malicious exploitation prior to the official patch being available for defenders." The fix is a signal. Between the commit and the release, the information favours the attacker, and anyone watching a public repository can see it.

Prioritising what to fix, using evidence rather than severity. A published catalogue of vulnerabilities known to have been exploited is maintained on explicit criteria, and the criteria are the instructive part. A vulnerability is added when it has an assigned identifier, when "there is reliable evidence that the vulnerability has been actively exploited in the wild", and when "there is a clear remediation action for the vulnerability, such as a vendor-provided update."

The exclusions matter more than the inclusions. The catalogue's criteria state that "a vulnerability's exploitability is not considered as criteria for inclusion", and that scanning, security research on an exploit, and proof-of-concept code do not count as active exploitation. Public availability of proof-of-concept code is not required either. So the list is evidence of exploitation, not of exploitability — which makes it a prioritisation input that a severity score cannot provide. At the time of writing it held over 1,700 entries.

How to combine the two signals. Severity tells you how bad it would be; the exploited catalogue tells you whether anyone is actually doing it. A high-severity flaw with no evidence of exploitation and a low-severity flaw being actively used are not the same priority, and a programme driven by severity alone will spend its effort in the wrong order. Lesson five's finding is the same point from the other direction: the most severe dependency flaw in recent memory was exploited less than predicted.

What to put in place. A published security contact and policy. A triage path that does not depend on one person. A commitment you can meet about response times. A record of what you were told and what you did. And a decision rule for prioritisation that uses exploitation evidence alongside severity.

Lesson 11 of 12

The regulation now arriving

Most of this course has described good practice. Parts of it are becoming law for anyone selling into the European Union, and the timetable has already begun.

What the regulation is. Regulation (EU) 2024/2847 of the European Parliament and of the Council of 23 October 2024, on horizontal cybersecurity requirements for products with digital elements — known as the Cyber Resilience Act. It was published in the Official Journal on 20 November 2024.

The dates, which are the part to act on. Its recitals state that the regulation applies from 11 December 2027, with two earlier milestones: the reporting obligations for actively exploited vulnerabilities and severe incidents apply from 11 September 2026, and notification of conformity assessment bodies applies from 11 June 2026. The reporting duties therefore bite more than a year before the regulation applies generally — which is an unusual sequencing and the thing most likely to be missed.

The obligation that changes behaviour. For an actively exploited vulnerability, an early warning notification is required within 24 hours of becoming aware of it. The recitals refer explicitly to "the 24-hour deadline for the early warning notification". A day is not long enough to complete an investigation, so the practical consequence is that you must be able to notify on partial information — which means knowing in advance who decides, who files, and where.

What that requires you to have ready. A named owner for the decision, out of hours. A view of which of your products fall in scope. A route to the reporting platform and credentials that work. And a template, because drafting a regulatory notification for the first time under a 24-hour clock during an active incident is a predictable failure.

The structural point about scope. The regulation addresses products with digital elements — that is, it is manufacturer-facing legislation of the kind the guidance in lesson nine argued for. The expectations in that guidance, which were voluntary, and the obligations here, which are not, are pointed in the same direction: responsibility shifting toward the party that can remove the defect.

For readers outside the European Union. The scope follows the market rather than the manufacturer's location, so selling into it brings the obligations. More broadly, the direction of travel is the point: product security expectations are moving from guidance to obligation, with deadlines attached. An organisation that builds the vulnerability response capability in lesson ten now will find the compliance work mostly done.

One honest caveat on detail. This lesson states only what has been verified from the regulation's own text — the identifier, the adoption and publication dates, the application dates in the recitals, and the 24-hour early warning deadline. The regulation also sets out further notification stages, support periods and detailed vulnerability handling requirements in its articles and annexes. If this applies to you, read those directly or take advice; do not take a course's summary as the specification.

Lesson 12 of 12

What to measure, and what to do first

Application security programmes are easy to fill with activity and hard to evaluate. This lesson is the short list of numbers that mean something and the three things to do if the list is too long.

The measurements, each of which came out of a lesson above.

  • Time to definitive answer. If a critical flaw in a common library were published this afternoon, how long to produce a complete list of every place it runs? Lesson six's number, and the most important one here.
  • Dependency manifest coverage. What proportion of your builds produce and retain a component list that is searchable later? This is what turns the above from weeks into days.
  • Exploited-catalogue currency. Of the vulnerabilities in the known-exploited catalogue that affect your estate, how many remain unremediated, and for how long? Evidence-based, unlike a severity backlog.
  • Coverage against the weakness classes. For each of the top five weaknesses in lesson three, what detects it and when was that detection last verified?
  • Inbound report capability. Is there a published route for an outside researcher, and what was the response time on the last report received?
  • Memory safety position. Which components are in memory unsafe languages, which of those handle untrusted input, and is there a roadmap?
  • Supplier answers. For your most critical purchased software, do you know its vulnerability disclosure policy and whether the supplier publishes a memory safe roadmap?

Why not defect counts or scan findings. Because they measure how much you looked, not how exposed you are. A rising finding count can mean better tooling or worse code and you cannot tell which. Every measurement above moves only when something real changes.

The three things to do first. Produce and retain a dependency manifest for every build, because it is cheap, mostly automatic, and the single largest reduction in lesson six's number. Publish a security contact and disclosure policy, because it costs almost nothing and changes what happens when somebody finds a flaw in your software. And measure the time-to-definitive-answer honestly, once, because it is the figure that will persuade whoever controls the budget.

What this course has argued, in one paragraph. Most of your attack surface is software you did not write, acquired through a supply chain you do not control, and the decisive capability is not preventing defects but locating and remediating them after disclosure. A statutory board that reviewed the clearest example of this found that good upstream process coexisted with a bad downstream outcome, that nobody using bills of materials had used them for the purpose during the event, and that the tooling to do this properly does not exist. That is not a reason for inaction. It is a reason to build the modest, unglamorous capabilities above rather than to buy a product that claims to have solved it.

The closing caution about currency. Two of the published lists this course relies on were revised within the last year, and one of the documents it cites was withdrawn and replaced while this course was being written. Application security references go stale faster than the practices they describe. Check the edition before you cite it, and date your own standards so that the next reader knows what they are looking at.

Knowledge check

Fifty questions

Answer all fifty questions, then check your answers. You need 45 out of 50 to pass and receive a certificate. If you score less, you will see which answers were right and wrong, and then go through the course again before you retake the check. Your answers, progress and times are kept only in this browser.

Standards and frameworks covered

The published frameworks this course teaches toward. Saolix is not affiliated with, endorsed by or accredited by any of these bodies; naming a standard is a citation, not a claim about its author.

Sources

The official documents this course relies on. Laws and guidance change, so check the current version.

  1. Review of the December 2021 Log4j Event · Cyber Safety Review Board, US Department of Homeland Security
  2. OWASP Top 10:2025 · Open Worldwide Application Security Project
  3. NIST SP 800-218: Secure Software Development Framework (SSDF) Version 1.1 · US National Institute of Standards and Technology
  4. Shifting the Balance of Cybersecurity Risk: Principles and Approaches for Secure by Design Software · CISA with seventeen US and international partners
  5. The Case for Memory Safe Roadmaps · CISA, National Security Agency, FBI and international partners
  6. The Minimum Elements For a Software Bill of Materials (SBOM) · US National Telecommunications and Information Administration
  7. 2025 CWE Top 25 Most Dangerous Software Weaknesses · The MITRE Corporation, sponsored by CISA
  8. 2024 CWE Top 25 Most Dangerous Software Weaknesses · The MITRE Corporation, sponsored by CISA
  9. Known Exploited Vulnerabilities Catalog · US Cybersecurity and Infrastructure Security Agency
  10. Reducing the Significant Risk of Known Exploited Vulnerabilities · US Cybersecurity and Infrastructure Security Agency
  11. Regulation (EU) 2024/2847 on horizontal cybersecurity requirements for products with digital elements (Cyber Resilience Act) · European Parliament and Council of the European Union
  12. NIST SP 800-190: Application Container Security Guide · US National Institute of Standards and Technology
  13. NIST SP 800-61 Revision 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management · US National Institute of Standards and Technology
  14. Review of the Attacks Associated with Lapsus$ and Related Threat Groups · Cyber Safety Review Board, US Department of Homeland Security
  15. ENISA Threat Landscape 2025 · European Union Agency for Cybersecurity
  16. CIS Benchmarks · Center for Internet Security
  17. Cloud security principle 14: secure use of the service · UK National Cyber Security Centre
  18. Cloud security guidance · UK National Cyber Security Centre