Leaving this course?

Lesson 1 of 14

Why a compliance platform is not an ordinary application

Most of what is written about running systems well applies here. Change control, monitoring, incident response, capacity planning — the Tech Ops courses in this catalogue cover those, and nothing in this course replaces them.

What this course adds is the set of properties that make a compliance system different, because they are the properties that get missed when a competent engineering team applies ordinary practice to an unusual problem.

The output is a decision about a person, and it has to be explainable long afterwards. An ordinary application is judged on whether it works now. A screening engine, a monitoring platform or a case system is judged on whether it can show what it decided two years ago, why, and on what configuration — to an auditor, a supervisor, a court, or a customer who was refused.

That single requirement cascades into most of this course:

  • The configuration is part of the answer, so the configuration has to be versioned as carefully as the code.
  • The reference data is part of the answer, so a list update is a production change even though no code moved.
  • The absence of an output is a finding. An ordinary system that produces nothing is idle; a monitoring system that produces nothing may be broken, and nobody will complain.
  • You cannot freely use production data to test, because the data is the thing you are required to protect.
  • The system is frequently bought rather than built, so what you control is configuration, and what you must still answer for is outcomes.

There is a second-order effect worth naming early, because it explains a lot of friction. Engineering teams are rewarded for making systems simpler, faster and cheaper to change, and almost every property in this course pulls the other way — more versioning, more retention, more immutability, more recorded detail. That is not a disagreement about quality. It is a genuine difference in what the system is for, and it is resolved by stating the requirement in engineering terms rather than appealing to compliance. "We must be able to reconstruct any decision for seven years" is a requirement a good engineer can design against. "Compliance needs an audit trail" is not.

One framing to carry through. In ordinary engineering the question is "does it work?". Here it is "can I show what it did, and would I be content to be judged on that record?" Those sound similar and they lead to different designs — the second one requires you to decide, in advance, what you will be able to prove.

Lesson 2 of 14

Reproducibility is the core requirement

If there is one engineering property a compliance system needs above all others, it is this: given the same input and the same point in time, you can reconstruct the same decision, and show the reasoning.

That is harder than it sounds, because a decision depends on far more than the transaction. It depends on the rule set, the thresholds, the matching configuration, the reference lists, the customer's risk rating at that moment, the reference data the enrichment used, the model version if a model was involved, and sometimes the order in which things arrived.

So the practical definition of reproducibility is: everything that can change an outcome is versioned, and every decision records which version of each of those things applied.

What that means concretely:

  • Stamp the decision, not just the record. The alert or the decision should carry the identifiers of the rule set, configuration, list snapshot and model in force when it was made. Reconstructing them later from change tickets is unreliable and will be contested.
  • Keep snapshots, not just deltas. A list of changes lets you compute a past state in theory. Under time pressure, with a supervisor waiting, it does not.
  • Treat time as data. Record both when something happened and when you learned about it. The difference matters for every question about whether you acted promptly.
  • Freeze what you cannot version. Where a dependency cannot be pinned — an external enrichment service, say — record its response verbatim at the time, because you will not be able to reproduce it later.

One cost worth being honest about: this is not free. Versioning configuration, snapshotting reference data and stamping every decision adds storage, complexity and work at every change. The reason it is worth it is asymmetric — the cost is spread over years in small amounts, and the benefit arrives all at once, on the day somebody asks a question you cannot otherwise answer. Institutions that have had that day do not need persuading; the argument is only ever difficult beforehand.

The test of whether you have this: pick a decision from eighteen months ago at random and reconstruct it, end to end, without asking the vendor. Most institutions discover they cannot, and discover it during the review rather than before it.

Lesson 3 of 14

Reference data is a production change

Compliance systems are unusual in that their behaviour changes daily without anybody deploying anything. Sanctions lists are updated, politically exposed person data refreshes, country risk tables move, internal watchlists gain entries. Every one of those alters what the system does, and almost none of them goes through the change process that governs code.

The Wolfsberg Group's guidance treats this as a discipline in its own right — list management, "the end-to-end process of determining and managing regulatory and internal lists used for screening" — and sets out what has to be decided and documented:

  • List selection. Which lists are relevant, including regulatory lists and "other lists designed to comply with regulatory requirements and to manage risk". The guidance is specific about how wide this goes: internal or private lists of parties "known to have a sanctions nexus", lists of geographic terms including cities, towns, regions and ports, banking terms such as BICs, and lists of prohibited securities and prohibited goods where applicable.
  • Sourcing. Which lists are generated internally and which come from vendors, and the processes for generating or ingesting them.
  • Maintenance. The processes for adding and removing lists or entries, with "appropriate controls to ensure lists remain up-to-date and that only appropriate individuals can add or remove lists or list entries".
  • Data enhancement. Whether certain entries should be modified or augmented.

Read that third point as an access-control requirement, because that is what it is. A system where anybody in operations can add a watchlist entry, or remove one, has an unlogged control change path — and removing an entry is the more dangerous direction.

The guidance is also realistic about selection not being a maximising exercise: screening against lists of prohibited goods "is currently unlikely to be conducted outside the context of trade finance transactions". More lists is not more compliance; it is more alerts, which is the subject of the next lesson.

Lesson 4 of 14

A tuning change is a capacity event

In most systems, capacity is driven by what customers do. In a compliance system, a large part of the load is driven by your own configuration. Loosen a matching threshold and alert volume rises; add a list and it rises again. The work arrives because somebody changed a setting, not because the world changed.

The Wolfsberg guidance makes the point directly in the context of list selection: institutions "should consider the impact that the introduction of new lists and terms, which could generate significant alert volumes, or spikes, may have on operational risk".

Note the framing. It is not that the extra alerts are inconvenient. It is that they are an operational risk, because a queue that exceeds the capacity to work it produces exactly the outcome the control exists to prevent — alerts nobody looks at, with a record showing the institution generated them.

So treat a tuning or list change as a capacity change, and run it accordingly:

  • Estimate the volume before you deploy it. Replay a period of real traffic through the proposed configuration and count the alerts. The number is usually a surprise and sometimes an order of magnitude.
  • Check the capacity to work them, not the capacity to generate them. Compute is cheap; analysts are not.
  • Stage it. A change that quintuples volume can be introduced for one segment or one list first.
  • Agree the rollback trigger in advance, in terms of queue age rather than opinion.
  • Record the decision either way. "We chose not to add this list because we could not work the volume" is a legitimate, documented risk decision. The same outcome arrived at by never asking is not.

The failure mode to watch for is the reverse of the obvious one: a tuning change made to reduce volume, because the queue was unmanageable, that is recorded as a technical optimisation rather than as the reduction in control coverage it actually is.

Lesson 5 of 14

Replay, and the trap inside it

Replay — re-running historical traffic through a changed configuration — is the most valuable testing technique available here, and the most commonly misread.

What it tells you reliably: the volume and shape of what a change would produce. How many alerts, on which segments, at what times, with what overlap against the current rules. That is exactly the capacity question above, and replay answers it well.

What it does not tell you: whether the new configuration is better. The trap is to replay a new rule set over a historical period, compare the alerts it produces against the alerts the old rules produced, and treat agreement as validation. That measures similarity to the old system, which is not the goal. Worse, it anchors on the old system's blind spots — anything the old rules never caught is not in the comparison set at all, so a change that finds a new class of activity looks like a false-positive increase.

To get beyond that you need outcomes, not alerts:

  • Compare against confirmed cases, where you know what the activity turned out to be. That set is small and biased towards what you previously detected, and it is still better than comparing alerts to alerts.
  • Look specifically at what the new configuration catches that the old one missed, and have somebody review a sample of those by hand. This is the only part of the exercise that can tell you the change is an improvement.
  • Be explicit about the unknowable. You cannot measure what neither configuration caught. Saying so in the record is more honest than a precision figure that quietly assumes the old system's alerts were the universe.

One operational caution. A replay runs against historical data as it is stored now, which may have been enriched, corrected or re-keyed since. If the data has moved, the replay is not a reconstruction of the past, and treating it as one will produce a confident wrong answer.

Lesson 6 of 14

The audit trail is a feature, not logging

Every system has logs. Very few have an evidential record, and the difference is a design decision taken early and expensively reversed.

Application logs are written for engineers, retained for weeks, rotated for cost, formatted for debugging, and freely changed between releases. An evidential record is written for somebody hostile reading it years later. It needs to be complete for the decisions it covers, retained for the regulatory period, immutable, and stable enough in structure that a query written next year still works.

What belongs in it, at the moment of the decision:

  • The inputs as received, not as later corrected.
  • The configuration and reference data in force, by version.
  • The outcome, and the path to it — which rule fired, what score, which threshold applied.
  • Who did what, and when, for every human step, including who viewed it.
  • What did not happen and why, where a control was skipped, suppressed or overridden. This is the part almost always missing, and the part most often asked about.

Two design rules that save a great deal of later pain. Write the record at decision time, from the deciding system — anything reconstructed later from other sources is weaker evidence and will be challenged on exactly that ground. And separate retention from operation: the operational store can be pruned aggressively for performance provided the evidential record is kept elsewhere for the full period, which for the FATF's Recommendation 11 is at least five years after the relationship ends, including "the results of any analysis undertaken".

The test is the one from the first lesson. Take a decision at random, produce its record, and ask whether you would be content for that record to be the whole of your defence.

Lesson 7 of 14

Data lineage, and the question nobody can answer

Ask of any alert: where did each field in this come from? In most institutions the answer takes days and involves people rather than systems, and that is the clearest sign of a lineage problem.

It matters for three practical reasons. When a decision is challenged, you need to show the input was right. When a data source turns out to have been wrong, you need to know which decisions it touched. And when a field is empty, you need to know whether it was absent at source, dropped in transit, or never mapped.

Lineage breaks in predictable places:

  • Enrichment, where a record gains data from another system and the join is not recorded. Afterwards there is no way to tell which version of the enrichment applied.
  • Format conversion, where fields are mapped into a narrower structure. The data that vanishes here vanishes silently, because every component behaves correctly.
  • Corrections, where somebody fixes a record and the system keeps only the corrected value. The decision was made on the old one.
  • Deduplication and entity resolution, where two records become one. Which of them did the alert actually rest on?

What to do about it, in descending order of value: record the identity and version of every source for each decision; keep the input as received alongside the input as used; make corrections additive rather than destructive, so the record shows both values and the time of the change; and log the join keys for enrichment, not just the result.

One connection worth making to the wider standard. The FATF's Recommendation 16 requires payment information to be structured to the extent possible. Structure is what makes lineage tractable: a field has a provenance, a free-text block does not. The compliance obligation and the engineering property are the same property seen from two sides.

Lesson 8 of 14

Testing with data you are not allowed to copy

Compliance systems need realistic data to test, and the realistic data is precisely the data you are least free to use. This tension is specific to this domain and it is where otherwise careful teams take their largest risks.

Copying production into a test environment is the default because it is easy, and it is the single most common serious data-protection exposure in financial technology. The test environment typically has weaker access control, wider access, less monitoring, no retention limit and no record of who looked. The data is the same.

The law here is not uniform, which is itself the lesson. India's Digital Personal Data Protection Act, 2023 puts the duty on the Data Fiduciary for processing done "on its behalf by a Data Processor" and makes it responsible "irrespective of any agreement to the contrary", with the Schedule pricing a failure of reasonable security safeguards at up to 250 crore rupees — so a test environment run by a supplier is squarely your problem. Japan's Act on the Protection of Personal Information requires a business handling personal information to report leaks to its Commission in situations prescribed by Commission Order, and to notify the identifiable person, which applies to a leak from a test system exactly as to one from production.

The options, with what each actually costs:

  • Masked or tokenised production data. Keeps volume and shape. Breaks anything that depends on real name characteristics, which for a screening system is most of what you wanted to test.
  • Synthetic data. Safe and fully controllable. Only contains the cases somebody thought to generate, which means it cannot find the surprise.
  • A small, curated real set under production-grade control, used only for the tests that genuinely need real names. Honest, defensible, and requires somebody to own the boundary.
  • Testing in production, deliberately, with a non-acting path for known test payloads. Powerful and dangerous, and acceptable only if the test traffic can never be mistaken for real activity.

Most mature teams end up with a combination, and the thing that makes it defensible is not the technique. It is a written statement of which tests use which kind of data, and why.

Lesson 9 of 14

You configure, you do not code

Much compliance technology is bought. That is a sensible decision and it changes the engineering problem fundamentally: what you control is configuration, and what you answer for is outcomes.

The Wolfsberg guidance states the consequence more bluntly than most vendors would like. Screening "can never detect every possible sanctions risk due to the wide range of variables in which a string of text could be altered and still convey the same meaning", and effectiveness "will vary between FIs, even where FIs are using the same third-party screening solution".

Read that twice if you have ever heard a control described as adequate because of what was purchased. Two institutions running the identical product can have materially different control effectiveness, because effectiveness lives in the configuration, the data quality and the risk assessment — all of which are yours.

What follows for anyone running these systems:

  • Configuration is your change control, not the vendor's. Every setting that affects an outcome needs an owner, an approval, a version and a record. Vendors frequently ship configuration as an administrative screen with no change history, and that gap is yours to close.
  • Understand what you cannot inspect, and write it down. If a matching algorithm or a model is proprietary, you cannot explain its internals. You can still characterise its behaviour by testing it, and you are expected to — the guidance asks institutions to document how systems are configured, so as to demonstrate the configuration "is reasonably expected to detect and manage the specific sanctions risks" they face.
  • An upgrade is a control change. A vendor release that improves matching alters your alert volume and your coverage. It needs the same before-and-after measurement as a tuning change, and "the vendor upgraded it" is not a risk record.
  • Defaults are decisions you did not make. Every shipped default is somebody else's risk appetite. Reviewing them once, on implementation, and recording the ones you changed and the ones you accepted, is a day's work that answers a recurring question for years.
Lesson 10 of 14

Failing closed, under availability pressure

A compliance control sits in the path of something the business needs to happen. When the control is slow or unavailable, there is pressure to proceed without it, and that pressure arrives with a commercial argument attached.

So decide, in advance and in writing, what each control does when it cannot run:

  • Fail closed — the activity does not proceed. Correct for anything that must not pass, such as a sanctions screen.
  • Fail open with a record — the activity proceeds and is queued for retrospective review. Defensible for some detective controls, provided the queue is genuinely worked and the volume is bounded.
  • Degrade — a reduced version of the control runs, such as exact matching when fuzzy matching is unavailable. The most useful option and the one requiring the most thought, because somebody has to decide what "reduced but still adequate" means.

Three things that make this real rather than a policy statement:

  • Test it. A configured failure mode that has never been exercised is a belief. Inject the failure and watch what the chain does.
  • Make the fallback visible. If a control is degraded, something should say so loudly, and the record of every item processed during that window should be marked. A degraded window that cannot be identified afterwards is a population you cannot remediate.
  • Put a time limit on degradation, with an escalation when it expires. Temporary arrangements survive for years precisely because nobody set an expiry.

And beware the quiet version of this failure: not an outage, but a dependency that returns an empty answer instead of an error. An enrichment service that responds with no data looks like a customer with no adverse information. Treat empty and error as distinct, and make sure the system does too — this is a specific case worth testing for, because it fails silently and in the dangerous direction.

Lesson 11 of 14

Backfills, and what you owe after a bug

Sooner or later a defect means a control did not run. A rule was disabled in a release, a feed stopped arriving, a mapping change emptied a field, a list failed to load. Nobody noticed, because the system was producing fewer alerts and nothing alerts on fewer alerts.

The response has a shape, and getting the order right matters:

  • Stop the bleeding. Fix the defect, confirm the control is running again, and verify with a test case rather than an absence of errors.
  • Bound the exposure. Determine precisely when it started and ended. This is where the versioned configuration and the evidential record earn their cost; without them the honest answer is "we don't know", which is a much worse position.
  • Quantify what was missed. Replay the affected period through the correct configuration and see what the control would have produced.
  • Work it, by risk. Replayed alerts are real alerts. A backfill of forty thousand items cannot be worked like a normal queue, so prioritise and say openly what you will not reach.
  • Answer the reporting question separately. If the replay surfaces activity that would have been reported, the duty to report has not expired because the detection was late. Under the FATF's Recommendation 20 the report goes to the financial intelligence unit promptly once you have the suspicion — and you now have it.
  • Record it as an incident with a cause, not as a data-quality ticket. A control that failed silently for a quarter is a governance event.

The hardest part is cultural rather than technical. A silent control failure is embarrassing, the queue is large, and there is always a case for scoping the backfill narrowly. The test to apply: would the scope you have chosen look like a reasonable attempt to find out what happened, or like an attempt to limit what you found? The answer is usually obvious, and it is read by somebody else later.

Lesson 12 of 14

Monitoring the monitoring

The previous lesson's defect has one root cause worth its own treatment: nothing was watching the control itself.

Ordinary system monitoring asks whether the service is up, fast and error-free. A compliance control can be all three and still be doing nothing useful. The alert count is zero because the rule was disabled; the throughput is healthy because messages pass straight through; latency is excellent because no matching ran.

So monitor the control's work, not just its health:

  • Volumes against an expected range, by rule and by segment. An alert on a rule that has stopped firing is the single highest-value monitor in this domain, and almost nobody has it.
  • Feed arrival and freshness. Every list and every enrichment source should have an expected arrival time and a maximum age, with an alert on both. A list that silently stops updating degrades the control daily and looks perfectly healthy.
  • Coverage. The proportion of in-scope activity that was actually screened or monitored, computed from an independent count of what should have been in scope. If this is derived from the same system, it will always say 100%.
  • Configuration drift. An alert when a setting changes outside the change process, which catches both error and a deliberate adjustment somebody preferred not to raise.
  • Suppression and override rates. The Wolfsberg guidance describes suppression rules and "Good Guys" lists for managing common false positives, and whitelisting rules for eliminating hits caused by frequently recurring terms. These are legitimate and necessary — and they are also the mechanism by which coverage is quietly reduced one entry at a time, so the rate of growth belongs on a dashboard somebody reads.

There is also a question of who receives these monitors, and it is usually answered badly. Volume anomalies and feed-freshness alerts are routed to a technology on-call rota, where the responder can confirm the system is healthy and close the ticket — because by every ordinary measure it is. The person who can tell whether a rule firing half as often as last week is a problem sits in compliance, and frequently has no route into the alerting at all. Give at least the coverage and volume monitors a path to somebody who understands what the control is for.

One general principle: an absence is not an event, and compliance failures are nearly always absences. Every monitor in this list exists to turn an absence into an event.

Lesson 13 of 14

Decommissioning without losing the record

Systems are retired, and the obligations attached to what they did are not. A platform switched off in 2026 may hold the only record of decisions whose retention period runs to 2032, and the moment of migration is when that record is most likely to be lost.

The retention floor is specific. The FATF's Recommendation 11 requires records of transactions, records obtained through customer due diligence (CDD, which much of the industry calls KYC), account files and business correspondence — "including the results of any analysis undertaken" — to be kept for at least five years after the business relationship ends or the occasional transaction, and national law frequently requires longer. Your retired case management system holds the results of analyses.

What goes wrong, in order of how often:

  • The export loses the reasoning. Cases migrate as records with outcomes, and the notes, attachments and decision trails do not come across. The outcome without the reasoning fails every test in this course.
  • The configuration is not kept. Decisions are migrated but the rule sets and list snapshots that produced them are not, so nothing can be reconstructed.
  • The data is kept and becomes unreadable. An archive in a proprietary format, with no application able to read it, is a compliance liability rather than a record. Prefer open formats and keep a documented schema.
  • Nobody can search it. A legal hold or a supervisory request has to be answerable in days. An archive that requires a project to query will not be.

What a defensible decommission looks like: a documented inventory of what the system held, a migration or archive that includes the reasoning and the configuration, a verified sample restored and read after the system is off, a stated retention end date, and a named owner of the archive. The verification step is the one most often skipped, and it is the only one that proves the rest worked.

Lesson 14 of 14

Proving it to somebody who was not there

Everything in this course converges on one deliverable: being able to show, to an auditor or a supervisor, that your controls did what your policy said they would.

They will not read your code and they will not accept a demonstration. What answers the question is an evidence pack, assembled before it is asked for:

  • What the control is for, in a sentence, traced to the risk assessment that requires it. The Wolfsberg guidance frames the whole programme this way: an institution should "assess its own risks in order to define the manner, extent and circumstances in which screening is employed".
  • How it is configured, by version and with dates. Institutions are expected to document configuration so as to demonstrate it is reasonably expected to detect the specific risks they face.
  • What it does not do. The guidance asks explicitly for "transparency of any system limitations or risk based decisions which the screening controls are not designed to detect". A stated limitation is a professional position; an undiscovered one is a finding.
  • Evidence it works. Test results over time, "supported by metrics, analysis and reporting", as the guidance puts it — not a one-off implementation test.
  • Change history, with who approved what and the measured effect.
  • Exceptions and their lifecycle — every suppression, whitelist entry and override, with an owner and a review date.

Two habits make this cheap rather than a project. Produce the pack annually whether or not anybody asks, because assembling it finds the gaps while you still have time to fix them. And write it for somebody hostile, since a document written to persuade a friendly reader is exactly the document that fails with an unfriendly one.

The closing thought is the one the first lesson opened with. In ordinary engineering you are judged on whether the system works. Here you are judged on the record of what it did — so the record is not documentation of the work. It is part of the work.

Knowledge check

Fifty questions

Answer all fifty questions, then check your answers. You need 45 out of 50 to pass and receive a certificate. If you score less, you will see which answers were right and wrong, and then go through the course again before you retake the check. Your answers, progress and times are kept only in this browser.

Standards and frameworks covered

The published frameworks this course teaches toward. Saolix is not affiliated with, endorsed by or accredited by any of these bodies; naming a standard is a citation, not a claim about its author.

Sources

The official documents this course relies on. Laws and guidance change, so check the current version.

  1. Wolfsberg Guidance on Sanctions Screening (2019) · The Wolfsberg Group
  2. The FATF Recommendations (Recommendations 11, 16 and 20, updated October 2025) · Financial Action Task Force
  3. The Digital Personal Data Protection Act, 2023 (No. 22 of 2023) · Ministry of Law and Justice, Government of India
  4. Act on the Protection of Personal Information (Act No. 57 of 2003), English text · Japanese Law Translation, Ministry of Justice, Japan