Advanced: fraud models in production.

Models, model risk and the current guidance

A fraud model that works well on launch day can quietly fail six months later. Banking supervisors treat this as model risk and expect it to be managed.

The United States: SR 26-2 (April 2026). The Federal Reserve, the FDIC and the OCC issued revised Supervisory Guidance on Model Risk Management on 17 April 2026. It replaces SR 11-7, the 2011 guidance that shaped model risk management worldwide for fifteen years, and a 2021 statement on anti-money laundering models. Key points:

• It is aimed mainly at banking organisations with more than US$30 billion in assets.
• A model is a complex quantitative method that applies statistical, economic or financial theories to turn input data into quantitative estimates. Simple, deterministic rule-based processes without such theories are excluded, so a pure fraud rules engine may fall outside it.
• Model risk means the chance of financial harm from decisions made using a model's output.
• Generative and agentic AI models are outside its scope; its principles apply to traditional statistical models and to non-generative machine learning.
• It expects effective challenge by objective experts with the expertise, independence and standing to make change happen, and ongoing monitoring for changes in products, clients, data or markets.

SR 11-7 remains useful history: it set out the three core elements of validation, namely conceptual soundness, ongoing monitoring and outcomes analysis.

The United Kingdom: PRA SS1/23 (in effect since 17 May 2024). The Prudential Regulation Authority's model risk principles apply formally to banks and building societies permitted to calculate capital with their own internal models, though others may find them useful. They set five principles:

1. model identification and risk classification;
2. governance;
3. development, implementation and use;
4. independent validation;
5. model risk mitigants.

SS1/23 defines models broadly, including those using qualitative or expert-judgement inputs. It asks firms to consider explainability, transparency and bias when tiering models, and has specific expectations for dynamic models that recalibrate or change on their own.

Why fraud models drift

Fraud models face problems most other models do not.

Fraud is rare. In the academic study by Dal Pozzolo and colleagues on credit card fraud (2018), based on more than 75 million transactions, frauds were typically under 1% of transactions. This class imbalance means a model can be "99% accurate" while catching no fraud at all.

Fraud changes on purpose. Criminals adapt to controls, so the patterns a model learned stop working. The same study names concept drift, the statistical relationship between inputs and fraud changing over time, as a central problem. The Bank for International Settlements (BIS) notes that drift in input data can undermine a model and require retraining.

The truth arrives late. Whether a transaction was really fraud is often known only days or weeks later, when a customer complains or a chargeback arrives. The study calls this verification latency. A model trained on last month's labels may be learning from incomplete data.

Feedback changes the data. The model's own decisions shape what is investigated and labelled. Fraud the model never flagged may never be confirmed, so the next version can inherit its predecessor's blind spots.

What supervisors expect in practice:

• Monitoring from day one in production, as SR 11-7 put it: tracking inputs, scores, alert rates and outcomes over time.
• Triggers for action: when performance deteriorates, adjust, overlay, retrain or redevelop the model, as SR 26-2 describes.
• Benchmarking and challenger models, sensitivity testing and analysis of overrides, as listed in SS1/23.
• Documented retraining, treating a significant change as a new model version with its own validation.

How common is machine learning? In the Bank of England and FCA's 2024 survey of UK financial firms, 75% were already using AI and a further 10% planned to within three years. Anti-money laundering and combating fraud ranked among the top areas of benefit, and 46% of firms said they understood the AI they use only partially. In the 2019 survey, firms said machine learning does not necessarily create new risks but can amplify existing ones.

Measuring a fraud model honestly

Accuracy is the wrong number. With fraud under 1% of transactions, a model that says "not fraud" every time is over 99% accurate and useless.

The useful measures, using standard definitions:

• Precision: of the transactions the model flagged, how many were really fraud? It tells you how much of your investigators' time is well spent.
• Recall, or detection rate: of all the real fraud, how much did the model flag? It tells you how much fraud gets through.
• False positive rate: of the genuine transactions, how many were wrongly flagged? With very many genuine transactions, even a tiny rate means many upset customers.

A 2015 study by Saito and Rehmsmeier showed that for imbalanced data like fraud, precision-recall curves are more informative than the commonly used ROC curves, which can look good while precision is poor.

Measure what investigators can handle. Dal Pozzolo and colleagues argue that the most meaningful practical measure is alert precision: the share of fraud among the top alerts a team can actually review each day. A model is only as good as the alerts that reach a person.

Measure money as well as counts. Practitioners also track the value of fraud detected against the value missed, and the cost of reviewing false alerts and of blocking good customers. These are practice measures rather than regulatory ones, but they connect the model to the business.

Report with context. Every figure should state the period, the population, the threshold and how fraud was confirmed. A model's precision on last quarter's confirmed cases may not reflect fraud that was never discovered.

Sharing fraud data under privacy law

Criminals move between institutions; each institution sees only part of the picture. Sharing data helps, but privacy and confidentiality law has traditionally held it back.

The FATF's view. In its July 2022 report on data protection and private sector information sharing, the FATF said fighting financial crime and protecting data are not in opposition, and recommended talking to data protection authorities early, building data protection into designs, and using privacy-enhancing technologies where they suit the task. Its 2021 stocktake listed such technologies, including homomorphic encryption, zero-knowledge proofs, secure multi-party computation and federated learning, which let institutions work together without revealing the underlying private data, while noting there were not yet technical standards for them.

The United Kingdom. The Economic Crime and Corporate Transparency Act 2023, sections 188 and 189, in force since 15 January 2024, lets regulated businesses share information directly, or through an intermediary, to prevent, detect or investigate economic crime, including fraud. Such sharing does not breach a duty of confidence to the customer. It does not override data protection law, which still applies.

The European Union. Article 75 of the new Anti-Money Laundering Regulation (EU) 2024/1624, applying from 10 July 2027, allows information-sharing partnerships only where strictly necessary, after supervisors have checked the arrangements, with safeguards including pseudonymisation, records of every exchange, a data protection impact assessment, and no decisions based solely on shared information. Information produced by AI, machine learning or algorithms may only be shared if those processes "were subject to adequate human oversight".

The lesson for model teams. Shared data can make models far better, but it comes with legal conditions: a lawful basis, minimisation, records and human oversight. Design for those from the start.

Governing automated fraud decisions

A fraud model's output often becomes a decision that affects a customer: a blocked payment, a frozen account, a closed relationship.

People's rights. Under the EU GDPR, Article 22 gives people the right not to be subject to decisions based solely on automated processing that have legal or similarly significant effects, with safeguards such as human intervention, the right to express a view and the right to contest. In the UK, the Data (Use and Access) Act 2025 replaced that article with new provisions from 5 February 2026, which define a solely automated decision as one with no meaningful human involvement.

Explaining decisions. The US Consumer Financial Protection Bureau said in 2022 that lenders may not use complex algorithms if that means they cannot give specific and accurate reasons when they decline credit. It concerns credit, not fraud, but the principle that "the model is too complex to explain" is no excuse applies widely.

Supervisors' expectations. The FATF's 2021 report on new technologies says institutions must be able to "explain, and remain responsible for" the tools they use, that supervisors need to understand the models, and that manual review and human input remain very important. The BIS has noted that explainability, model risk management and data governance are areas where more guidance may be needed.

In practice:

• keep an inventory of every model and rule set, its owner, its purpose and its tier;
• have independent validation before launch and on a schedule;
• give investigators reasons they can use, not just a score;
• make sure human review of significant decisions is real, with the authority to overturn;
• give customers a route to challenge decisions, and learn from the outcomes.

In Kenya: the Data Protection Act, 2019 gives people the right not to be subject to a decision based solely on automated processing, including profiling, that produces legal effects or significantly affects them, subject to exceptions such as contract, law or consent; where such a decision is made, the person must be told and can ask for a new decision not based solely on automated processing. In a 2025 survey, the Central Bank of Kenya found half of banks had adopted AI, 40% of those using it for fraud risk management. In 2026 the Central Bank published a draft guidance note on AI in banking for comment, which treats fraud flagging and account freezing as critical decisions and expects model inventories, independent validation and human review of significant adverse decisions. It was a draft as at October 2026; check the current version.

Your certificate of completion

Enter your name as you want it to appear, then save the certificate as a PDF. In the print window, choose Save as PDF. A certificate is issued once per completion of the course.

Saolix does not record who takes this course, so it cannot verify these certificates. The certificate confirms completion of a free self-paced course and is not an accredited qualification.