Leaving this course?

Lesson 1 of 6

The question has changed

For most of the history of document and media examination, the question was "has this been altered?" You started from a presumption that a photograph, a recording or a video was a capture of something that happened, and you looked for signs of tampering.

That presumption is gone. Generating a convincing image, voice or video no longer requires skill, equipment or time, which changes three things at once.

The cost asymmetry. Producing a fake is cheap and getting cheaper; examining one is expensive and getting harder. Any defence whose cost scales with the volume of fakes loses eventually. This is the same shape as spam, and it ended the same way: not by getting better at identifying bad messages one at a time, but by changing what was trusted in the first place.

The generalisation problem. Detectors learn the artefacts of the generators they were trained on. A new model, a new sampler, a different compression path, and performance falls — often sharply, and without warning, because nothing announces that the distribution has moved.

The liar's dividend. Once everyone knows convincing fakes exist, genuine evidence can be dismissed as fake. This is the under-discussed half of the problem and in some settings the more damaging one: a real recording of a real instruction, denied. An institution that cannot establish the provenance of its own records has no answer to that.

So the question becomes "where did this come from?" — a provenance question rather than a forensic one. The two halves of the answer are:

  • Provenance, which travels with content and makes a verifiable claim about origin and history. Covered in lesson four.
  • Process, which stops relying on the authenticity of media for consequential decisions. Covered in lesson five, and it is the one that works today.

What this course does not re-cover. The onboarding camera — presentation attacks, injection attacks, liveness and what to demand from a biometric provider — is treated in the identity fraud and biometrics courses, including the CEN technical specification on detecting injection attacks. This course is about everywhere else, which is where most institutions have no controls at all.

Terminology: the FATF standards say suspicious transaction report (STR), while the United States and the United Kingdom say suspicious activity report (SAR). They are the same filing, and this course uses whichever term its own sources use.

Lesson 2 of 6

Where it actually costs money

Onboarding gets the attention. These are the places losses are being taken.

Voice in payment instruction. A call to a finance team or a relationship manager, in a familiar voice, authorising or amending a payment. Voice is the easiest modality to fake convincingly, needs only seconds of sample audio — a conference talk, a voicemail greeting, a social post — and arrives through a channel built on recognition rather than verification. The FBI has warned that criminals use generative AI to make exactly this kind of approach more convincing and more scalable.

Executive impersonation and invoice redirection. The long-standing business email compromise, now with a voice or video call to resolve the recipient's doubt. The call is the control that was supposed to catch the email, which is why adding a call-back to the same contact details solves nothing.

Video in remote meetings. Real-time synthetic video in a scheduled call with a supplier, a client or a colleague. Still harder than voice, no longer exotic, and devastating where a payment or an onboarding decision follows "we met them".

Fabricated documents and evidence. Statements, invoices, utility bills, payslips, certificates of incorporation, audit letters, screenshots of balances. Generative tools produce these at quality that defeats visual inspection, which matters for lending, insurance claims, disputes and know-your-business checks.

Disputes and claims. A customer submits a photograph, a recording or a chat transcript as evidence. Both directions now have a problem: fabricated evidence may be accepted, and genuine evidence may be dismissed.

Market-moving fakes. A fabricated announcement, a cloned executive statement, a fake regulatory letter. The exposure is short-lived and can be large, and it sits with treasury and communications rather than with fraud.

Internal process abuse. Approvals, sign-offs and recorded authorisations within the institution itself. If a control relies on "the manager said yes on the call", that control has the same weakness as everything above.

The pattern across all of them. Synthetic media is rarely the whole attack. It is used to resolve the one doubt that stands between the attacker and a decision someone is already minded to make — which is why controls that create a second, independent doubt work better than controls that try to examine the media.

Lesson 3 of 6

Detection, honestly

Detection has a role. It should be bought and operated with its limits understood, because the marketing in this field is some distance ahead of the evidence.

What detectors do. Look for traces a generator leaves: frequency-domain artefacts, inconsistent noise, implausible physiology such as blink rate or pulse signals in video, lighting and reflection inconsistencies, lip-sync mismatch, or statistical fingerprints in audio. Some are model-specific, some general.

Five limits to hold in mind:

  • Generalisation. Accuracy reported against the generators in a benchmark does not transfer to a generator released afterwards. Ask what it was evaluated against and when.
  • Degradation. Compression, re-encoding, screenshots, messaging apps and video conferencing destroy exactly the subtle artefacts detectors rely on. A detector that performs well on pristine files may perform near chance on a WhatsApp forward.
  • Adversarial pressure. Where detection is deployed at scale, generation adapts to defeat it. This is an arms race with a structural advantage for the attacker, who can test against the defender's detector.
  • Base rates. At realistic prevalence, even a strong detector produces mostly false positives. A tool that is 95 per cent accurate, applied where one item in a thousand is fake, accuses roughly fifty genuine items for every real one it finds.
  • Explainability. "Our model says 0.82" is not something you can put in a decision letter, a dispute response or a court. A score is evidence towards a judgement, not a finding.

How to use one properly. As a triage signal that routes to human review, never as an automatic decision. With a recorded threshold and a measured false-positive rate. Re-evaluated on a schedule, because its performance decays silently. And never as the only control standing between an input and a payment.

What to ask a vendor: which generators and datasets it was evaluated against, and when; performance on degraded and re-encoded media, not laboratory files; the false-positive rate at your operating threshold; whether it was tested against adversarially modified content; and what it outputs besides a number.

Lesson 4 of 6

Provenance, and what it proves

The other half of the answer is to stop asking whether content looks genuine and start asking whether it can prove where it came from.

Content Credentials and C2PA. The Coalition for Content Provenance and Authenticity publishes an open specification for attaching a cryptographically signed manifest to a piece of media. The manifest carries assertions — what captured or created it, when, what edits were applied, and by whom — signed by the issuing tool or service. Downstream tools can verify the signature and display the history.

What a Content Credential actually proves. That a particular signer asserted a particular history, and that the content has not changed since signing. That is valuable and it is narrower than people assume:

  • It does not prove the content is true, accurate or fairly presented.
  • It does not prove the absence of a credential means anything — most media has none, and stripping one is trivial, so missing provenance is uninformative rather than suspicious.
  • It does mean a signed capture from a trusted device or service is materially stronger evidence than an unsigned file, and it gives you something to verify rather than something to squint at.

Watermarking. Signals embedded in generated content, visible or imperceptible, to mark it as synthetic. Useful at the ecosystem level and weak as a guarantee: watermarks can be degraded by ordinary processing and removed deliberately, and open models can be run without applying one. Treat a present watermark as informative and an absent one as meaningless.

Hashing and fingerprinting. For content you hold, the strongest and cheapest control: hash it at the point of receipt, store the hash where it cannot be altered, and you can prove afterwards that your copy is the copy you received. This answers the liar's dividend directly and most institutions could do it this quarter.

Metadata is not provenance. Camera data, timestamps and geotags are trivially editable and routinely stripped by platforms. Their presence proves nothing; their absence proves nothing.

Where this is heading. Provenance is being built into capture devices, creation tools and platforms, and policy in several jurisdictions is moving towards marking synthetic content. The realistic expectation is a world where signed content is a minority that is strongly verifiable, and everything else is judged on process rather than appearance — which is why lesson five is the operational answer.

Lesson 5 of 6

After it happens

Four lessons on preventing it, and nothing on the morning after. That gap is where most of the avoidable damage occurs, because the instinctive handling of a synthetic-media incident destroys the evidence and asks the wrong question.

Preserve the artefact, and preserve it correctly. Keep the original file exactly as received. Hash it straight away. Record where it arrived from, through which channel, and when. Then stop touching it — do not re-encode it, do not forward it through a messaging application, and do not work from a screenshot. Compression and re-encoding destroy precisely the traces any later examination would depend on, and the handling people apply by reflex is the handling that ruins it. A screenshot of a video is not the video. A recording of a call played back into a phone is not the recording.

The money is on a clock and the evidence is not. Recall and recovery depend on hours. The person who discovers the fraud is rarely the person who can start a recall, and working that route out during the incident costs the window. Know beforehand who makes that call, out of hours included.

Then decide which question you are answering. "Was it a deepfake?" is usually the wrong one: often unanswerable, and unnecessary. You do not need to establish that a voice was synthetic to establish that the instruction was not authorised — and the second question can be settled in minutes by the out-of-band check that should have happened first. Chasing the forensic question delays the payment question, which is the one with money attached.

Report it in terms that keep the typology visible. FinCEN's 2024 alert on deepfake media exists because institutions described what they had actually seen. A narrative that says "impersonation" loses the signal entirely. Say that synthetic voice, video or a generated document was used, say how it arrived, and say what it was used to overcome.

Tell your own people the same day. The same pretext gets reused across an organisation within days — the same cloned voice, the same story, the same channel. A short, specific internal notice prevents more loss than any detector, and a vague one prevents none.

The other direction, which is the harder one. Sooner or later someone denies genuine evidence: a customer says the recorded instruction was not them, an employee says the call never happened. What you can establish is what you received and when — which is the payoff for hashing at receipt. If you did not hash it, say what you actually have and no more. Overstating the strength of an unsigned, unhashed file is how an institution loses a dispute it should have won.

And the post-mortem question that matters. Not "how did we fail to spot the fake?" but "why did a payment depend on spotting it?" Nearly every one of these cases resolves to a missing out-of-band verification rather than a missing detector. A review that concludes by buying detection has answered the easier question, and bought something whose performance will decay quietly while the real gap stays open.

Lesson 6 of 6

The controls that survive a perfect fake

Design on the assumption that the media is indistinguishable from real, because for a determined attacker it will be.

1. Out-of-band verification, and it is the whole lesson. For any instruction that moves money, changes payment details or grants access, verify through a channel the requester did not supply, to contact details you already held. A call back on the number in your records, not the number in the email signature or given on the call. This defeats a perfect voice clone, a perfect video and a perfect document, because it does not depend on judging any of them.

2. Dual authorisation for changes, not just payments. Most losses here come from a change — new bank details, a new beneficiary, an updated contact. The change is the control point and is frequently weaker than the payment control that follows it.

3. Verify the person, not the media. Shared secrets established in advance, pre-agreed safe words for high-value instructions, confirmation through an authenticated channel such as a banking application, or in-person for the largest decisions. Challenge questions drawn from information that is not publicly available.

4. Slow down the irreversible. Scam and impersonation tradecraft depends on urgency. A mandatory delay on first payments to a new beneficiary, and on changes to standing details, costs little and removes the attacker's main lever.

5. Train for the right thing. Not "how to spot a deepfake" — that advice ages badly and gives false confidence. Train on process: which requests require out-of-band verification, who may authorise a change, what to do when someone senior is applying pressure, and explicit permission to delay a payment to verify, including when the apparent requester is the chief executive.

6. Protect your own records. Hash and retain recordings, instructions and submitted evidence, so that when something genuine is denied you can establish what you received and when.

7. Report it. FinCEN's 2024 alert on deepfake media in identity documents exists because institutions reported what they saw, which is how a typology becomes visible. Where synthetic media features in an attempt, say so explicitly in the report rather than describing it as impersonation.

The judgement to take away. Detection is a triage signal with a short shelf life, provenance is valuable where present and silent where absent, and the control that still works is the one that does not depend on the media being genuine. Institutions that invested in verification process rather than in detection technology are the ones this problem has not reached.

Knowledge check

Twenty questions

Answer all twenty questions, then check your answers. You need 18 out of 20 to pass and receive a certificate. If you score less, you will see which answers were right and wrong, and then go through the course again before you retake the check. Your answers, progress and times are kept only in this browser.

Standards and frameworks covered

The published frameworks this course teaches toward. Saolix is not affiliated with, endorsed by or accredited by any of these bodies; naming a standard is a citation, not a claim about its author.

Sources

The official documents this course relies on. Laws and guidance change, so check the current version.

  1. C2PA specifications (Content Credentials) · Coalition for Content Provenance and Authenticity
  2. FinCEN Alert on Fraud Schemes Involving Deepfake Media Targeting Financial Institutions (November 2024) · Financial Crimes Enforcement Network (FinCEN), US Treasury
  3. Criminals Use Generative Artificial Intelligence to Facilitate Financial Fraud (PSA, December 2024) · Federal Bureau of Investigation, IC3
  4. NIST AI 600-1: Generative Artificial Intelligence Profile · National Institute of Standards and Technology
  5. Artificial Intelligence Risk Management Framework (AI RMF 1.0) · US National Institute of Standards and Technology
  6. Financial Trend Analysis: Identity-Related Suspicious Activity (January 2024) · Financial Crimes Enforcement Network (FinCEN), US Treasury
  7. Illicit Financial Flows from Cyber-enabled Fraud (November 2023) · FATF, Egmont Group and INTERPOL
  8. 2025 Report on Payment Fraud (December 2025) · European Banking Authority and European Central Bank
  9. Internet Crime Report 2025 · Federal Bureau of Investigation, Internet Crime Complaint Center (IC3)