Choosing the right intervention
Fine-tuning is the most expensive answer to most questions people ask it. The first lesson of this course is how to tell which problem you actually have, because the wrong intervention wastes a quarter.
There are four levers and they fix different things. A better prompt, with examples. Retrieval, to supply knowledge. Constrained decoding, to force a format. Fine-tuning, to change the model's behaviour. Each is cheap or expensive in a different currency.
If the model lacks facts, retrieve. Fine-tuning is a poor way to install knowledge: it is expensive per fact, it cannot be updated without retraining, it provides no provenance, and the model will still state things that are not in the training data. Retrieval solves all four.
If the model will not hold a format, constrain the decoding. Enforcing valid structured output at the sampling step is exact and immediate. Fine-tuning for format works and is a slow way to achieve something a schema does in an afternoon.
If the model misunderstands your domain's language consistently, consider fine-tuning. A specialist vocabulary used in a particular way, a house style, a classification scheme with conventions that are genuinely idiosyncratic. This is the case fine-tuning is good at: shaping behaviour rather than adding information.
If the model is capable but too expensive, consider distillation. Use a strong model to produce high-quality outputs on your task, then train a smaller one on them. This is fine-tuning in service of cost, and it is one of the most reliably successful uses of it.
If the model's judgement does not match yours, you need preference data. Where the issue is which of two acceptable answers is better, supervised examples are a blunt instrument and preference optimisation is the designed one. That is lesson eight.
Latency and cost are legitimate reasons on their own. A fine-tuned small model that matches a large one on a narrow task, at a fraction of the price and a third of the latency, is a good outcome even with no quality gain.
The order to try things in. Prompt and examples, then constrained output, then retrieval, then distillation of a working prompt into a smaller model, then fine-tuning for behaviour, and only then preference optimisation. Each step is cheaper than the next and fixes a class the next one does not.
And measure before you start. Without a baseline on a case set, you cannot tell whether fine-tuning helped, and every later decision becomes an argument about impressions. The evaluation in lesson twelve is the prerequisite, not the final step.
What fine-tuning actually does
Fine-tuning continues training on new data, with the same objective as pre-training, at a much lower learning rate. Being precise about that explains both what it can achieve and what it damages.
The mechanism is more gradient descent. Examples are presented, loss is computed on the tokens you want the model to produce, and the weights move. There is nothing categorically different from pre-training except the scale and the data.
It moves a distribution; it does not insert records. After fine-tuning, text resembling your examples is more probable. That is why it shapes style, format and classification behaviour well and installs facts badly: a fact appearing a handful of times competes with everything else the model saw.
The loss mask matters. In instruction tuning you usually compute loss only on the response, not on the prompt, so the model learns to produce the answer rather than to reproduce the question. Getting this wrong is a common and quiet defect.
Catastrophic forgetting is real and measurable. Training on narrow data degrades performance elsewhere. Published work on continual fine-tuning of large models documents this directly, and it is why a model fine-tuned for one task often becomes worse at instruction following, at other languages, or at its own refusal behaviour.
Which means you must evaluate what you were not trying to change. A set of general cases, kept alongside your task cases, run before and after. A model that improved on your task by ten points and lost fifteen elsewhere is not an improvement, and nobody notices without the second set.
Alignment behaviour is fragile under fine-tuning. Safety and refusal properties were trained in after pre-training, and further training can weaken them, including unintentionally. If you fine-tune a model with safety behaviour you depend on, test that behaviour afterwards.
Small amounts of high-quality data go a long way for style. Published work training on a thousand carefully curated examples produced a strongly instruction-following model, which supports the view that for shaping behaviour the capability was already present and was being surfaced rather than created.
The corollary is that data quality dominates quantity. Ten thousand mediocre examples with inconsistent labelling will teach inconsistency. A few hundred examples that are right, consistent and representative will usually beat them.
And it is a model you now own. A fine-tuned model is an artefact to version, store, evaluate, serve and eventually retire, and it ages as the base model it was built from is superseded. That operational weight is part of the decision.
The data is the work
Every fine-tuning project is a data project wearing different clothes. Teams budget for compute and lose the time on labelling, and the ones that succeed invert that.
Decide the format precisely and write it down. What the input looks like, what the output looks like, which fields are required, how edge cases are represented, and what the model should do when the input is unusable. Inconsistency here is learned faithfully.
Write a labelling guide before labelling. Definitions, examples of each category, examples of near misses, and the rule for ambiguous cases. Two people labelling the same hundred items and comparing will reveal whether the guide is usable, and the disagreements are where the definitions are wrong.
Measure agreement, and fix the guide rather than the labellers. If two careful people agree only four times in five, the model cannot do better than that ceiling, and the ceiling is in the definitions.
Represent the distribution you will actually see. Training only on clean examples produces a model that fails on the messy majority. Sample from real traffic, including the rare and awkward cases, in something resembling their true proportions.
Include the negative and the refusal cases. Inputs where the right answer is nothing, or unknown, or a request for clarification. A training set containing only successful answers teaches a model to always produce one.
Deduplicate, and watch for leakage between splits. Near-duplicates across your training and evaluation sets will make the evaluation look excellent and the deployment disappointing. This is the single most common way a fine-tuning result turns out to be false.
Generated data is legitimate, with care. Using a strong model to produce or augment examples works well, particularly for distillation. The risks are inherited errors, a narrowed distribution and licence terms that may prohibit training on outputs. Check all three, and have a person review a sample.
Personal data in training sets is a commitment. Once personal data is in the weights it cannot be deleted from them, which makes erasure requests a genuine problem. Minimise, pseudonymise where possible, and document the basis. Data protection law applies to this processing as to any other; in Kenya, for example, that is the Data Protection Act, 2019.
Hold out a test set and do not look at it. A training set, a validation set used for tuning, and a test set touched once at the end. Teams that skip this discover their improvement was a measurement artefact after deployment.
And keep the data under version control with the model. Which dataset version produced which model, with the labelling guide of the day. Without that pairing, a result is not reproducible and a regression cannot be diagnosed.
Full fine-tuning and its cost
Updating every parameter is the original method, and understanding what it costs is what makes the parameter-efficient alternatives in the next lesson make sense.
The memory arithmetic is the constraint. Training requires the weights, the gradients, the optimiser state and the activations. With a common optimiser keeping two values per parameter, the optimiser state alone is several times the size of the model, and the total requirement is roughly an order of magnitude more memory than inference.
Which puts full fine-tuning of a large model out of reach for most teams. A model that serves comfortably on one accelerator may need a cluster to train, with the distribution strategy, interconnect and engineering time that implies.
Mixed precision is standard and is not free. Training in reduced precision with a higher-precision copy for the updates saves memory and introduces numerical care: loss scaling, and watching for divergence that only appears at scale.
The hyperparameters that matter are few. The learning rate, which is the one most likely to ruin a run and should be far lower than for pre-training. The number of epochs, where too many overfit a small set quickly. The batch size, which interacts with the learning rate. And a warm-up and decay schedule.
Overfitting arrives fast on small datasets. With a few thousand examples a model can memorise within two or three passes. Watch validation loss rather than training loss, stop when it turns, and prefer the earlier checkpoint.
Checkpointing is operationally important. Keep several, evaluate them rather than assuming the last is best, and record which data version produced each.
It is the right choice in a minority of cases. A large dataset, a substantial shift in domain, or a need to change the model's behaviour broadly rather than for one task. Below that threshold the parameter-efficient methods reach the same place for a fraction of the cost.
And full fine-tuning forfeits the base model's updates. Once you own a derived model, a better base model means doing the work again. With adapters, as the next lesson explains, the separation between base and adaptation keeps that migration much cheaper.
Parameter-efficient methods
The methods in this lesson are why fine-tuning became accessible. They train a small number of new parameters and leave the model's own weights untouched, and for most purposes they reach the same quality.
The idea began with adapters. Published work inserted small trainable modules between the layers of a frozen pre-trained model and showed that training those alone approached full fine-tuning quality on a wide range of tasks, at a tiny fraction of the trained parameters.
Low-rank adaptation is the method now in general use. Instead of inserting modules, it adds a low-rank update to selected weight matrices: two small matrices whose product has the shape of the original. Only those are trained. The published result was that this matched full fine-tuning on the tasks tested while training a small fraction of the parameters.
Why low rank is a reasonable assumption. The change needed to adapt a model to a task appears to lie in a much smaller space than the full weight matrix. That is an empirical finding rather than a theorem, and it has held well enough to become the default.
The practical knobs are few. The rank, which trades capacity against size; which matrices to adapt, with the attention projections the usual starting point; a scaling factor; and the usual learning rate, which for these methods is typically higher than for full fine-tuning.
The operational advantages are the real argument. An adapter is small enough to store many of them, to swap per customer or per task at serving time, and to version independently of the base model. Several adapters can be served from one copy of the base weights, which changes the economics of per-customer models entirely.
Quantised training made it cheaper again. Published work combined a four-bit quantised frozen base model with low-rank adapters and showed that models previously requiring a cluster could be fine-tuned on a single accelerator, with quality close to sixteen-bit fine-tuning.
Prompt and prefix tuning are the lighter cousins. Rather than adapting weights, they learn a set of continuous vectors prepended to the input. Published work showed this becomes competitive as model scale grows. It is the lightest option and generally weaker than low-rank adaptation at small scale.
Merging is possible and has caveats. An adapter can be folded into the base weights to remove any inference overhead, at the cost of losing the ability to swap it. Combining several adapters by merging works unevenly and should be evaluated rather than assumed.
The default recommendation. Start with low-rank adaptation on a quantised base, at a modest rank, on a few hundred to a few thousand well-curated examples. It is the configuration that gives most teams most of the available benefit for the least cost and the least operational weight.
Instruction tuning
The step that turns a text continuation engine into something that answers requests is worth understanding separately, because it explains the behaviour of every model you use and it is the thing you are usually extending.
The objective is unchanged; the data is different. Pairs of instruction and response, with loss computed on the response. The model learns that text of the form of an instruction is followed by text of the form of a compliant answer.
The published results established the generalisation. Fine-tuning on a collection of tasks phrased as instructions produced models that performed well on tasks they had never been trained on, which was the surprising part: instruction following transferred rather than being learned per task.
Scaling the task mixture helped substantially. Later work scaled the number of tasks and the model size and found consistent gains, including on reasoning when chain-of-thought examples were included in the mixture. The composition of the mixture, not merely its size, was doing work.
But a small, excellent set can be enough. Training on a thousand carefully curated examples produced a model that followed instructions well, supporting the interpretation that instruction tuning surfaces a capability rather than installing one. For a team adapting a model, that is encouraging: your dataset does not have to be enormous, it has to be good.
What this means for your own fine-tuning. You are almost always extending an already instruction-tuned model, so your examples should match the format it expects, including its conversation template. Mismatching the template is a frequent cause of a fine-tune that behaves strangely.
Keep some general instruction data in the mixture. Training exclusively on your narrow task is what causes the forgetting in lesson two. A proportion of general instruction examples alongside your own measurably reduces the loss of general ability.
Format and style transfer easily; judgement does not. Instruction tuning will reliably teach a model to produce your report structure. It will not reliably teach it which of two defensible answers your organisation prefers, which is what preference methods are for.
And do not fine-tune away a prompt you could have written. Many instruction-tuning projects reproduce, at considerable cost, what a well-structured prompt with four examples already achieved. Establish the prompt-only baseline first; it is the comparison the whole project should be judged against.
Distillation
Training a small model on the outputs of a large one is the most dependable way to reduce cost in this subject, and it deserves to be better known than it is.
The original idea is older than the current models. Published work on knowledge distillation trained a small network to match the output distribution of a large one, using the full distribution rather than hard labels, and found the small network learned better than from the labels alone.
The modern form is simpler in practice. Run a strong model on a large sample of your real inputs, keep the outputs that pass review, and fine-tune a small model on those pairs. You are distilling a prompt and a model into something cheap.
It works because the task is narrow. A small model cannot match a large one in general. It frequently can match it on one well-specified task, which is the only thing you need.
The gains are often dramatic. Teams routinely report a small fine-tuned model matching a large prompted one on a narrow classification or extraction task, at a small fraction of the cost and latency. Of all the things in this course, this is the one with the most reliable return.
Quality control on the generated data is the whole game. The small model will learn the large model's errors faithfully. Filter by confidence, by agreement between two runs, by a verifier where one exists, and by human review of a sample. An unfiltered distillation set propagates every systematic mistake.
Keep the inputs real. Distilling on synthetic inputs produces a model good at synthetic inputs. Use your actual traffic, with the awkward cases included in their real proportion.
Check the licence before you start. Some providers' terms restrict using their outputs to train competing models. This is a legal question with a clear answer per provider, and it should be settled before the data is generated rather than after.
Evaluate against the teacher, not against perfection. The question is whether the small model is close enough to the large one on your cases at your acceptable cost. A three-point drop for a tenfold cost reduction is usually a good trade, and it is a decision for a product owner rather than an engineer.
And keep the teacher available. A hybrid in which the small model handles the confident majority and defers uncertain cases to the large one captures most of the saving with little of the quality loss, and the deferral rate becomes a tunable parameter.
Learning from preferences
Supervised examples teach a model what to say. Preference methods teach it which of two things to say, and that distinction is what they exist for.
The problem they solve. For many outputs there is no single right answer, and what you want is a judgement: this response is better than that one. Writing a supervised example forces you to produce the ideal answer; expressing a preference only requires you to compare two.
The original approach learned from human comparisons. Published work on reinforcement learning from human preferences showed that a reward model trained on comparisons could guide an agent without a hand-written objective, and the method was later applied to language models.
Then it was applied to instruction following at scale. The widely cited work on training language models to follow instructions used supervised fine-tuning followed by a reward model and reinforcement learning, and found that a much smaller model trained this way was preferred to a far larger one that had not been. That result is the reason every assistant model you use has had this treatment.
It is also where the distortions come from. Raters prefer longer, more structured, more confident and more agreeable answers, so models trained this way acquire those traits. Verbosity and a tendency to agree with the user are not accidents; they are what was optimised.
Direct preference optimisation removed most of the machinery. Published work showed that the same objective can be optimised directly on preference pairs, without training a separate reward model or running reinforcement learning. It is simpler, more stable, cheaper, and for most teams it is the only version of this that is practical.
Which means preference tuning is now accessible. A few thousand comparison pairs from your own reviewers, applied on top of a supervised fine-tune, is a project a small team can complete. The hard part is the pairs, not the training.
Collecting good pairs needs discipline. Present two responses to the same input, anonymised and in random order, and ask which is better against stated criteria. Use several raters on a subset and measure agreement. Criteria that raters cannot apply consistently will produce a model that optimises noise.
Feedback from a model is a documented option. Published work on using a model's own critiques against a written set of principles, rather than human labels, showed it could substitute for much of the human comparison work. It is cheaper and it inherits the judging model's biases, which is a trade to make deliberately.
And evaluate on what you did not optimise. Preference tuning changes tone and willingness as well as the specific behaviour you wanted. Run the general case set and the safety cases afterwards, because this is the stage most likely to move both.
Making it small enough to serve
A fine-tuned model is one you have to run. Quantisation and the related techniques are how it fits on hardware you can afford, and they have well-documented trade-offs.
Quantisation reduces the precision of the weights. From sixteen bits to eight or four, with a scheme for mapping the range. Memory falls roughly in proportion, and on memory-bound generation the speed improves as well.
Eight-bit inference is essentially free in quality terms. Published work identified that the difficulty lies in a small number of outlier features and handled them separately, making eight-bit matrix multiplication practical for large models with negligible degradation.
Four-bit is a real trade and usually a good one. Published post-training quantisation methods achieve four bits with modest loss on most tasks, which brings large models within reach of single-accelerator serving. Measure it on your own task: degradation is uneven and falls hardest on the most demanding cases.
Post-training is the common path. Quantise a trained model using a small calibration sample. No retraining, minutes of work, immediately deployable. Quantisation-aware training does better and costs a training run, and is worth it only at the extreme end.
Weights and activations are different problems. Weight-only quantisation is the easy, widely supported case. Quantising activations as well gives more speed and is harder to do without quality loss.
The key-value cache can be quantised too. In long-context serving the cache often dominates memory, and reducing its precision extends the practical context length. This is a separate decision from quantising the weights and often the more valuable one.
Adapters complicate it slightly. An adapter trained against a quantised base should be served against the same quantisation. Training against one precision and serving at another is a quiet source of degradation.
Pruning and sparsity exist and help less. Removing weights, in structured or unstructured patterns, reduces size but needs hardware support to produce real speed. For most teams quantisation gives more benefit for less effort.
Measure three things after any of it. Quality on your case set, latency at the relevant percentile, and memory under realistic concurrency. A technique that improves two and ruins the third is common, and only the third is usually discovered in production.
And remember the alternative. A smaller model fine-tuned on your task may beat a quantised large one on all three measures. Distillation and quantisation are competing answers to the same question, and the comparison is worth running rather than assuming.
Running the training
The mechanics of a training run are not difficult and they have a short list of ways to waste a week. This lesson is that list.
Start with the smallest thing that can fail. A few dozen examples, one short run, end to end: data loading, template formatting, training, saving, loading the result, generating from it. Most pipeline faults appear here and cost minutes rather than hours.
Check the formatted examples by eye. Print ten of them exactly as the model will see them, including the conversation template and any special tokens. An incorrect template is the single most common cause of a fine-tune that produces strange output, and it is invisible in the metrics.
Verify the loss mask. Confirm that loss is computed on the response and not the prompt. A quick test is whether the model, after training, completes prompts rather than answering them.
Watch validation loss, not training loss. Training loss falls whether or not anything useful is happening. Validation loss turning upward is the signal to stop, and on small datasets it turns within two or three epochs.
Sweep the learning rate first, narrowly. It is the parameter most likely to decide the outcome. Three values an order of magnitude apart, short runs, pick the region, then refine. Everything else matters less than this.
Fix the seeds and record them. Seed, library versions, base model revision, dataset version, and the full configuration, stored with the resulting weights. Without them a result cannot be reproduced, and a reproducible failure is the only kind you can fix.
Evaluate checkpoints, do not assume the last. Generate from several and score them on the case set. The best checkpoint is frequently not the final one, particularly on small datasets.
Keep the cost visible. Accelerator hours per run, number of runs, and the staff time around them. Fine-tuning projects overrun on the number of iterations rather than the cost of one, and counting the iterations is what keeps that honest.
Expect data problems to surface as training problems. A loss that will not fall, or a model that produces one answer regardless of input, is usually a labelling or formatting fault rather than a hyperparameter one. Go back to the data before tuning the optimiser.
And write down what you tried. A short log of each run with its configuration and its score. Teams without one repeat experiments, argue from memory, and cannot explain how they arrived at the model they are serving.
Serving and operating your own model
Once you have a model of your own you have an operational commitment that a hosted interface did not impose. These are the parts teams underestimate.
Serving throughput is mostly about batching and memory. Continuous batching, paged management of the key-value cache, and prefix sharing between requests are the techniques current serving stacks use, and the difference between a naive server and a good one is large.
Capacity planning is the cache arithmetic again. Model weights plus the cache for concurrent sessions at your context length. Decide the maximum context you will accept, because it multiplies by the number of users and determines how many fit on a device.
Adapters change the deployment pattern. Several low-rank adapters can be served from one copy of the base weights, with the adapter selected per request. That makes per-customer or per-task models affordable, and it is the main operational reason to prefer adapters over full fine-tuning.
Version everything and serve explicitly. Base model revision, adapter version, quantisation, serving stack version, and the prompt template. Record them with every response, because a complaint three weeks later is otherwise unanswerable.
Keep a rollback path. The previous model, loadable without a rebuild. Fine-tuned models regress in ways evaluation sets miss, and the ability to go back within minutes is what makes deploying one reasonable.
Monitor the distribution of inputs, not only the outputs. The reason a fine-tuned model degrades is usually that the traffic has moved away from the training distribution. Tracking input characteristics gives you warning; tracking only quality gives you a complaint.
Plan for the base model being superseded. A better base model arrives and your adaptation has to be redone. With adapters this is a retraining run on existing data; with full fine-tuning it is the whole project again. Either way, the dataset is the durable asset, not the weights.
Decide where the model runs and why. Hosted fine-tuning, your own infrastructure, or on-premises for data residency. The argument is usually about data rather than cost, and running open-weight models yourself is the reason this course is practical for a small organisation at all.
Document the model. What it was trained on, what it is for, what it is not for, how it was evaluated, its known weaknesses and its version. This is ordinary engineering documentation and it is also what a regulator, a customer or your own successor will ask for.
Evaluating an adaptation
Everything in this course is unfalsifiable without measurement, and the specific thing to measure is not only whether the task improved.
Three sets, not one. Task cases, for the thing you are trying to improve. General cases, for the abilities you need to keep. Safety cases, for the refusal behaviour that fine-tuning can weaken. Run all three before and after, every time.
The baseline is the prompted base model. Not a worse prompt, not a smaller model: the best prompt-only configuration you could build. If a fine-tune does not clearly beat that, the project has not succeeded, whatever the loss curve did.
Score the task properly. Field-level accuracy for extraction, the usual measures and a confusion matrix for classification, a written rubric for generation. Decide the metric before training rather than choosing the one that flatters the result.
Report cost and latency with quality. The usual reason to fine-tune is economics, so a result without the price is not a result. Cost per request, latency at the relevant percentile, and the memory footprint under concurrency.
Hold the test set back genuinely. Used once, at the end, after the configuration is settled. A test set consulted repeatedly during development has become a validation set and no longer measures generalisation.
Check for leakage before believing a good number. Near-duplicates between training and evaluation sets are the most common cause of an excellent score that does not survive deployment. Check by similarity, not only by exact match.
Measure the forgetting explicitly. The general set is not a formality: published work on continual fine-tuning documents substantial degradation outside the trained task, and the only way to know your own is to measure it.
Test the model's behaviour on inputs it should refuse or defer on. Out-of-scope requests, missing information, and the cases where the right answer is a question. Fine-tuning on a set of confident answers makes a model confident, including where it should not be.
Keep the comparison runnable by one command. Base against adapter against the previous adapter, on all three sets, with cost and latency, in one table. The teams that iterate successfully are the ones for whom this is cheap to run.
And decide the acceptance criteria in advance. What improvement, on which metric, at what cost, with what tolerance for regression elsewhere. Written down before the first run, because afterwards every number has an advocate.
The governance and legal side
A fine-tuned model is a new artefact with obligations attached, and most of them are easier to handle at the start than to retrofit.
Know your rights in the base model. Open-weight models carry licences with real terms: permitted uses, redistribution conditions, naming requirements, and in some cases restrictions above a user threshold. Read the licence of the specific model rather than assuming a category.
Know your rights in the training data. Customer data, licensed third-party content, scraped material and another provider's model outputs each raise a different question. The last is the one most often overlooked and the easiest to check.
Personal data in weights is effectively permanent. You cannot locate and delete one person's contribution to a set of weights. That makes minimisation at the data stage the only real control, and it makes an erasure request against a trained model a problem you should not create for yourself.
Document the processing before you train. Lawful basis, purpose, categories of data, retention, and a risk assessment where the processing is higher risk. Data protection law frames this wherever you are, and the questions are the ordinary ones asked of any processing.
Where the output affects people, the transparency duties follow the use. A model that scores, ranks or decides something about a person brings obligations about explanation, human involvement and challenge. That is a design question at the point you decide what the model is for.
The emerging regulatory picture is use-based. The European regulation on artificial intelligence places obligations according to the risk of the application rather than the technique, with additional duties for general-purpose models. Fine-tuning a model for a high-risk use puts the obligations on the use, not on the fact that you fine-tuned.
Keep a record of the model's provenance. Base model and revision, datasets and their versions, training configuration, evaluation results and known limitations. This is the documentation that answers a customer's due diligence questionnaire and a supervisor's question with the same file.
Decide who owns the model. A named person accountable for its evaluation, its version in production, and its retirement. Fine-tuned models outlive the projects that created them more often than anything else in this subject.
And be honest in what you claim about it. A model fine-tuned on your data is not thereby accurate, unbiased or specialist. Describe what was measured and what it was measured on, and resist the temptation to describe the technique as though it were a result.
One adaptation, end to end
A credit team classifies incoming documents into eleven types and extracts four fields from each. Volume is about twelve thousand documents a month. A prompted large model worked and the cost and latency were both unacceptable. This is the project.
The baseline, measured properly. The best prompt they could write, with six examples and constrained output, on a large hosted model: 93.5 per cent classification accuracy, field extraction correct on 89 per cent of documents, at a cost and a latency the team recorded precisely. Everything afterwards was compared against those four numbers.
The decision. Not retrieval, because no knowledge was missing. Not a better prompt, because that was the baseline. Distillation into a small model, because the task was narrow, the volume was high and the economics were the problem.
The data. Fourteen thousand documents run through the prompted large model, with the outputs filtered: agreement between two runs at different temperatures, and a confidence threshold. That removed about 11 per cent. A person then reviewed a random sample of four hundred and found an error rate of 2 per cent, concentrated in two document types that were genuinely ambiguous.
What the review changed. The two ambiguous types were re-defined with a written rule, and eight hundred examples of them were re-labelled by a person rather than distilled. This was the most valuable two days of the project.
The split. Training, validation and test, split by document source rather than randomly, after a near-duplicate check removed 600 documents that appeared in more than one split. The random split had been tried first and the test score was four points higher, which was the leakage.
The training. A small open-weight instruction model, low-rank adaptation at modest rank on a four-bit quantised base, on one accelerator. Learning rate swept over three values in short runs. Three epochs, with the best checkpoint at the end of the second on validation loss.
The result. Classification 93.1 per cent, field extraction 90 per cent, against the baseline's 93.5 and 89. Cost per document roughly a thirtieth. Latency about a quarter. The general case set fell by two points, which did not matter because the model does one job.
The deferral path. Where the model's confidence on classification was low, the document was sent to the large model instead. That was 6 per cent of volume, cost very little, and took classification accuracy on the combined system to 95.8 per cent - above the original baseline.
What they operate. Adapter and base versions recorded per document. Monthly re-evaluation on a fresh sample of three hundred documents labelled by a person. A monitor on the distribution of document types, which three months later flagged a new type the model had never seen and was confidently misclassifying.
What it cost. Four weeks, most of it on data. One accelerator. The training runs were a small part of the budget, and the two days spent re-defining the ambiguous categories improved the result more than any other change.
Fifty questions
Answer all fifty questions, then check your answers. You need 45 out of 50 to pass and receive a certificate. If you score less, you will see which answers were right and wrong, and then go through the course again before you retake the check. Your answers, progress and times are kept only in this browser.
Your answers
Your certificate of completion
Enter your name as you want it to appear, then save the certificate as a PDF. In the print window, choose Save as PDF. A certificate is issued once per completion of the course.
Rate this course
How would you rate this course? Your rating is anonymous — just a number, with no name and no account.
Bring this to your team
Your colleagues can take this course too — it is free, needs no sign-up, and nothing they enter leaves their browser. If your institution wants the same ground covered as structured training for a team, with an instructor and your own cases, we run courses in a classroom or online for groups of ten or more.
Talk to us about training your team → See the paid courses →
Saolix does not record who takes this course, so it cannot verify these certificates. The certificate confirms completion of a free self-paced course and is not an accredited qualification.
Sources
The official documents this course relies on. Laws and guidance change, so check the current version.
- LoRA: Low-Rank Adaptation of Large Language Models · Hu and others, arXiv 2106.09685
- QLoRA: Efficient Finetuning of Quantized LLMs · Dettmers, Pagnoni, Holtzman and Zettlemoyer, arXiv 2305.14314
- Parameter-Efficient Transfer Learning for NLP, the adapter method · Houlsby and others, arXiv 1902.00751
- The Power of Scale for Parameter-Efficient Prompt Tuning · Lester, Al-Rfou and Constant, arXiv 2104.08691
- Finetuned Language Models Are Zero-Shot Learners · Wei and others, arXiv 2109.01652
- Scaling Instruction-Finetuned Language Models · Chung and others, arXiv 2210.11416
- LIMA: Less Is More for Alignment · Zhou and others, arXiv 2305.11206
- Training language models to follow instructions with human feedback · Ouyang and others, arXiv 2203.02155
- Deep reinforcement learning from human preferences · Christiano and others, arXiv 1706.03741
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model · Rafailov and others, arXiv 2305.18290
- Constitutional AI: Harmlessness from AI Feedback · Bai and others, arXiv 2212.08073
- Distilling the Knowledge in a Neural Network · Hinton, Vinyals and Dean, arXiv 1503.02531
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale · Dettmers, Lewis, Belkada and Zettlemoyer, arXiv 2208.07339
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers · Frantar, Ashkboos, Hoefler and Alistarh, arXiv 2210.17323
- An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning · Luo and others, arXiv 2308.08747
- Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence · Official Journal of the European Union, via EUR-Lex
- Data Protection Act, 2019 (No. 24 of 2019) · Office of the Data Protection Commissioner, Kenya



