LLMs, SLMs, generative AI and RAG.

What generative AI is

Generative AI is AI that produces new content, such as text, images, audio or computer code, rather than only sorting or scoring what already exists. It learns patterns from very large amounts of example data. It then uses those patterns to produce something new in response to an instruction, called a prompt.

It helps to compare it with the AI that came before:

• Predictive AI answers a narrow question about existing data: is this mobile money transaction likely to be fraud, which category does this document belong to, what will demand be next month?
• Generative AI produces an open-ended output: a summary, a draft letter, an answer to a question, an image.

Language models are the best-known kind of generative AI, and they are the focus of this course. Similar ideas also power image, speech and code generation.

Generative AI is useful because it is flexible: one model can summarise, translate, draft and answer questions. That same flexibility is the risk. Every answer is freshly generated, so nobody has checked it before you read it.

How large language models work

A large language model (LLM) works with tokens: small pieces of text, often a whole word or part of one. Given a sequence of tokens, the model estimates which token is likely to come next, adds one, and repeats. A long answer is built one token at a time. Languages that were less common in the training data, including many African languages, often need more tokens for the same sentence, which can make them slower and more costly to process.

Most current language models are based on the transformer, a neural-network design described in a 2017 research paper, "Attention Is All You Need". Its key idea, called attention, lets the model weigh how each part of the text relates to every other part.

A model is usually built in stages:

• Pre-training on a very large body of text. By predicting the next token again and again, the model learns language and a broad picture of the world.
• Fine-tuning on examples of instructions and good answers, so the model follows requests instead of just continuing the text.
• Alignment, often using feedback from people, to make answers more helpful and to refuse harmful requests.

Three terms come up constantly. Parameters are the learned numbers inside the model; "large" means billions of them. The context window is the number of tokens the model can consider at once, including your prompt and its answer. Temperature is a setting for how much randomness goes into picking each token. Low values make answers more predictable; higher values make them more varied.

Small language models

A small language model (SLM) uses the same ideas with far fewer parameters. There is no official cut-off. The term usually means models from a few hundred million to a few billion parameters. The largest models have hundreds of billions.

Small models have real advantages:

• They run on ordinary hardware, including laptops and, for the smallest, phones.
• They can keep data private, because they can run inside your own systems or on the device.
• They are cheaper and faster to run, which matters when you process large volumes.

The trade-off is breadth. A small model knows less about the world and is weaker at open-ended reasoning. It works well when the task is narrow and well defined, such as pulling fields out of a standard message or sorting documents into categories. It works even better when it has been fine-tuned (given extra training) on examples of that task.

Some small models are trained partly by distillation: a smaller model learns to imitate the outputs of a larger one. The practical rule: test candidates on your own examples and choose the smallest model that meets your quality bar.

Retrieval-augmented generation (RAG)

A language model only knows what was in its training data. That data stops at a date and does not include your organisation's documents. Retrieval-augmented generation (RAG), a term introduced in a 2020 research paper, addresses this: the system first finds relevant material, then gives it to the model together with the question.

A typical RAG system works in two phases:

• Preparation: documents are split into short passages, called chunks. Each chunk is turned into an embedding, a list of numbers that represents its meaning, and stored in a searchable index, often called a vector database.
• Answering: the question is also turned into an embedding. The chunks with the closest meaning are retrieved and placed in the prompt, with an instruction to answer only from them and to cite them.

Many systems combine this meaning-based search with ordinary keyword search (called hybrid search). Keyword search is better at exact items such as names, account numbers and product codes, which embeddings can miss.

RAG brings three benefits. Answers can use current and private information without retraining the model. Each answer can point to its sources so a person can check them. And access rules can be applied, so people only get answers from documents they are allowed to see. Its limits matter just as much. If retrieval finds the wrong passages, the answer will be wrong, and a model can still misstate what a passage says. RAG reduces made-up answers; it does not remove them.

Limits and safe use

Every language model, large or small, with or without RAG, has limits you need to design around:

• Hallucination: fluent, confident answers that are simply not true.
• Variability: the same question can get a different answer, especially after a change of model, prompt or settings.
• Prompt injection: instructions hidden in documents or web pages can take over the model's behaviour. This matters most when the model can use tools.
• Bias carried over from training data.
• Data exposure: text sent to an outside AI service leaves your control. If it contains personal data, Kenya's Data Protection Act, 2019 applies, including its rules on sending data outside Kenya.

When models are given tools and allowed to take steps on their own, they are called AI agents. The more an agent can do, the more important it is to limit its permissions, record what it did and require a person's approval for actions that matter.

The safest pattern for regulated work is to let the model advise (summarise, explain, suggest) while fixed rules and people decide. Test the system on your own cases, record the model version, and measure how often its suggestions are right before relying on them.

Your certificate of completion

Enter your name as you want it to appear, then save the certificate as a PDF. In the print window, choose Save as PDF. A certificate is issued once per completion of the course.

Saolix does not record who takes this course, so it cannot verify these certificates. The certificate confirms completion of a free self-paced course and is not an accredited qualification.