Free and open AI models.

What "free" and "open" mean

An AI model is mostly a very large set of numbers, called weights (or parameters), that were learned during training, plus the code that uses them to produce answers. When people call a model "free" or "open", they usually mean that anyone can download the weights and run them.

That is not the same as open source in the traditional software sense. It helps to separate three ideas:

• Open weights: you can download the trained model. The training data and the training code may stay private.
• Open source AI: a stricter idea. In October 2024 the Open Source Initiative published the Open Source AI Definition 1.0. As well as the weights, it expects the full code used to train and run the model, and enough detail about the training data for a skilled person to build a similar system.
• Free of charge: you pay nothing to download. This says nothing about what you may do with the model. The licence decides that.

Many popular downloadable models are open-weight, not fully open source. Knowing which kind you have stops you from assuming rights you do not have.

One more distinction matters for privacy. An open model that you download and run on your own machines keeps your data with you. The same model used through someone else's online service does not: your text is sent to that service, as with any other cloud AI.

Licences decide what you may do

Every downloadable model comes with a licence. What you may do depends on that licence, not on the fact that the download cost nothing. You will usually meet one of three kinds:

• Permissive open-source licences, such as Apache 2.0 and MIT. They allow commercial use, changes and redistribution. If you pass the model on, you must keep the copyright and licence notices. Apache 2.0 also requires changed files to carry a notice saying they were changed.
• Custom model licences written by the model's publisher. Many allow commercial use but add conditions: an acceptable-use policy listing banned uses, a requirement to credit the model, or a separate agreement once your service passes a stated number of users.
• Non-commercial or research-only licences, which do not allow commercial use at all.

Three habits keep you safe. Read the licence of the exact model and version you use, because terms can change between versions. Keep the licence and notices with any copy you pass on. And do not suggest that the publisher endorses your product unless it does.

A model licence covers the model only. It does not cover the data you put into it or the laws that apply to your use. In Kenya, for example, the Data Protection Act, 2019 still applies to any personal data you process with the model.

Choosing a model

Open models are shared through public model hubs (websites that host thousands of models) and through publishers' own sites. A well-published model comes with a model card: a document that explains what the model is for, how it was trained, what it should not be used for, its known limits and its licence. Read it first.

Compare candidates on:

• Size, measured in parameters (for example 1 billion or 7 billion). Larger models are usually more capable but need more memory and run more slowly.
• Context length: how much text the model can take into account at once.
• Languages and tasks it was trained and tested for. If your users write in Kiswahili or mix languages, test that directly.
• Licence, as in the previous lesson.

Published benchmark scores are a starting point, but they seldom match your exact task. A more reliable test is a small set of your own real examples, say 20 to 50, each with the answer you expect, run against every candidate.

Also check the file format. Some older formats can run hidden code on your computer when the file is opened, so a model file from an unknown source is a security risk. Formats designed to hold only the weights, such as safetensors, avoid this. Download models only from their official publisher or a well-known hub.

Running a model yourself

The main reason to use an open model is that you can run it on hardware you control, so your data does not have to leave it. The first question is memory. A rough guide is the number of parameters multiplied by the bytes stored for each one:

• at 16-bit precision, each parameter takes 2 bytes, so a 7-billion-parameter model needs about 14 GB for its weights alone;
• at 4-bit precision, it needs about a quarter of that, around 3.5 GB, plus extra working memory while it runs.

Storing the weights at lower precision like this is called quantisation. It makes models small enough for ordinary laptops, and the smallest ones even for phones, at the cost of some quality. How much quality is lost depends on the model and the task, so test it on your own examples.

Three kinds of free software are common. Some run quantised models efficiently on an ordinary computer. Some are desktop tools that download and run a model in a few steps. Others share one model among many users on servers with graphics processors (GPUs). A GPU makes answers much faster, but small models can run on a normal processor.

Running a model yourself also makes you responsible for it: keeping the software updated, controlling who can reach it on the network, and recording which model version is in use.

Using open models responsibly

Open models share the weaknesses of all language models, and more of the responsibility now sits with you:

• They can be confidently wrong. Models produce text that sounds right, not checked facts. Check anything that matters.
• Their knowledge stops at the date their training data was collected, and they may know little about local context, Kenyan law or recent events.
• They can reflect bias in their training data. Test them on the kinds of people and cases they will actually affect.
• They come without a warranty. Open licences usually provide the model "as is", so if it fails, the risk is yours.
• They can be manipulated by instructions hidden in the text they are asked to read. This is called prompt injection.

Good practice: pin the exact model version and keep a copy of its licence. Test it on your own cases before and after any change. And keep a person, or fixed rules, responsible for any decision that affects people or money.

Your certificate of completion

Enter your name as you want it to appear, then save the certificate as a PDF. In the print window, choose Save as PDF. A certificate is issued once per completion of the course.

Saolix does not record who takes this course, so it cannot verify these certificates. The certificate confirms completion of a free self-paced course and is not an accredited qualification.