Private AI: run models yourself or use a third-party service?
When your business uses an AI assistant, everything you type into it, and every document you attach, goes to whoever runs the model. With a third-party service that is the provider, in its data centres and under its terms. With a model you run yourself, it is your own server. Both can be sensible choices. This article explains what each involves: what you need to run open models yourself, what the main providers' terms say about your data, and what South Africa's privacy law asks when personal information is involved.
Running a model yourself
An open-weight model is one whose trained weights are published, so anyone can download it and run it on their own hardware. Nothing you type leaves the server it runs on. A self-hosted set-up has two parts.
The runtime
The runtime loads the model and answers requests.
- Ollama (MIT licence) is the simplest way to start. It downloads and runs models with one command and offers an API that other tools can use. Its documentation says that when it runs locally it does not see your prompts or data. It also offers optional cloud models, which can be switched off.
- llama.cpp (MIT licence) is the engine many tools are built on. It runs on a wide range of hardware, including ordinary processors, and uses quantisation, a way of storing the model at lower precision, to cut the memory it needs.
The front end
The front end is the chat window your staff use, with accounts and history.
- LibreChat (MIT licence) is a multi-user chat interface that works with local models through Ollama as well as with commercial APIs.
- Open WebUI connects to Ollama and to any service with an OpenAI-compatible API, and adds user roles, groups and search over your own documents. Its licence is a BSD-style licence with an extra clause on keeping its branding for larger deployments, so check it if you plan to rebrand it.
Because both front ends can talk to local and commercial models, a business can start with one and mix the two later.
What hardware it takes
The model sets the hardware. OpenAI's open-weight gpt-oss models, released under the permissive Apache 2.0 licence, are a useful yardstick: the smaller one runs within 16 GB of memory, while the larger one needs a single 80 GB graphics card. Smaller models run on a processor alone, more slowly. A graphics card (GPU) makes a large difference to speed, and Ollama can split a model between the processor and the GPU when it does not fit on the card.
Keep it behind a login
Ollama listens only on the server itself by default and has no login of its own. That default matters. A model exposed directly to the internet can be used by anyone who finds it. Keep the runtime private and put an authenticated front end, served over HTTPS, in front of it.
Open weights are not always open source
"Open" means different things for models. gpt-oss uses Apache 2.0, which lets you use and adapt it freely. Meta's Llama 3.1 licence, by contrast, requires a "Built with Llama" notice, sets naming rules for models derived from it and imposes an acceptable use policy. The Open Source Initiative's definition of open-source AI asks for more than weights: information about the training data and the code used to train and run the model. Read the licence of any model you build on.
Using a third-party service
A commercial AI service gives you the largest models with nothing to run. The question is what happens to your data, and the answer depends on the provider and on whether you use a free or a paid tier. As at October 2026, the providers' own terms say:
| Provider | Training on your data | Retention |
|---|---|---|
| OpenAI API | Not used for training by default since March 2023, unless you opt in | Abuse-monitoring logs kept up to 30 days by default; zero retention only with OpenAI's approval |
| Anthropic API and business plans | Commercial terms say Anthropic may not train models on customer content | Deleted within 30 days by default, longer where content is flagged under its usage policy |
| Google Gemini API, free tier | Content used to improve Google's products, and may be read by human reviewers; Google says not to submit sensitive or personal information | Per Google's terms |
| Google Gemini API, paid tier | Not used to improve products | Per Google's terms |
Consumer chat apps often work differently from the business APIs. Anthropic's consumer Claude plans, for example, may use chats for training if the user allows it. Check the plan your staff actually use, not only the provider's name.
What POPIA asks
If staff paste customer, supplier or employee details into an AI service, the business is processing personal information, and the Protection of Personal Information Act (POPIA) applies.
- Section 72 allows personal information to be sent to a third party in another country only in set cases, for example where the recipient is bound by law or agreement to protection substantially similar to POPIA, or the person concerned has consented.
- Section 21 treats the provider as an operator, so the business needs a written contract requiring it to keep appropriate security safeguards.
A practical rule follows: decide which information may go to which service, write it down, and tell staff. Many businesses use a commercial service for general drafting and a self-hosted model for anything containing personal or confidential information.
Choosing between them
| Self-hosted model | Third-party service | |
|---|---|---|
| Where your data goes | Your own server | The provider |
| Model size and quality | Limited by your hardware | The largest models available |
| Cost | Hardware and running time | Per use or per seat |
| Who maintains it | You, or someone you appoint | The provider |
| POPIA position | Information stays with you | The provider is an operator, so you need a written contract (section 21), and section 72 applies if it is in another country |
Running AI on Noiz
You can install Ollama with LibreChat or Open WebUI yourself on a Noiz Cloud server: a self-managed Ubuntu or Debian server with full root access. Cloud servers are sized by processor, memory and storage, so they suit smaller models running on the processor. If you want to talk through what you have in mind, get in touch. Our help series on using AI with Noiz, at docs.noiz.ie/ai, covers connecting assistants to the services you already run with us.
Sources
Checked 19 October 2026.
Self-hosted tools and models
- Ollama, Ollama FAQ and GPU support
- llama.cpp
- Open WebUI and its licence
- LibreChat
- OpenAI gpt-oss
- Llama 3.1 Community License
- Open Source Initiative: Open Source AI Definition 1.0
Third-party services
- OpenAI: data controls in the OpenAI platform
- Anthropic Commercial Terms
- Anthropic: is my data used for model training? (commercial)
- Anthropic: how long do you store my organisation's data?
- Anthropic: is my data used for model training? (consumer)
- Gemini API Additional Terms of Service
The law
- Protection of Personal Information Act 4 of 2013: sections 21 and 72
Noiz
Note: while some of this content was produced with the help of AI, the final editing has been done by a real human being.
