How to Use Ollama: Run AI Locally Without Paying for Subscriptions

Table of Contents

• What Ollama is (and what it isn't)
• Why running AI locally can pay off for your business

  • Your data stays on the machine
  • Cost stops growing with usage
  • It works offline and without depending on a provider
  • When it doesn't make sense
• What hardware you need to run a local LLM

  • CPU, NVIDIA GPU or Apple Silicon
  • What quantization is
  • How to calculate the memory you need
• Which model to install first
• How to install Ollama step by step

  • On macOS
  • On Windows
  • On Linux
• Essential Ollama commands
• How to customize a model with a Modelfile
• How to use Ollama's local API (port 11434)

  • Connecting it to editors and coding agents
  • Connecting it to Open WebUI, Docker or other machines on the network
• Ollama vs LM Studio vs Jan: which one to choose
• Work out how much you save
• The takeaway

The short answer: Ollama is a free program that downloads open AI models and runs them on your own computer. You install it in a couple of minutes, pull a model with one command, and get a chat plus a local API that sends nothing to third-party servers. What decides how well it runs isn't your processor: it's how much memory you have to load the model.

This guide covers the whole path: why a small business or a startup would want to run AI locally, what hardware you need, how to install and configure Ollama, how to connect it to other tools, and when LM Studio or Jan is the better fit. If what you're after is which model to install, that's covered in detail in Best Ollama models in 2026.

What Ollama is (and what it isn't)

Ollama is an engine that packages, downloads and serves language models locally. It installs as a background service and gives you three ways to use it:

• The terminal, with commands like ollama run and ollama pull.
• A desktop app with a chat window on Mac and Windows (Linux is terminal only).
• An HTTP API at http://localhost:11434, compatible with the OpenAI format, for connecting editors, automations or your own apps.

What it isn't: Ollama isn't a model. It's the program that runs third-party models: Qwen, Gemma, gpt-oss, Llama, DeepSeek and others. Answer quality depends on the model you pick and on your hardware.

Why running AI locally can pay off for your business

For a small business or an early-stage startup, local AI isn't a tech fad. It solves three concrete problems that the subscription model has.

Your data stays on the machine

When you use Ollama locally, the prompt and the answer are processed on your computer. There's no provider retaining conversations and no data-use policy to read under a magnifying glass. That matters when you work with proprietary source code, financial models, contracts or customer lists, which is exactly what you shouldn't paste into a public chat (I cover that in what not to share with ChatGPT).

Local privacy has a limit: it protects data while it sits on your machine. If the disk isn't encrypted or half the team shares the computer, the risk moves somewhere else; it doesn't disappear.

Cost stops growing with usage

With a commercial API you pay per token: the more you automate, the more you pay. With a local model the cost is already sunk (the hardware), and each extra query costs whatever it draws in electricity. No hourly usage caps and no surprises on the end-of-month bill.

That doesn't make it free. You have to count the hardware, the setup time, and the fact that a local 9B model doesn't perform like a frontier one. I compare the scenarios with numbers in real AI prices in 2026.

It works offline and without depending on a provider

A local model keeps answering when the internet drops or the provider has an outage. For batch work (sorting hundreds of emails, extracting data from invoices, summarizing internal documents) that means you can leave it running without worrying about quotas or downtime.

When it doesn't make sense

Be honest about the task. If you need long reasoning, an agent coding across a large repository, or answers where a mistake costs money, a frontier cloud model is still better. The sensible strategy for most people is mixed: local for volume and sensitive data, cloud for the hard stuff. It's the same logic as not depending on a single AI provider.

What hardware you need to run a local LLM

The key question isn't which processor you have, but how much memory the model can use and how fast data moves within that memory.

CPU, NVIDIA GPU or Apple Silicon

• CPU only (x86): fine for testing and for small 2B to 4B models. It works, but slowly, because system RAM has little bandwidth.
• NVIDIA GPU: the fastest option in tokens per second, as long as the model fits in the card's VRAM. If it doesn't, Ollama splits layers between GPU and CPU, and speed drops sharply.
• Apple Silicon (M1 to M4): memory is unified, so the GPU can use most of the machine's RAM. A MacBook Pro with 64 or 128 GB loads models that would need several graphics cards on a PC. It's slower than a high-end NVIDIA card on the same model, but it reaches sizes the PC can't.

What quantization is

Quantization lowers the precision of the model's weights so it takes less memory. The levels you'll see:

• FP16: full precision, 2 bytes per parameter. Maximum fidelity, lots of memory.
• Q8: 8 bits, 1 byte per parameter. Half the memory and an almost unnoticeable quality loss.
• Q4_K_M: 4 bits, about 0.5 bytes per parameter. A quarter of FP16's memory and the practical standard for local use. Quality drops a little, but you barely notice it in everyday tasks.

Almost every Ollama model downloads in Q4 by default. In the models guide I explain the QAT and MLX variants, which improve this trade-off.

How to calculate the memory you need

A quick rule:

Memory (GB) ≈ parameters in billions × bytes per weight + 2 to 4 GB for context

An 8B model in Q4 takes about 4 GB of weights, plus room for the context (the KV cache): it starts with 6 GB and runs comfortably with 8 GB.

Model size	Quantization	Minimum memory	Comfortable memory
7B / 8B	Q4_K_M	6 GB	8 GB
7B / 8B	Q8	9 GB	12 GB
7B / 8B	FP16	16 GB	20 GB
14B	Q4_K_M	12 GB	16 GB
32B	Q4_K_M	20 GB	24 GB
70B	Q4_K_M	40 GB	48–64 GB

These are estimates: real usage grows with the context window you configure. If a model runs slowly, the first thing to check is whether it fits entirely on the GPU (I show you how below).

Which model to install first

You don't need to start with the biggest one. As a starting point in 2026:

• 8 GB of RAM: qwen3.5:4b.
• 16 GB: qwen3.5:9b, the best balance of quality and size.
• 24 GB of VRAM or a 32 GB Mac: qwen3.5:27b, or qwen3-coder:30b for programming.

The full comparison by memory, with Gemma 4, gpt-oss and embedding models, is in Best Ollama models in 2026. If you're torn between the two main families, see Qwen vs Gemma.

How to install Ollama step by step

On macOS

Download the installer from ollama.com/download, drag the app to Applications and open it. The app keeps the service running and adds the ollama command to the terminal.

On Windows

Download and run OllamaSetup.exe from the same page. It doesn't need WSL: it runs natively, detects compatible NVIDIA and AMD GPUs and sits as an icon in the system tray.

On Linux

A single command installs the service:

curl -fsSL https://ollama.com/install.sh | sh

As with any script you pipe into sh, you can download it and read it before running it. On Linux, Ollama runs as a systemd service.

To check it works:

ollama --version

Essential Ollama commands

Command	What it does
ollama run <model>	Downloads the model if you don't have it and opens a chat in the terminal
ollama pull <model>	Downloads the model without opening the chat
ollama list	Lists downloaded models, with size and date
ollama ps	Shows models loaded in memory and whether they run on GPU or CPU
ollama show <model>	Shows model details: parameters, quantization and context
ollama rm <model>	Deletes a model to free up disk space

A real first run:

ollama pull qwen3.5:9b
ollama run qwen3.5:9b

Inside the chat, /bye exits. Check ollama ps the first time: if the processor column says something like 40%/60% CPU/GPU, the model doesn't fit entirely on your GPU. Pick a smaller one or reduce the context.

How to customize a model with a Modelfile

A Modelfile is a text file that defines how a model behaves: the system prompt, the temperature, the context size. It gives you a fixed assistant for a task without repeating instructions in every chat.

FROM qwen3.5:9b

# Temperature: 0 = more deterministic answers, 1 = more creative
PARAMETER temperature 0.2

# Context window in tokens
PARAMETER num_ctx 8192

SYSTEM """
You are a senior software architect focused on security.
You answer directly and with technical precision, no filler, in Markdown.
"""

Save it as Modelfile and package it:

ollama create my-architect -f ./Modelfile
ollama run my-architect

About context: by default, Ollama sizes the context window to your VRAM, and below 24 GB it starts at 4,096 tokens. That falls short for summarizing long documents or running coding agents. Raise it with num_ctx in the Modelfile, with the OLLAMA_CONTEXT_LENGTH variable for the whole server, or from the app's settings. More context uses more memory, so check ollama ps again after raising it.

How to use Ollama's local API (port 11434)

Ollama exposes an HTTP server at http://localhost:11434 in two flavors:

• Native API: /api/generate and /api/chat.
• OpenAI-compatible API: /v1, so you can use Ollama with any SDK or tool that already talks to OpenAI.

A test call:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3.5:9b",
  "prompt": "Summarize in three points why running AI locally cuts costs",
  "stream": false
}'

With the OpenAI-compatible API, the only change in your code is the base URL:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
response = client.chat.completions.create(
    model="qwen3.5:9b",
    messages=[{"role": "user", "content": "Classify this email: ..."}],
)
print(response.choices[0].message.content)

The SDK requires the api_key, but Ollama ignores it.

Connecting it to editors and coding agents

Tools like OpenCode accept a custom provider: you point the base URL at http://localhost:11434/v1 and pick the model. That gives you completions and coding help without your code leaving your machine. I walk through the setup in how to install and configure OpenCode. For agents, use a model built for code and raise the context to 32K or more.

Connecting it to Open WebUI, Docker or other machines on the network

By default, Ollama only listens on 127.0.0.1. If you connect an interface like Open WebUI from a Docker container or from another machine on the network, set these environment variables on the service:

• OLLAMA_HOST=0.0.0.0 so it accepts connections from outside localhost.
• OLLAMA_ORIGINS with the allowed origins, if a web app calls the API from the browser.

Watch out for security: Ollama's API has no authentication. With 0.0.0.0, anyone on your network can use your models. Only do it on trusted networks or behind an authenticating proxy, and never open port 11434 to the internet. Avoid OLLAMA_ORIGINS="*" outside of testing.

Ollama vs LM Studio vs Jan: which one to choose

All three run models locally. The difference is what they're built for.

Criterion	Ollama	LM Studio	Jan
Focus	Service and API for plugging AI into other tools	Desktop app for discovering, testing and comparing models	Open-source chat app, a local alternative to ChatGPT
Interface	Terminal and API; simple chat app on Mac and Windows	Full graphical interface with visual settings	Graphical chat interface
Local API	Always on, port 11434	You start the server from the app	OpenAI-compatible server, enabled in settings
Models	Ollama catalog plus GGUF import	Downloads GGUF straight from Hugging Face	Model downloads and connections to other backends
Best for	Automations, headless servers, editors and agents	Comparing models and quantizations, testing prompts	Non-technical users who want a private chat

My take: Ollama if you're going to plug AI into something (an editor, a script, a workflow). LM Studio if you want to explore models before deciding. Jan if what you need is a private chat for someone who won't open a terminal. They're not mutually exclusive.

Work out how much you save

Moving part of the work to a local model only makes sense if the numbers add up. Two free Crea Cosas tools help you see it:

1. Profit margin calculator: subtract what you pay today in subscriptions and tokens from your project costs and see how much the margin improves.
2. Business Model Canvas: carry the change into the cost structure block. A cost that used to grow with every customer becomes a fixed cost you amortize.

Count the hardware and the setup time. If the monthly saving doesn't pay for the machine within a reasonable time, stay in the cloud and revisit it when volume grows.

The takeaway

Running AI locally is no longer a lab experiment. With Ollama, a 16 GB machine and a well-chosen model, you get a private assistant with no monthly fee and an API you can connect to your tools. It won't replace frontier models for everything, but it can take a good chunk of the daily volume, and the data that shouldn't leave the company, off your plate. Start with a small model, measure whether it helps, and grow from there.