How to Run AI on Your Own Computer: Ollama in 10 Minutes
AI models can already write texts, explain code, translate and answer questions. But you don’t always want to send your data to the cloud, pay for a subscription or depend on the internet. The good news: you can run a similar AI right on your own computer — for free and without signing up. The easiest way to do it is with Ollama.
What is Ollama?
Ollama is a free program for Windows, macOS and Linux that downloads open language models (Qwen, Gemma, gpt-oss, Llama and others) and runs them locally. All requests are processed on your PC: your data goes nowhere, and once a model is downloaded you don’t need the internet.
What your computer needs
- RAM: at least 8 GB, comfortably 16 GB or more.
- A graphics card is recommended (NVIDIA or AMD): with it, answers come several times faster. Models also run without a GPU, just slower.
- Disk space: 3 to 20 GB per model.
Step 1. Install Ollama
Download the OllamaSetup.exe installer from the official site ollama.com and run it. Or install it with one PowerShell command:
irm https://ollama.com/install.ps1 | iex
Step 2. Pick a model for your computer
| Your computer | Model | Size |
|---|---|---|
| Weak PC, no graphics card | qwen3.5:4b or gemma3:4b |
3–4 GB |
| GPU with 8 GB of memory | qwen3.5:9b |
~7 GB |
| GPU with 12–16 GB | gemma4:12b or gpt-oss:20b |
8–14 GB |
| GPU with 24 GB or more | qwen3.5:27b or gemma4:31b |
17–20 GB |
The rule is simple: the model should fit into the GPU’s memory with a little room to spare. If it doesn’t fit, it will still run, but noticeably slower.
Step 3. Run the model
Open Command Prompt or PowerShell and run (the model downloads on first start):
ollama run qwen3.5:9b
When the >>> prompt appears, just type your question and press Enter. To leave the chat, type /bye.
Useful commands
ollama list ollama pull gemma4:12b ollama rm qwen3.5:4b ollama ps
ollama list— which models are already downloaded;ollama pull— download a model without running it;ollama rm— delete a model and free up space;ollama ps— which models are loaded in memory right now.
What you can do with a local AI
- write and edit texts, emails and descriptions;
- explain and write code, find bugs in scripts;
- translate and summarize documents — without sending them to the internet;
- connect the model to other programs: Ollama works as a local server at
http://localhost:11434.
If your graphics card is weak
Large models need a powerful GPU with 24 GB of memory or more, and buying one just to experiment is expensive. In that case it’s easier to rent a GPU in the cloud for a couple of hours — for example, on SimplePod: it has a ready-made Ollama template, and an RTX 4090 costs less than a dollar per hour. More in my post about renting GPUs on SimplePod.
Where else to rent a GPU
- RunPod — one of the most popular GPU clouds: the cheaper Community Cloud and the more reliable Secure Cloud, plus lots of ready-made templates.
- Vast.ai — a GPU marketplace with private and business hosts: often the lowest prices, but reliability depends on the host.
- Clore.ai — a GPU rental marketplace that also accepts crypto payments.
- Vultr — H100 and other data-center GPUs with hourly billing: pricier than marketplaces, but a large cloud with data centers worldwide.
GPU prices on these platforms change daily, so compare the model you need on several of them before renting.
In short
Install Ollama, pick a model from the table, run it with one command — and you have your own AI that works offline and for free. Start with a small model, and if you like it, move to a bigger one.
The SimplePod, RunPod, Vast.ai, Clore.ai and Vultr links are affiliate links: signing up through them costs you nothing extra, and the blog gets a small reward.
