Running AI locally means the model's weights sit on your machine and the processing happens there. Nothing leaves your computer, nothing is billed per message, and it works offline. You need two pieces: a runtime app and an open-weight model.
For non-coders the runtime is a normal desktop app. Ollama, LM Studio, Jan, GPT4All, and Msty are all download-and-click. GPT4All even runs on a plain CPU with no graphics card, and its LocalDocs feature chats with your own files without an account.
The hardware rule of thumb, at the common 4-bit quantization: an 8-billion-parameter model wants about an 8 GB GPU or a 16 GB Mac. Small models (SmolLM3, Qwen's 0.6B-to-4B sizes, Gemma 2B and 4B, Phi-4-mini) run on a CPU-only laptop. The giant open flagships (Kimi K2, DeepSeek's full V-series) are server hardware, not laptop material, open weights or not.
Snapshot of 7 June 2026. 31 entries in this group, about 31 with a free option and 31 that can run on your own hardware. Prices and versions change often, and each name links to its official page. The full directory has search and filters across all categories.
How to Choose
Easiest first run:
LM Studio or GPT4All. Fully graphical: search a model, download, chat. GPT4All needs no GPU at all.
The popular default:
Ollama. One installer, a clean desktop app, and most other local-AI tools know how to talk to it.
Chat with your documents:
GPT4All's LocalDocs and AnythingLLM both do private document Q&A entirely on your machine.
Weak or old hardware:
SmolLM3 (3B), Qwen 0.6B-4B, Gemma 2B and 4B, and Phi-4-mini run on ordinary laptops without a dedicated GPU.
For developers:
llama.cpp is the engine under most of these apps, vLLM serves at production scale, and LocalAI mimics the OpenAI API on your own server.
All 31 Run on your computer Entries
| Tool | Maker | What it is | Cost | Where it runs | Hardware |
|---|---|---|---|---|---|
| AnythingLLM | Mintplex Labs | Runtime app: all-in-one desktop app for chat with docs, RAG, and agents. | Free | Cloud or local | Your PC or Mac |
| Cohere Command (open) | Cohere | Open model: enterprise RAG and agent models. Open weights are non-commercial. | Open-weight | Cloud or local | 2x H100. R7B ~8-12GB |
| DeepSeek (open) | DeepSeek | Open model: small distilled versions run locally. The flagships need a data center. | Open-weight | Cloud or local | Distills 1.5-8B local. Flagship data-center |
| EuroLLM | EU consortium | Open model: European multilingual model across all 24 EU official languages. | Open-weight | Cloud or local | 22B ~16-24GB |
| Falcon | TII (UAE) | Open model: UAE family including hybrid-attention and Mamba variants. | Open-weight | Cloud or local | 1-10B laptop to 10GB. 180B server |
| Gemma | Open model: Google open family (separate from Gemini), from mobile to server. | Open-weight | Cloud or local | 2-4B phone/laptop. 27B ~24GB | |
| GLM | Zhipu AI | Open model: Chinese open family, widely served, from local to server scale. | Open-weight | Cloud or local | Smaller ~8-24GB. GLM-5 server |
| gpt-oss | OpenAI | Open model: OpenAI first open LLMs since GPT-2. MoE with 128K context. | Open-weight | Cloud or local | 20b ~16GB. 120b ~80GB GPU |
| GPT4All | Nomic AI | Runtime app: one-click desktop app. Chat with your own files offline. | Free | Local | Your PC or Mac (CPU ok) |
| Granite | IBM | Open model: enterprise small and mid models with transparent training. | Open-weight | Cloud or local | 3B laptop. 8B ~8GB. 30B ~24GB |
| Jan | Menlo Research | Runtime app: open-source ChatGPT-style desktop app for local or cloud models. | Free | Cloud or local | Your PC or Mac |
| Kimi K2 | Moonshot AI | Open model: large MoE strong on agentic coding. Not laptop-runnable. | Open-weight | Cloud or local | ~1T MoE, data-center only |
| KoboldCpp | Community | Runtime: single-file llama.cpp fork with a UI. Popular for writing and roleplay. | Free | Local | Your PC or Mac |
| Llama | Meta | Open model: the most widely deployed open family. Llama 4 adds MoE and multimodal. | Open-weight | Cloud or local | 8B ~8GB GPU / 16GB Mac. 70B ~48GB+ |
| llama.cpp | ggml-org | Runtime: the C/C++ engine behind most local runners. CLI and a server. Developer tool. | Free | Local | Your PC or Mac (CPU or GPU) |
| Llamafile | Mozilla.ai | Runtime: packs a model into one executable file you just run. | Free | Local | Your PC or Mac (no install) |
| LM Studio | LM Studio | Runtime app: fully graphical desktop app that searches and runs open models. | Free | Local | Your PC or Mac |
| LocalAI | Community | Runtime: self-hosted drop-in replacement for the OpenAI and Anthropic APIs. | Free | Local | Your PC or server (CPU ok) |
| Mistral / Mixtral | Mistral AI | Open model: French lab with small dense and large MoE open releases. | Open-weight | Cloud or local | 7B ~6GB. Large MoE on a server |
| MLC LLM | MLC AI | Runtime: compiles models to run across GPUs, phones, and the browser. | Free | Local | PC, Mac, phone, or browser |
| MLX | Apple | Runtime: Apple framework with the fastest token generation on Apple Silicon. | Free | Local | Apple Silicon Mac |
| Msty | Msty | Runtime app: polished desktop app with multi-model compare and knowledge stacks. | Free + paid | Cloud or local | Your PC or Mac |
| Nemotron | NVIDIA | Open model: NVIDIA family tuned for agentic AI. Nano runs locally. | Open-weight | Cloud or local | Nano ~16-24GB. Ultra data-center |
| Ollama | Ollama | Runtime app: popular local runner with a desktop chat app. No terminal needed. | Free | Local | Your PC or Mac |
| OLMo 3 | Allen Institute (Ai2) | Open model: truly-open research models with all code, data, and checkpoints. | Open-weight | Cloud or local | 7B ~8GB. 32B ~24GB |
| Open WebUI | Open WebUI | Runtime: self-hosted web interface for local and cloud models. Multi-user. | Free | Cloud or local | Your server or PC |
| Phi | Microsoft | Open model: small reasoning-strong models built data-quality first. | Open-weight | Cloud or local | 14B ~8-10GB. Mini fits 8GB |
| Qwen | Alibaba | Open model: very active family, dense plus MoE, with strong coding variants. | Open-weight | Cloud or local | 8B ~8GB. 32B ~24GB. 235B server |
| SmolLM3 | Hugging Face | Open model: small, fully-open model with weights, data, and configs released. | Open-weight | Cloud or local | 3B on a laptop / 4-6GB |
| text-generation-webui | oobabooga | Runtime: Gradio web UI for local models with many backends and an API. | Free | Local | Your PC or Mac |
| vLLM | vLLM project | Runtime: high-throughput serving engine. OpenAI-compatible. Production tool. | Free | Cloud or local | Server-class GPU(s) |
Common Questions
What hardware do I need to run AI locally?
For 7-to-8B models at 4-bit quantization, about an 8 GB GPU or a 16 GB Apple Silicon Mac. For 13-to-14B, 10 to 12 GB. For 30-to-32B, a 24 GB GPU or a 32 GB Mac. Small 1-to-4B models run CPU-only. MoE flagships like Kimi K2 need server hardware regardless of being open-weight.
Is local AI actually private?
The model runs entirely on your machine, so prompts and documents stay there, and GPT4All works offline by default. The privacy boundary is the model runtime, not the apps around it: if you connect a local app to a cloud API, that traffic leaves your machine again.
Are local models as good as ChatGPT?
The models a laptop can run are much smaller than frontier cloud models, and it shows on hard tasks. The gap narrows every year, and for summarizing, drafting, and document Q&A a good 8-to-14B local model handles the work well. The trade is capability for privacy, cost, and offline use.
Keep Reading
Verified articles related to this page:
Other model pages: