Your first model in 10 minutes

Getting a capable model running locally is now a ten-minute job, not a weekend project. Two routes cover almost everyone: the terminal (Ollama) for people who like commands, and a GUI (LM Studio) for people who like progress bars. Both end in the same place — a quantized GGUF on your own disk, answering at interactive speed, with nothing phoning home. Pick one path, do it, then run the health checks at the end.

Before you start

Know three numbers about your machine:

VRAM (or unified memory): 8 GB runs 7–9B models at Q4; 16 GB handles 13–14B; 24 GB reaches the 30B class. Full sizing math in VRAM math.
Memory bandwidth: this, not compute, decides your tok/s — see speed.
Free disk: budget 50–100 GB if you plan to try more than one model.

Path A: the terminal route (Ollama)

Install Ollama — a one-line install script on Linux/macOS, an installer on Windows. It starts a local service and gives you an ollama command. No account, no key.

Pick a mid-size open model. As of this writing the 7–9B class — qwen3:8b or a Llama 3.1 8B-class model — is the right first model: genuinely useful, and it fits an 8 GB GPU or tolerates a CPU. If you write Korean, the Qwen and Gemma families have historically handled Korean text well — but test with a real paragraph of your own, because tokenizers differ (see tokens).

# fetch (~5 GB) and start chatting in one step
ollama run qwen3:8b

You are now in a chat loop. Type a message, press Enter, watch tokens stream out. The day-one slash commands:

/bye          # leave the chat
/show info    # quantization, context length, template in use

Back in your shell, two more:

ollama list   # what's on disk, and how big each one is
ollama ps     # what's loaded in memory right now, and where

ollama ps is your first real instrument. It reports resident size and a CPU/GPU split like 100% GPU or 48%/52% CPU/GPU. Any CPU in the split means the model is not fully on the GPU — a fixable configuration problem, not a hardware limit. See engine knobs.

Example of a healthy readout:

NAME         ID           SIZE     PROCESSOR    UNTIL
qwen3:8b     a1b2c3d4...  5.7 GB   100% GPU     4 minutes from now

SIZE larger than the file on disk is normal: resident size is weights plus compute buffers, not a second copy.

Path B: the GUI route (LM Studio)

Install LM Studio. Open the search tab, type a family name — Qwen3, Llama 3, Gemma 3 — and you get a wall of sizes and quantizations. Two rules keep it simple:

Match parameter count to memory first: an 8B at Q4 is roughly 4–5 GB before the KV cache.
Trust the per-download size hint — it includes headroom, so "fits" actually fits. Prefer Q4_K_M-class quants; the details are in quantization.

Download is a progress bar. Load the model in the chat tab and talk to it; the readout at the bottom shows tokens/sec while it generates — remember that number for the health check. LM Studio can also expose an OpenAI-compatible local server, which becomes useful later in serving.

If the tok/s readout looks low, do not uninstall anything yet — finish the health checks below before concluding something is broken.

What "pulling" actually does

Nothing magical. A model in this ecosystem is one GGUF file (see model formats): a single blob containing every layer, the tokenizer, and metadata. ollama pull and LM Studio's downloader fetch that blob in chunks — that is the layered progress you see — verify a digest, and register the file in a local index. Interrupted downloads resume where they stopped. "Installing a model" is literally downloading a big file and remembering where it lives. No account, no license server, no phone-home.

First-run health checks

Three numbers tell you whether your setup is sane:

Check	Where to look	Healthy looks like
Decode speed	tok/s readout (LM Studio) or timing in Ollama logs	30+ tok/s on a recent discrete GPU; the physics is in speed
Time to first token	How long until the reply starts	Under ~1 s warm for short prompts; long pastes take longer (prefill — see speed)
Memory fit	`ollama ps`, GPU meter, system monitor	Model + KV cache fully resident; the math is in VRAM math

If all three pass, stop tuning and go use the model. Prompt quality dominates everything else at this stage — see prompting.

The three classic first failures

Symptom	Real cause	Fix
Load fails or crashes with out-of-memory	Model does not fit	Drop a quant size (Q4 → Q3) or a model size (8B → 4B); see quantization
Fluent garbage, wrong language, echoing loops	Wrong chat template for the model	The runner must use the model's own template; see prompting and troubleshooting
1–2 tok/s despite having a GPU	Most layers still on CPU	Force full GPU offload; see engine knobs

All three are configuration, not hardware. None of them mean "this machine can't do it."

Where the files live, and disk hygiene

Ollama keeps everything under ~/.ollama — models are stored as content-addressed blobs under ~/.ollama/models. LM Studio uses ~/.lmstudio (older builds: ~/.cache/lm-studio). Both grow alarmingly fast: every pull is 2–20 GB, and deleting from a UI sometimes leaves the blob behind on disk.

Habits that pay off:

Run ollama list monthly and ollama rm anything you have not touched.
Know your models directory before the disk fills. Moving it later is one env var (OLLAMA_MODELS) plus a file copy, not a re-download.
One model per actual task beats five "just in case" downloads — cold storage still cost you the bandwidth.
LM Studio deletes from its index too, but verify the models folder actually shrank — trust, then check.

Related: Inference engines · Engine knobs · Speed · Choosing a model