What Is Context Size in Ollama? Context Window Explained for Beginners

Sat, 17 Oct 2026 18:00:00 +0530

Have you ever wondered what happens when you bump Ollama’s context size from 8K to 16K, 32K or even 256K? Does the AI get smarter? Does it remember more? And why does your memory usage jump every time you move that slider?

In this guide I’ll explain context size in plain language, show a real before-and-after test with a coding agent, and help you choose the right setting for your machine.

What Is Context Size?

Context size is the amount of information a model can work with at one time. Think of it as the AI’s working memory.

Say you’re asking a local model to help you build an app. You give it your requirements, some code, an error message and the earlier conversation. The model needs all of that in view to give a good answer. The context size decides how much of it fits.

When the conversation grows past that limit, the oldest information falls out of the window and the model simply can’t see it anymore.

Tokens, Not Words

Context size is measured in tokens, not words. A token can be:

a whole word,
part of a word,
a punctuation mark,
or a piece of code.

So 16K tokens is not 16,000 words. Code in particular tends to use a lot of tokens, which is why coding agents need bigger contexts than simple chat.

The Numbers You See in Ollama

Open Ollama’s Settings and you’ll find a Context length slider. The default is 4K, and you can raise it through 8K, 16K, 32K, 64K and 128K, up to 256K.

You can also set it per session or per model:

1
2

# inside an interactive `ollama run` session
/set parameter num_ctx 32768

1
2

# in a Modelfile
PARAMETER num_ctx 32768

A Real Test: 8K vs 64K

To see the difference, I connected Qwen to VS Code (here’s how to set that up) and gave the agent the same task twice:

Build a simple expense tracker where I can add an expense with amount, category, description and date.

At 8K, the agent couldn’t even get going. It stalled while using its tools and the turn was stopped. Coding agents send a large system prompt and tool definitions along with your request, so a small window leaves very little room to actually work.

At 64K, with a fresh session and the same prompt, it planned the app and started writing the classes straight away.

For writing larger amounts of code or more complex logic, 128K gives the agent even more breathing room.

The point of this test isn’t whether the generated code is perfect. It’s to show how much context size changes what a local model can do.

Why Bigger Contexts Use More Memory

The model has to keep its working memory in RAM or VRAM while it generates. A larger context means a larger memory footprint, which can make responses slower, or make the model fail to load on smaller machines.

How to Choose

Use case	Suggested context
Quick questions, short chats	4K–8K
Chatting about a document or longer conversation	16K–32K
Coding agents (VS Code, OpenCode, Claude Code-style tools)	64K and up
Large codebases or long agent sessions	128K+, if your hardware allows

Start with the smallest value that works for your task, then increase it if the model starts “forgetting” or agents stop mid-task.

Recap

Context size is the AI’s working memory, measured in tokens.
A larger context lets the model work with more information but needs more memory.
A smaller context saves resources but limits how much fits in one request.
For coding agents, the default 4K is far too small. Go to 64K or higher.

Tokens on The Code Street

What Is Context Size in Ollama? Context Window Explained for Beginners