What if you could have a GitHub Copilot-style assistant inside VS Code that runs entirely on your own machine, costs nothing, and never sends your code to the cloud?
That’s exactly what you get by combining Qwen 3, Ollama and the Qwen Code Companion extension. In this guide we’ll set everything up from scratch, fix the most common setup issues, and generate our first piece of code.
What You Need
- A Windows, macOS or Linux machine with enough RAM/VRAM for the model you pick (more on that below)
- VS Code
- About 15 minutes
Step 1: Install Ollama
Ollama is the easiest way to run open models locally. Download the installer for your OS from the website (on Windows you can also paste the install command into PowerShell) and run it.
Once installed, Ollama runs in the background and exposes a local API that other tools, including VS Code extensions, can talk to.
Step 2: Pick and Download a Qwen Model
Open the Models page on the Ollama website and search for Qwen. You’ll notice two kinds of models:
- Cloud models need you to sign in and pay for a subscription.
- Local models download to your machine and run for free. That’s what we want.
Qwen comes in several sizes. Bigger models are smarter but need more memory, so pick the largest one your machine can comfortably run. In the video I use a recent Qwen 3 release; the larger variants are around 24 GB on disk.
To download a model, copy its ollama run / ollama pull command from the model page and paste it into a terminal:
|
|
When the download finishes, open the Ollama app and send it a quick message. The first reply can be slow while the model loads into memory; after that it’s much faster.
Tip: Ask the model “What kind of tasks can you do?” It will tell you it can write and debug code in Python, JavaScript, HTML, CSS, SQL and more, but that it can’t browse the internet or run code on your machine. For that, you need an agent, which is exactly what we’ll plug it into next.
Step 3: Install the Qwen Code Companion Extension
In VS Code, open the Extensions panel and search for Qwen Code (not “Qwen Coder”). Install Qwen Code Companion.
It’s still in preview, but it already gives you a native IDE experience: chat in the sidebar, and review, edit and accept file changes right inside VS Code. Think of it as a free, local alternative to GitHub Copilot Chat.
Step 4: Connect It to Ollama
Click the new Qwen icon in the activity bar to open the setup pop-up, then:
-
Choose Custom provider.
-
Select OpenAI compatible. Ollama exposes an OpenAI-compatible API, so any tool that speaks OpenAI can use your local models.
-
Set the base URL to:
1http://localhost:11434/v1 -
For the API key, type anything. Ollama doesn’t check it, but the field can’t be empty.
-
For the model name, enter the exact name of the model you downloaded. The safest way is to copy it from the Ollama app or from
ollama list.
Press Enter, confirm, and say “hi”. If you get a reply, you’re connected.
Step 5: Generate Your First App
Open a folder as your workspace and give Qwen a small task, for example:
Write a simple HTML page with a list I can add items to.
Qwen writes the file directly into your workspace. Open it in the browser and it works. Then push it further:
Apply dark mode CSS and make it responsive.
In my test this took about two to three minutes on local hardware, and the result even included a dark/light mode toggle.
Troubleshooting
| Problem | Fix |
|---|---|
| Extension can’t connect | Make sure Ollama is running and the base URL ends with /v1 |
| “Model not found” | The model name must match ollama list exactly, including the tag |
| Very slow responses | Try a smaller Qwen variant, or close other GPU-heavy apps |
| Something else is broken | Use the extension’s Doctor command to diagnose it |
Wrapping Up
You now have a private, free AI coding assistant in VS Code. Next, I’ll push this setup harder by building real apps and showing where Qwen shines and where it struggles. I’ll also try the same model with OpenCode.
If your assistant forgets things in long sessions, read about context size in Ollama. It’s the setting that matters most for local coding agents.