On this page
Qwen 3.8 27B runs on Linux on your own GPU, with nothing to compile. Off Grid AI Desktop runs it free and offline on Ubuntu 24.04 or newer. It downloads its own NVIDIA runtime, uses Vulkan on AMD cards, and runs Bonsai 2 27B, a 7.2 GB build of the same model, on a machine with no GPU at all.
You need a working graphics driver and nothing else. No CUDA toolkit, no Docker, no Python environment.
Download Off Grid AI Desktop (free, AppImage or .deb, x64)
Which Qwen 3.8 27B file fits your machine?
Qwen 3.8 27B is a dense 27B model that reads text, images and video, under Apache 2.0 with a 262K context (model card). Dense means every parameter works on every word, so the file should fit in your card’s VRAM. Whatever doesn’t fit runs from system RAM, more slowly.
| Your GPU’s VRAM | File to pick | Size |
|---|---|---|
| 32 GB | UD-Q6_K, or Q8_0 with a short context | 22.0 GB / 29.0 GB |
| 24 GB | Qwen 3.8 27B 4-bit (in the app’s list) | 16.5 GB |
| 16 GB | UD-IQ3_XXS or UD-Q3_K_XL (3-bit) | 10.9 GB / 13.1 GB |
| 12 GB | UD-Q2_K_XL (2-bit) | 9.95 GB |
| 8 GB or none | Bonsai 2 27B (in the app’s list) | 7.2 GB |
Sizes are from the Unsloth file list and the app’s catalog, checked 16 November 2026. The vision file adds 0.9 GB (0.63 GB for Bonsai 2) and loads only when you send an image.
A headless Linux box has one advantage over a desktop. If no monitor runs on the card, more of its VRAM is free for the model.
Bonsai 2 27B is Prism ML’s build of Qwen 3.8 27B with weights stored as -1, 0 or +1. Prism ML says it keeps 98.2% of the full model’s benchmark score. The app bundles Prism’s engine on Linux for CPU, Vulkan and NVIDIA, so you don’t build their fork yourself.
What you need
| Minimum | Comfortable | |
|---|---|---|
| Distribution | Ubuntu 24.04 or newer, x64 | Same |
| System RAM | 16 GB (Bonsai 2 27B) | 32 GB, the app’s guidance for the 4-bit file |
| GPU | None | 24 GB NVIDIA or AMD card |
| Driver | None for CPU | Working NVIDIA driver, or Mesa Vulkan for AMD |
| Free disk | about 8 GB | about 18 GB, plus about 7 GB for the NVIDIA download |
There’s no Linux ARM build yet.
How to set it up on Linux
- Download Off Grid AI Desktop as an AppImage or a .deb and install it.
- Check your driver. On NVIDIA,
nvidia-smishould list your card. On AMD,vulkaninfo --summaryshould list it. - For NVIDIA, open Settings → GPU performance. In Use your NVIDIA GPU, choose Download, then Restart app. The app checks the download’s size and SHA-256 before it installs.
- Open Models.
- On a 24 GB card, download Qwen 3.8 27B from Models > Text.
- On 8 GB or no GPU, download Bonsai 2 27B.
- On 12 or 16 GB, type Qwen3.8-27B into Search models…, open the Unsloth upload and pick the file from the table.
- Start a chat and pick the model. In Model settings → Text, leave the backend on Auto, or choose CUDA on NVIDIA.
- Send a prompt, then read the backend’s Now value to see what loaded, and GPU layers under Performance to see how much sits on the card.
A good first test:
Here’s a bash script that’s meant to back up only files changed today, but it copies everything. Find the bug and fix it.
Qwen 3.8 thinks before answering by default, so expect a short pause on the first reply.
You can also watch the card from a terminal while it answers:
watch -n 1 nvidia-smi
Memory use on the card should jump by about the file size when the model loads.
Use it from your other machines
A Linux box with a 24 GB card makes a good Qwen 3.8 server for the house or the team. Off Grid AI Desktop can share its loaded model over your own network, so a laptop can use the 27B without holding it. Our guide to running a local AI server for your team on Linux has the steps.
What goes wrong, and the fix
| Symptom | Check | Fix |
|---|---|---|
| The NVIDIA download isn’t offered | nvidia-smi |
Install or repair the NVIDIA driver first |
| Now shows CPU on an AMD card | vulkaninfo --summary |
Install your distribution’s Mesa Vulkan driver |
| Out of memory when it loads | File size against free VRAM | Pick the next smaller file from the table |
| Fast at first, slow in long chats | Context window in Model settings | Lower it, or start a new chat |
| GPU layers shows few layers on the card | The file spills to system RAM | Use a smaller file so it all fits |
| Bonsai 2 27B won’t load | The app version | Update the app; Bonsai 2 needs the bundled Prism engine |
If Ollama says the model needs more memory than you have, see how to fix “model requires more system memory” on Linux.
Need the full-size model?
Every file above is a compressed Qwen 3.8 27B. The full 16-bit model is about 54 GB, which needs datacenter cards. If you want it at full size, Off Grid AI Boost is a flat-price API for open models on our GPUs, with Qwen 3.8 and DeepSeek V4.1 Flash on the list.
Boost starts at $4.99 a month for the first cohort, with regular prices from $9.99 to $199.99. We don’t store your prompts or train on them. That’s our promise today, and the full policy comes with launch. Usage limits aren’t published yet.
Boost starts in January; pre-order to lock in half price. API keys arrive by January 2027.
FAQ
Do I need the CUDA toolkit or Docker to run Qwen 3.8 27B on Linux? No. Off Grid AI Desktop downloads its own NVIDIA runtime from Settings → GPU performance. You only need a working NVIDIA driver. Docker and the toolkit are for building the app from source.
Which Linux distributions are supported? The release targets Ubuntu 24.04 or newer on x64, as an AppImage or a .deb. Other distributions may run the AppImage, but they aren’t the documented target.
Does Qwen 3.8 27B run on an AMD GPU on Linux? Yes, through Vulkan, which Off Grid AI Desktop includes on Linux. Pick the file that fits your card’s VRAM, the same as on NVIDIA.
Can I run Qwen 3.8 27B on Linux without a GPU? Yes. Use Bonsai 2 27B, a 7.2 GB build of Qwen 3.8 27B in the app’s list. It runs on the CPU with 16 GB of RAM.
Can I share it with other machines on my network? Yes. Off Grid AI Desktop can serve the loaded model to your team over your own network, so one Linux box with a big GPU can answer for everyone.
Related guides
- How to Use Your NVIDIA GPU for Local AI on Linux in 2026
- How to Run Local AI on Linux in 2026
- Is a 16 GB GPU Enough to Run Qwen 3.8 27B in 2026?
- How to Run Qwen 3.8 27B on Windows in 2026
Run nvidia-smi or vulkaninfo --summary, note your VRAM, and download Off Grid AI Desktop to put Qwen 3.8 27B on it.