On this page
Qwen3.8-Flash-Next runs on a 64 GB Mac with llama.cpp, because its 28.8 GB n-gram table can stay on your SSD while the rest of the model, 37.6 GB at 1-bit or 50.7 GB at 2-bit, sits in memory. Under 64 GB, use the cloud: Alibaba’s Qwen3.8-Flash API is the hosted version, at $0.15 per million input tokens. If you want the Qwen 3.8 family fully offline on a 32 GB computer, Off Grid AI Desktop runs Qwen 3.8 27B for free.
Download Off Grid AI Desktop (free, Mac, Windows and Linux)
What Qwen3.8-Flash-Next is
Flash-Next is Alibaba’s preview of the architecture behind Qwen4. Its model card lists 125B parameters with 6B active per token, plus a 51B n-gram embedding table and a 4B multi-token prediction layer. It reads text and images and has a 262K context, which can stretch to 1M.
The n-gram table is why it fits a Mac at all. It’s a lookup table, so the model only touches a few rows per token, and Unsloth’s run guide says you can leave it on the SSD and memory-map it. Unsloth also says that running it from RAM is closer to GPU speed than usual for a model this size, which suits Macs.
It beats the dense 27B on agent coding, 58.7 against 42.2 on DeepSWE 1.1 in Alibaba’s tables. On SWE-bench Pro the gap is small, 62.5 against 61.7, so the 27B is still a fair pick for most chat.
The licence is the Qwen Community License 1.0. You can use it for free, commercially too. A company that sells it as a hosted model API needs a separate licence from Qwen first.
Which build fits your Mac
Every Unsloth build is a set of files where the last one is the 28.8 GB n-gram table. The “in memory” column is the rest.
| Unsloth build | Download | In memory | 32 GB Mac | 64 GB Mac | 128 GB Mac |
|---|---|---|---|---|---|
| 1-bit (UD-IQ1_S) | 66.4 GB | 37.6 GB | Too big | Runs well | Runs well |
| 2-bit (UD-Q2_K_XL) | 79.6 GB | 50.7 GB | Too big | Tight | Runs well |
| 3-bit (UD-Q3_K_XL) | 89.4 GB | 60.6 GB | Too big | Too big | Runs well |
| 4-bit (UD-Q4_K_XL) | 111.3 GB | 82.5 GB | Too big | Too big | Runs well |
Sizes are from the Hugging Face file list, checked 16 November 2026. The ratings use the rule Off Grid AI uses for models on a computer: a file up to 65% of memory runs well, up to 82% is tight. That leaves room for macOS, your apps and the context.
Unsloth’s own table is looser: 32 GB for 1-bit and 64 GB for 4-bit. You can try that, but on 64 GB the 4-bit build has 82.5 GB to place, so expect it to read from the SSD and slow down.
On a 64 GB Mac, start with the 2-bit build and close heavy apps while it runs. If the Mac feels slow, drop to 1-bit. In Unsloth’s tests the 1-bit agrees with the full model’s top answer 77% of the time and the 2-bit 83%.
Set it up on a 64 GB Mac
1. Build llama.cpp
You need Apple’s command line tools and CMake (xcode-select --install, then brew install cmake). Then:
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=OFF
cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server
Metal is on by default on a Mac, so the GPU does the work.
2. Download the 2-bit build
pip install -U "huggingface_hub[cli]"
hf download unsloth/Qwen3.8-Flash-Next-GGUF \
--local-dir unsloth/Qwen3.8-Flash-Next-GGUF \
--include "*UD-Q2_K_XL*"
That’s 79.6 GB, so check your free disk space first.
3. Start the server
./llama.cpp/build/bin/llama-server \
--model unsloth/Qwen3.8-Flash-Next-GGUF/UD-Q2_K_XL/Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00004.gguf \
--temp 1.0 \
--top-p 0.95 \
--top-k 20 \
--min-p 0.0 \
--port 8080
These are Unsloth’s settings for thinking mode. Thinking effort defaults to the highest level. For shorter answers, add --chat-template-kwargs '{"reasoning_effort":"medium"}'.
llama-server has a basic chat page at http://127.0.0.1:8080. Unsloth’s own desktop app can also download and run the model, if you’d rather not use the terminal.
4. Chat with it in Off Grid AI Desktop
Off Grid AI Desktop can use a llama-server as a remote model, with chat history, documents and the thinking switch.
- Open Settings and find Remote model server.
- Under Saved servers, click Add server.
- Type a Server name and the Address:
http://127.0.0.1:8080on the same Mac, or the Mac’s network address from another computer (start the server with--host 0.0.0.0for that). - Click Test connection, pick the model under Text + Vision, then click Save.
The model appears in the chat’s model picker with a Remote tag. Nothing leaves your network.
The app’s own engine is a recent llama.cpp and it downloads split GGUF files, but we haven’t tested Flash-Next inside the app yet. That’s why this guide runs it in llama.cpp.
Not 64 GB? Use the cloud
Qwen3.8-Flash is the hosted version. Alibaba describes it as built on Flash-Next, with a 1M context by default and built-in tools such as web search.
| Option | Price per 1M tokens (in / out) | What happens to your prompts |
|---|---|---|
| Qwen Cloud (Alibaba) | $0.15 / $0.47 | Alibaba’s Model Studio says it never trains on your data, and that it stores data generated by model calls |
| OpenRouter | $0.15 / $0.47, plus 5.5% on credits | Routed to Alibaba, the only provider listed |
| Off Grid AI Boost | Flat price from $4.99 a month | Not stored or trained on; check the model list at launch |
Checked 16 November 2026. Alibaba is the only Flash host on OpenRouter today.
Need it at a flat price?
Off Grid AI Boost is a flat-price API for open models on our own GPUs. The plan list names Qwen 3.8 and DeepSeek V4.1 Flash “and more”, but it doesn’t say which Qwen 3.8 build, so check the Boost model list at launch for Flash-Next. DeepSeek V4.1 Flash is named outright.
Boost starts at $4.99 a month for the first cohort, with regular prices from $9.99 to $199.99. We don’t store your prompts or train on them. That’s our promise today, and the full policy comes with launch. Usage limits for each plan aren’t published yet.
Boost starts in January; pre-order to lock in half price. API keys arrive by January 2027.
The fully offline option for 32 GB
Qwen 3.8 27B is the smallest official Qwen 3.8, and it runs inside Off Grid AI Desktop with no terminal. Its UD-Q4_K_M file is 16.56 GB, which runs well on 32 GB. On a 64 GB Mac, the 8-bit file (29.05 GB) runs well too.
- Open Models, then Text.
- Type Qwen3.8-27B in Search models…. Results from Hugging Face show under More from Hugging Face.
- Pick a file. Each one shows its size and whether it’s Tight or Too big for this computer.
- Choose Download, then Use when it finishes.
For 16 GB, see how to run Qwen 3.8 on a 16 GB laptop.
What goes wrong, and the fix
| Symptom | Check | Fix |
|---|---|---|
| The Mac slows to a crawl | The “in memory” size against your RAM | Close heavy apps, or drop from 2-bit to 1-bit |
| llama.cpp fails to load the model | Whether all four files downloaded | Rerun the hf download line; it resumes |
| Very long thinking before each answer | Reasoning effort | Add '{"reasoning_effort":"medium"}' or "low" |
| Test connection fails in Off Grid AI Desktop | The address and port | Use http://127.0.0.1:8080 on the same Mac; start with --host 0.0.0.0 for other computers |
| Out of disk during download | Free SSD space | The 2-bit build needs 79.6 GB free |
FAQ
Can a 64 GB Mac run Qwen3.8-Flash-Next?
Yes, with llama.cpp and Unsloth’s GGUF. The 28.8 GB n-gram table stays on the SSD, so only 37.6 GB (1-bit) or 50.7 GB (2-bit) has to sit in memory. The 2-bit build is tight on 64 GB, so close heavy apps.
How much memory does Qwen3.8-Flash-Next need?
Unsloth lists 32 GB for its 1-bit build and 64 GB for 4-bit, with the n-gram table on the SSD. By the stricter rule we use to rate models, 1-bit runs well on 64 GB and the 4-bit build needs 128 GB.
Is Qwen3.8-Flash-Next the same as Qwen3.8-Flash?
Close. Qwen3.8-Flash is Alibaba’s hosted API version, built on Flash-Next with a 1M context by default and built-in tools. Flash-Next is the open-weight model you download.
Can I run it in Ollama on a 64 GB Mac?
Not well. Ollama’s smallest Flash-Next download is 105 GB (MLX) and its Q4_K_M is 120 GB. On 64 GB, use Unsloth’s 1-bit or 2-bit GGUF with llama.cpp.
Does Off Grid AI Boost serve Qwen3.8-Flash-Next?
Boost names Qwen 3.8 on its plan list but hasn’t said which Qwen 3.8 build. Check the Boost model list at launch. Boost is a pre-order with API keys by January 2027.
Related guides
- What Mac Do You Need to Run Qwen 3.8 or DeepSeek in 2026?
- How to Run Qwen 3.8 Privately on a 16GB Laptop in 2026
- How to Run DeepSeek V4.1 Flash in 2026 (What It Really Takes, and the Private Option)
- How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Your Mac in 2026
Start with what your Mac already runs: download Off Grid AI Desktop and try Qwen 3.8 27B tonight.