Alibaba

Qwen 3.5 397B A17B

Name: Qwen 3.5 397B A17B
Rating: 90 (2 reviews)
Author: Alibaba

Frontier

Jun 2025Released131K tokensContextApache 2.0License100 ExceptionalQuality

Qwen 3.5 397B A17B (397B parameters) requires approximately 246.5 GB of VRAM with Q4_K_M quantization. As a Mixture of Experts model with 17B active parameters, it uses less memory than its total parameter count suggests. For the best balance of quality and speed, we recommend hardware with at least 284 GB of VRAM.

Get started

— copy & paste to run locally

Copy-paste commands to run Qwen 3.5 397B A17B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "Qwen/Qwen3.5-397B-A17B-Instruct" \
  --hf-file "Qwen3.5-397B-A17B-Instruct-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters397B (17B active)

Architecturemoe (MoE)

Context131K tokens

Modalitytext

Min RAM154.8 GB

Rec. RAM242.2 GB (Q4_K_M)

LicenseApache 2.0

FamilyQwen

✓ Code✓ Chat✓ Reasoning

About this model

Qwen3.5 397B A17B is the flagship of the Qwen3.5 family — a massive MoE model with frontier-level quality across all tasks.

•Flagship model — top-tier quality across coding, reasoning, math, and agentic tasks
•397B total params with 17B active — strong quality with MoE inference efficiency
•Requires multi-GPU or high-memory setups (150+ GB VRAM at Q4_K_M)

Related models

Inference speed

Qwen 3.5 397B A17B inference speed — tokens per second by GPU & Mac

Estimated decode speed (tokens/sec) for Qwen 3.5 397B A17B at Q4_K_M across popular GPUs and Apple Silicon, including multi-GPU rigs, using the fastest local runtime per device. Fastest is Mac Studio M3 Ultra 256GB at ~7 tok/s. Speed is memory-bandwidth bound, so cards that fit the whole model in VRAM run far faster than ones that offload to system RAM.

GPU / Mac	Memory	Quant	Speed (tok/s)	Fits?
Mac Studio M3 Ultra 256GB	256 GB	Q4_K_M	6.9	Too big
Mac Studio M2 Ultra 128GB	128 GB	Q4_K_M	4.4	Too big
Mac Studio M1 Ultra 128GB	128 GB	Q4_K_M	4.1	Too big
MacBook Pro M4 Max 128GB	128 GB	Q4_K_M	3.2	Too big
MacBook Pro M4 Max 64GB	64 GB	Q4_K_M	3.2	Too big
RTX 5090 32GB	32 GB	Q4_K_M	2.4	Too big
2× RX 7900 XTX 24GB	48 GB	Q4_K_M	2.4	Too big
MacBook Pro M3 Max 64GB	64 GB	Q4_K_M	2.3	Too big
MacBook Pro M1 Max 64GB	64 GB	Q4_K_M	2.1	Too big
RTX 4090 24GB	24 GB	Q4_K_M	2.0	Too big
RTX 4080 Super 16GB	16 GB	Q4_K_M	2.0	Too big
RTX 3090 24GB	24 GB	Q4_K_M	2.0	Too big
RTX 4070 12GB	12 GB	Q4_K_M	2.0	Too big
RTX 3060 12GB	12 GB	Q4_K_M	2.0	Too big
RTX 4060 8GB	8 GB	Q4_K_M	2.0	Too big
RX 7900 XTX 24GB	24 GB	Q4_K_M	2.0	Too big
MacBook Pro M4 Pro 48GB	48 GB	Q4_K_M	2.0	Too big
2× RTX 4090 24GB	48 GB	Q4_K_M	2.0	Too big
2× RTX 3090 24GB	48 GB	Q4_K_M	2.0	Too big
4× RTX 3060 12GB	48 GB	Q4_K_M	2.0	Too big

Estimates for single-stream decoding at Q4_K_M; real tokens/sec varies with prompt length, context, batch size, and runtime build. Prompt processing (prefill) is faster than the decode figures shown here.

Quick picks

Best overallS

AMD Instinct MI350X 288GB~$8,000 — 79 tok/s

Best hardware

Top picks for Qwen 3.5 397B A17B

AMD Instinct MI350X 288GBS

288 GB

AMD Instinct MI325X 256GBA

256 GB

Run this model

Qwen 3.5 397B A17B on AMD Instinct MI350X 288GB Qwen 3.5 397B A17B on AMD Instinct MI325X 256GB

Quantization

Qwen 3.5 397B A17B quantization — VRAM & quality by quant level

How much VRAM Qwen 3.5 397B A17B (397B) needs at each GGUF quant, and whether it fits a 24 GB card (RTX 4090 / 3090). The recommended Q4_K_M uses ~242.2 GB — about 43% less VRAM than Q8_0, at a small quality cost.

Quant	Bits	VRAM (weights)	Quality	Fits 24 GB?
Q2_K	2	154.8 GB	Low	Too big
Q3_K_S	3	194.5 GB	Low	Too big
NVFP4	4	222.3 GB	Medium	Too big
Q4_K_Mrecommended	4	242.2 GB	Medium	Too big
Q5_K_M	5	285.8 GB	High	Too big
Q6_K	6	325.5 GB	High	Too big
Q8_0	8	424.8 GB	Very High	Too big
F16	16	813.8 GB	Maximum	Too big

VRAM shown is quantized weights only; add ~1–3 GB runtime overhead plus KV cache for your context length. Lower quants trade quality for memory — Q4_K_M is the usual sweet spot; Q2/Q3 only when you must fit a bigger model.

Quality benchmarks

Qwen 3.5 397B A17B benchmark scores

Benchmark verified

Coding

SWE-bench Verified76.4%

HumanEval+—

Aider Polyglot—

LiveCodeBench83.6%

Reasoning

MMLU-Pro87.8%

GPQA Diamond88.4%

MATH-500—

ARC Challenge—

General

Chatbot Arena—

IFEval92.6%

Source: official · 2025-06-25

Hardware compatibility

Fit estimates across all hardware

Open calculator

Computing compatibility...

Memory breakdown

Reference: RTX 2060 6GB

Weights242.2 GB

KV Cache2.9 GB

Runtime0.9 GB

Headroom0.6 GB

Frequently asked questions

FAQ — Qwen 3.5 397B A17B

How much VRAM does Qwen 3.5 397B A17B need?

Qwen 3.5 397B A17B (397B parameters) requires approximately 246.5 GB of VRAM with Q4_K_M quantization. Lower quantizations like Q4_K_M use less memory but may reduce quality.

What is the best quantization for Qwen 3.5 397B A17B?

The recommended quantization for Qwen 3.5 397B A17B is Q4_K_M, which offers the best balance between model quality and memory efficiency. Higher quantizations preserve more quality but require more VRAM.

What hardware is recommended for Qwen 3.5 397B A17B?

The top recommended hardware for Qwen 3.5 397B A17B: AMD Instinct MI350X 288GB (score: 97/100), AMD Instinct MI325X 256GB (score: 84/100). These provide the best combination of memory, bandwidth, and compute for running this model locally.

Is Qwen 3.5 397B A17B good for chat?

Yes, Qwen 3.5 397B A17B is well-suited for chat as well as coding, reasoning, agentic. It was designed with these use cases in mind.