DeepSeek V4 Flash needs ~189.0 GB VRAM. AMD Instinct MI350X 288GB has 288.0 GB. With NVFP4 quantization, expect ~126 tok/s.
Operating mode
Interactive favors responsiveness, while light API and scale-out lean harder on serving readiness. The fit stays the same, but the recommendation lens changes.
Current mode
Balanced
Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.
Select quantization to explore
Fit status
Runs well
Decode
125.8 tok/s
TTFT
1539 ms
Safe context
1.0M
Memory
189.0 GB / 288.0 GB
This setup is broadly balanced for this model.
No major red flags
This recommendation has enough memory headroom and acceptable estimated speed for the selected workload.
| Workload | Grade | Fit | Decode | TTFT | Context |
|---|---|---|---|---|---|
| Chat | S | Runs well | 125.8 tok/s | 839 ms | 1.0M |
| Coding | S | Runs well | 125.8 tok/s | 1539 ms | 1.0M |
| Agentic Coding | S | Runs well | 125.8 tok/s | 2238 ms | 1.0M |
| Reasoning | S | Runs well | 125.8 tok/s | 1819 ms | 1.0M |
| RAG | S | Runs well | 125.8 tok/s | 2798 ms | 1.0M |
How DeepSeek V4 Flash (284B params) fits at each quantization level on AMD Instinct MI350X 288GB (288.0 GB usable).
| Quant | Bits | VRAM | Quality | Fit |
|---|---|---|---|---|
Q2_K | 2 | 110.8 GB | Low | S87 |
Q3_K_S | 3 | 139.2 GB | Low | S89 |
NVFP4 | 4 |
Copy-paste commands to run DeepSeek V4 Flash on your machine.
Run
docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
--hf-repo "deepseek-ai/DeepSeek-V4-Flash" \
--hf-file "DeepSeek-V4-Flash-NVFP4.gguf" \
-c 4096 -ngl 99Your hardware
| Model | Params | Grade | Decode | Capabilities |
|---|---|---|---|---|
| 397B | S | 78.9 tok/s |
Yes, AMD Instinct MI350X 288GB can run DeepSeek V4 Flash with a S grade (Runs well). Expected decode speed: 125.8 tok/s.
DeepSeek V4 Flash (284B parameters) requires approximately 189.0 GB of memory with NVFP4 quantization.
The recommended quantization for DeepSeek V4 Flash is NVFP4, which balances quality and memory efficiency.
On AMD Instinct MI350X 288GB, DeepSeek V4 Flash achieves approximately 125.8 tokens per second decode speed with a time-to-first-token of 1539ms using NVFP4 quantization.
For coding workloads, DeepSeek V4 Flash on AMD Instinct MI350X 288GB receives a S grade with 125.8 tok/s and 1.0M context.
On AMD Instinct MI350X 288GB, DeepSeek V4 Flash can safely use up to 1.0M tokens of context. The model's official context limit is 1.0M, but available memory constrains the safe maximum.
Paste this snippet into any page to show a live fit card.
<iframe src="https://willitrunai.com/embed/deepseek-v4-flash-on-instinct-mi350x-288gb" width="400" height="180" frameborder="0" style="border:none;border-radius:12px;overflow:hidden;" title="Will It Run AI — fit result"></iframe>
Preview:
159.0 GB |
| Medium |
| S90 |
Q4_K_M | 4 | 173.2 GB | Medium | S90 |
Q5_K_M | 5 | 204.5 GB | High | S90 |
Q6_KBest for your GPU | 6 | 232.9 GB | High | S90 |
Q8_0 | 8 | 303.9 GB | Very High | F0 |
F16 | 16 | 582.2 GB | Maximum | F0 |