DeepSeek
DeepSeek V2.5 236B
Aktuell
8.9KDownloads734LikesSep 2024Veröffentlicht131K TokenKontextDeepSeekLizenz80 StarkQualität
DeepSeek V2.5 236B (236B parameters) requires approximately 204.1 GB of VRAM with Q4_K_M quantization. As a Mixture of Experts model with 21B active parameters, it uses less memory than its total parameter count suggests. For the best balance of quality and speed, we recommend hardware with at least 235 GB of VRAM.
Loslegen
— kopieren & einfügen, um lokal auszuführenCopy-paste commands to run DeepSeek V2.5 236B on your machine.
Run
docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
--hf-repo "deepseek-ai/DeepSeek-V2.5" \
--hf-file "DeepSeek-V2.5-Q4_K_M.gguf" \
-c 4096 -ngl 99Quick specs
Parameters236B (21B active)
Architecturemoe (MoE)
Context131K tokens
Modalitytext
Min RAM92 GB
Rec. RAM144 GB (Q4_K_M)
LicenseDeepSeek
FamilyDeepSeek
✓ Chat✓ Reasoning
About this model
Verwandte Modelle
Schnellauswahl
Beste Hardware
Top-Empfehlungen für DeepSeek V2.5 236B
Dieses Modell ausführen
Quantisierungsoptionen
VRAM-Schätzungen nach Quantisierungsstufe
No hardware detected — fit column shows raw VRAM estimates
| Quant | Bits | VRAM | Quality | Fit |
|---|---|---|---|---|
Q2_K | 2 | 92.0 GB | Low | — |
Q3_K_S | 3 | 115.6 GB | Low | — |
NVFP4 | 4 | 132.2 GB | Medium | — |
Q4_K_M | 4 | 144.0 GB | Medium | — |
Q5_K_M | 5 | 169.9 GB | High | — |
Q6_K | 6 | 193.5 GB | High | — |
Q8_0 | 8 | 252.5 GB | Very High | — |
F16 | 16 | 483.8 GB | Maximum | — |
Quality benchmarks
DeepSeek V2.5 236B benchmark scores
Reasoning
MMLU-Pro—
GPQA Diamond—
MATH-50082.8%
ARC Challenge—
Source: official · 2024-09-05
Hardware-Kompatibilität
Eignungsschätzungen für alle Hardware
Computing compatibility...
Speicheraufschlüsselung
Reference: RTX 2060 6GB
Weights144.0 GB
KV Cache58.6 GB
Runtime0.9 GB
Headroom0.6 GB
Häufig gestellte Fragen
FAQ — DeepSeek V2.5 236B
Siehe auch