Will It Run AI

MistralMistral

Mixtral 8x7B

Aktuell
huggingfaceHuggingFaceollamaOllama
889.2KDownloads4.7KLikesDec 2023Veröffentlicht33K TokenKontextApache 2.0Lizenz38 BasisQualität

Mixtral 8x7B (47B parameters) requires approximately 32.4 GB of VRAM with Q4_K_M quantization. As a Mixture of Experts model with 13B active parameters, it uses less memory than its total parameter count suggests. For the best balance of quality and speed, we recommend hardware with at least 38 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run Mixtral 8x7B on your machine.

Run

ollama run mixtral

Quick specs

Parameters47B (13B active)
Architecturemoe (MoE)
Context33K tokens
Modalitytext
Min RAM18.3 GB
Rec. RAM28.7 GB (Q4_K_M)
LicenseApache 2.0
FamilyMixtral
✓ Chat✓ Reasoning

About this model

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer from mistral_common.protocol.instruct.messages import UserMessage from mistral_common.protocol.instruct.request import ChatCompletionRequest

Verwandte Modelle

Deine Hardware

Erkennung...

Schnellauswahl

Bestes BudgetB
Mac mini M4 64GB~$1,099 — 6 tok/s
NVIDIA
Beste GesamtwahlA
RTX PRO 5000 Blackwell 48GB~$4,999 — 81 tok/s

Beste Hardware

Top-Empfehlungen für Mixtral 8x7B

NVIDIA
RTX PRO 5000 Blackwell 48GBA
48 GB
NVIDIA
RTX 6000 Ada 48GBA
48 GB
NVIDIA
NVIDIA L40S 48GBA
48 GB
NVIDIA
NVIDIA L40 48GBA
48 GB
NVIDIA
NVIDIA L20 48GBA
48 GB

Dieses Modell ausführen

Mixtral 8x7B on RTX PRO 5000 Blackwell 48GBMixtral 8x7B on RTX 6000 Ada 48GBMixtral 8x7B on NVIDIA L40S 48GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
18.3 GB
Low—
Q3_K_S
3
23.0 GB
Low—
NVFP4
4
26.3 GB
Medium—
Q4_K_M
4
28.7 GB
Medium—
Q5_K_M
5
33.8 GB
High—
Q6_K
6
38.5 GB
High—
Q8_0
8
50.3 GB
Very High—
F16
16
96.4 GB
Maximum—

Quality benchmarks

Mixtral 8x7B benchmark scores

Benchmark verified

Coding

SWE-bench Verified—
HumanEval+64.0%
Aider Polyglot—
LiveCodeBench—

Reasoning

MMLU-Pro54.0%
GPQA Diamond7.0%
MATH-50047.0%
ARC Challenge—

General

Chatbot Arena—
IFEval62.0%

Source: official · 2023-12-11

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights28.7 GB
KV Cache2.0 GB
Runtime1.2 GB
Headroom0.6 GB

Häufig gestellte Fragen

FAQ — Mixtral 8x7B

Siehe auch

QuantisierungsleitfadenBewertungsmethodikRechner öffnen