Will It Run AI

MetaMeta

Llama 3.1 70B

Legacy
huggingfaceHuggingFaceollamaOllama
742.1KDownloads921LikesJul 2024Veröffentlicht128K TokenKontextCommunityLizenz73 StarkQualität

Llama 3.1 70B (70B parameters) requires approximately 49.4 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 57 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run Llama 3.1 70B on your machine.

Run

ollama run llama3.1

Quick specs

Parameters70B
Architecturedense
Context128K tokens
Modalitytext
Min RAM27.3 GB
Rec. RAM42.7 GB (Q4_K_M)
LicenseCommunity
FamilyLlama
✓ Chat✓ Reasoning

About this model

Llama 3.1 70B is Meta's high-capability open model with 128K context window. Excels at complex reasoning, multilingual tasks, code generation, and tool use with quality competitive with leading proprietary models.

Verwandte Modelle

Deine Hardware

Erkennung...

Schnellauswahl

Bestes BudgetA
MacBook Pro M4 Max 96GB~$2,499 — 15 tok/s
NVIDIA
Beste GesamtwahlS
NVIDIA H100 80GB~$40,000 — 72 tok/s

Beste Hardware

Top-Empfehlungen für Llama 3.1 70B

NVIDIA
NVIDIA H100 80GBS
80 GB
NVIDIA
NVIDIA H800 80GBS
80 GB
NVIDIA
NVIDIA GH200 96GBS
96 GB
NVIDIA
NVIDIA H20 96GBS
96 GB
NVIDIA
NVIDIA A100 80GBS
80 GB

Dieses Modell ausführen

Llama 3.1 70B on NVIDIA H100 80GBLlama 3.1 70B on NVIDIA H800 80GBLlama 3.1 70B on NVIDIA GH200 96GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
27.3 GB
Low—
Q3_K_S
3
34.3 GB
Low—
NVFP4
4
39.2 GB
Medium—
Q4_K_M
4
42.7 GB
Medium—
Q5_K_M
5
50.4 GB
High—
Q6_K
6
57.4 GB
High—
Q8_0
8
74.9 GB
Very High—
F16
16
143.5 GB
Maximum—

Quality benchmarks

Llama 3.1 70B benchmark scores

Benchmark verified

Coding

SWE-bench Verified—
HumanEval+80.5%
Aider Polyglot—
LiveCodeBench—

Reasoning

MMLU-Pro66.4%
GPQA Diamond46.7%
MATH-50068.0%
ARC Challenge94.8%

General

Chatbot Arena—
IFEval87.5%

Source: official · 2024-07-23

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights42.7 GB
KV Cache4.9 GB
Runtime1.2 GB
Headroom0.6 GB

Häufig gestellte Fragen

FAQ — Llama 3.1 70B

Siehe auch

QuantisierungsleitfadenBewertungsmethodikRechner öffnen