DeepSeek

DeepSeek LLM 67B

Name: DeepSeek LLM 67B
Rating: 55 (51 reviews)
Author: DeepSeek

Legacy

HuggingFace

1.7KDownloads206LikesJan 2024Veröffentlicht4K TokenKontextDeepSeekLizenz26 EinstiegQualität

DeepSeek LLM 67B (67B parameters) requires approximately 48.2 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 56 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run DeepSeek LLM 67B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "deepseek-ai/deepseek-llm-67b-chat" \
  --hf-file "deepseek-llm-67b-chat-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters67B

Architecturedense

Context4K tokens

Modalitytext

Min RAM26.1 GB

Rec. RAM40.9 GB (Q4_K_M)

LicenseDeepSeek

FamilyDeepSeek

✓ Chat✓ Reasoning

About this model

Introducing DeepSeek LLM, an advanced language model comprising 67 billion parameters. It has been trained from scratch on a vast dataset of 2 trillion tokens in both English and Chinese. In order to foster research, we have made DeepSeek LLM 7B/67B Base and DeepSeek LLM 7B/67B Chat open source for the research community.

•Home Page:: DeepSeek
•Repository:: deepseek-ai/deepseek-LLM
•Chat With DeepSeek LLM:: DeepSeek-LLM

Verwandte Modelle

Schnellauswahl

Bestes BudgetB

MacBook Pro M4 Max 96GB~$2,499 — 16 tok/s

Beste GesamtwahlB

NVIDIA H100 80GB~$40,000 — 75 tok/s

Beste Hardware

Top-Empfehlungen für DeepSeek LLM 67B

Dieses Modell ausführen

DeepSeek LLM 67B on NVIDIA H100 80GB DeepSeek LLM 67B on NVIDIA H800 80GB DeepSeek LLM 67B on NVIDIA A100 80GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	26.1 GB	Low	—
Q3_K_S	3	32.8 GB	Low	—
NVFP4	4	37.5 GB	Medium	—
Q4_K_M	4	40.9 GB	Medium	—
Q5_K_M	5	48.2 GB	High	—
Q6_K	6	54.9 GB	High	—
Q8_0	8	71.7 GB	Very High	—
F16	16	137.4 GB	Maximum	—

Quality benchmarks

DeepSeek LLM 67B benchmark scores

Benchmark verified

Coding

SWE-bench Verified—

HumanEval+73.8%

Aider Polyglot—

LiveCodeBench—

Reasoning

MMLU-Pro32.7%

GPQA Diamond8.8%

MATH-5009.3%

ARC Challenge—

General

Chatbot Arena—

IFEval55.9%

Source: official · 2024-01-05

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights40.9 GB

KV Cache5.8 GB

Runtime0.9 GB

Headroom0.6 GB

Häufig gestellte Fragen

FAQ — DeepSeek LLM 67B

Siehe auch

Quantisierungsleitfaden Bewertungsmethodik Rechner öffnen