DeepSeek

DeepSeek LLM 67B

Name: DeepSeek LLM 67B
Rating: 55 (51 reviews)
Author: DeepSeek

Legado

HuggingFace

1.7KDownloads206CurtidasJan 2024Publicado4K tokensContextoDeepSeekLicença26 InicialQualidade

DeepSeek LLM 67B (67B parameters) requires approximately 48.2 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 56 GB of VRAM.

Comece agora

— copie e cole para rodar localmente

Copy-paste commands to run DeepSeek LLM 67B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "deepseek-ai/deepseek-llm-67b-chat" \
  --hf-file "deepseek-llm-67b-chat-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters67B

Architecturedense

Context4K tokens

Modalitytext

Min RAM26.1 GB

Rec. RAM40.9 GB (Q4_K_M)

LicenseDeepSeek

FamilyDeepSeek

✓ Chat✓ Reasoning

About this model

Introducing DeepSeek LLM, an advanced language model comprising 67 billion parameters. It has been trained from scratch on a vast dataset of 2 trillion tokens in both English and Chinese. In order to foster research, we have made DeepSeek LLM 7B/67B Base and DeepSeek LLM 7B/67B Chat open source for the research community.

•Home Page:: DeepSeek
•Repository:: deepseek-ai/deepseek-LLM
•Chat With DeepSeek LLM:: DeepSeek-LLM

Modelos relacionados

Escolhas rápidas

Melhor custo-benefícioB

MacBook Pro M4 Max 96GB~$2,499 — 16 tok/s

Melhor no geralB

NVIDIA H100 80GB~$40,000 — 75 tok/s

Melhor hardware

Melhores opções para DeepSeek LLM 67B

Rodar este modelo

DeepSeek LLM 67B on NVIDIA H100 80GB DeepSeek LLM 67B on NVIDIA H800 80GB DeepSeek LLM 67B on NVIDIA A100 80GB

Opções de quantização

Estimativas de VRAM por nível de quantização

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	26.1 GB	Low	—
Q3_K_S	3	32.8 GB	Low	—
NVFP4	4	37.5 GB	Medium	—
Q4_K_M	4	40.9 GB	Medium	—
Q5_K_M	5	48.2 GB	High	—
Q6_K	6	54.9 GB	High	—
Q8_0	8	71.7 GB	Very High	—
F16	16	137.4 GB	Maximum	—

Quality benchmarks

DeepSeek LLM 67B benchmark scores

Benchmark verified

Coding

SWE-bench Verified—

HumanEval+73.8%

Aider Polyglot—

LiveCodeBench—

Reasoning

MMLU-Pro32.7%

GPQA Diamond8.8%

MATH-5009.3%

ARC Challenge—

General

Chatbot Arena—

IFEval55.9%

Source: official · 2024-01-05

Compatibilidade de hardware

Estimativas de compatibilidade para todo o hardware

Abrir calculadora

Computing compatibility...

Detalhamento de memória

Reference: RTX 2060 6GB

Weights40.9 GB

KV Cache5.8 GB

Runtime0.9 GB

Headroom0.6 GB

Perguntas frequentes

FAQ — DeepSeek LLM 67B

Veja também

Guia de Quantização Metodologia de Pontuação Abrir calculadora