Tsinghua/Zhipu

CodeGeeX 4 9B

Name: CodeGeeX 4 9B
Rating: 78 (185 reviews)
Author: Tsinghua/Zhipu

Aktuell

HuggingFace

11.4KDownloads269LikesJul 2024Veröffentlicht131K TokenKontextApache 2.0Lizenz66 GutQualität

CodeGeeX 4 9B (9B parameters) requires approximately 7.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 10 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run CodeGeeX 4 9B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "THUDM/codegeex4-all-9b" \
  --hf-file "codegeex4-all-9b-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters9B

Architecturedense

Context131K tokens

Modalitytext

Min RAM3.5 GB

Rec. RAM5.5 GB (Q4_K_M)

LicenseApache 2.0

FamilyCodeGeeX

About this model

We introduce CodeGeeX4-ALL-9B, the open-source version of the latest CodeGeeX4 model series. It is a multilingual code generation model continually trained on the GLM-4-9B, significantly enhancing its code generation capabilities. Using a single CodeGeeX4-ALL-9B model, it can support comprehensive functions such as code completion and generation, code interpreter, web search, function call, repository-level code Q&A, covering various scenarios of software development. CodeGeeX4-ALL-9B has achieved highly competitive performance on public benchmarks, such as BigCodeBench and NaturalCodeBench.

Schnellauswahl

Bestes BudgetA

Intel Arc B570 10GB~$219 — 41 tok/s

Beste GesamtwahlA

RTX 3080 10GB~$699 — 97 tok/s

Beste Hardware

Top-Empfehlungen für CodeGeeX 4 9B

Dieses Modell ausführen

CodeGeeX 4 9B on RTX 3080 10GB CodeGeeX 4 9B on RTX 3080 Ti 12GB CodeGeeX 4 9B on RTX 3080 12GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	3.5 GB	Low	—
Q3_K_S	3	4.4 GB	Low	—
NVFP4	4	5.0 GB	Medium	—
Q4_K_M	4	5.5 GB	Medium	—
Q5_K_M	5	6.5 GB	High	—
Q6_K	6	7.4 GB	High	—
Q8_0	8	9.6 GB	Very High	—
F16	16	18.5 GB	Maximum	—

Quality benchmarks

CodeGeeX 4 9B benchmark scores

Benchmark verified

Coding

SWE-bench Verified—

HumanEval+82.3%

Aider Polyglot—

LiveCodeBench—

Source: official · 2024-07-10

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights5.5 GB

KV Cache0.6 GB

Runtime1.2 GB

Headroom0.6 GB

Häufig gestellte Fragen

FAQ — CodeGeeX 4 9B

Siehe auch

Quantisierungsleitfaden Bewertungsmethodik Rechner öffnen