CodeLlama 7B Instruct

Name: CodeLlama 7B Instruct
Rating: 72 (142 reviews)
Author: Meta

Legado

44.3KDownloads258CurtidasAug 2023Publicado16K tokensContextoCommunityLicença56 BomQualidade

CodeLlama 7B Instruct (7B parameters) requires approximately 13.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 16 GB of VRAM.

Comece agora

— copie e cole para rodar localmente

Copy-paste commands to run CodeLlama 7B Instruct on your machine.

Run

lms load CodeLlama-7b-Instruct-hf && lms server start

Quick specs

Parameters7B

Architecturedense

Context16K tokens

Modalitycode

Min RAM2.7 GB

Rec. RAM4.3 GB (Q4_K_M)

LicenseCommunity

FamilyCodeLlama

✓ Code

About this model

Code Llama is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 34 billion parameters. This is the repository for the 7B instruct-tuned version in the Hugging Face Transformers format. This model is designed for general code synthesis and understanding. Links to other models can be found in the index at the bottom.

•[x] Code completion
•[x] Instructions / chat
•[ ] Python specialist

Modelos relacionados

Escolhas rápidas

Melhor custo-benefícioA

RX 7600 XT 16GB~$329 — 39 tok/s

Melhor no geralA

RX 7900 XT 20GB~$899 — 98 tok/s

Melhor hardware

Melhores opções para CodeLlama 7B Instruct

Rodar este modelo

CodeLlama 7B Instruct on RX 7900 XT 20GB CodeLlama 7B Instruct on RTX A4500 20GB CodeLlama 7B Instruct on RTX 3090 24GB

Opções de quantização

Estimativas de VRAM por nível de quantização

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	2.7 GB	Low	—
Q3_K_S	3	3.4 GB	Low	—
NVFP4	4	3.9 GB	Medium	—
Q4_K_M	4	4.3 GB	Medium	—
Q5_K_M	5	5.0 GB	High	—
Q6_K	6	5.7 GB	High	—
Q8_0	8	7.5 GB	Very High	—
F16	16	14.3 GB	Maximum	—

Quality benchmarks

CodeLlama 7B Instruct benchmark scores

Benchmark verified

Coding

SWE-bench Verified—

HumanEval+34.8%

Aider Polyglot—

LiveCodeBench—

Source: official · 2023-08-24

Compatibilidade de hardware

Estimativas de compatibilidade para todo o hardware

Abrir calculadora

Computing compatibility...

Detalhamento de memória

Reference: RTX 2060 6GB

Weights4.3 GB

KV Cache7.8 GB

Runtime1.2 GB

Headroom0.6 GB

Perguntas frequentes

FAQ — CodeLlama 7B Instruct

Veja também

Guia de Quantização Metodologia de Pontuação Abrir calculadora