AllenAI

OLMo 2 13B

Name: OLMo 2 13B
Rating: 76 (149 reviews)
Author: AllenAI

Aktuell

HuggingFace

Nov 2024Veröffentlicht33K TokenKontextApache 2.0Lizenz64 GutQualität

OLMo 2 13B (13B parameters) requires approximately 11.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 14 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run OLMo 2 13B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "allenai/OLMo-2-13B-Instruct" \
  --hf-file "OLMo-2-13B-Instruct-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters13B

Architecturedense

Context33K tokens

Modalitytext

Min RAM5.1 GB

Rec. RAM7.9 GB (Q4_K_M)

LicenseApache 2.0

FamilyOLMo

✓ Chat

About this model

OLMo 2 13B is AI2's fully open research model with transparent training data and methodology. Designed for reproducible research with competitive performance on reasoning and general knowledge tasks.

Verwandte Modelle

Schnellauswahl

Bestes BudgetA

RX 7600 XT 16GB~$329 — 23 tok/s

Beste GesamtwahlA

RTX 4080 Super 16GB~$999 — 87 tok/s

Beste Hardware

Top-Empfehlungen für OLMo 2 13B

RTX 5080 Laptop 16GBA

RTX 4070 Ti Super 16GBA

16 GB

RTX 5070 Ti 16GBA

16 GB

Dieses Modell ausführen

OLMo 2 13B on RTX 5080 Laptop 16GB OLMo 2 13B on RTX 4080 Super 16GB OLMo 2 13B on RTX 5080 16GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	5.1 GB	Low	—
Q3_K_S	3	6.4 GB	Low	—
NVFP4	4	7.3 GB	Medium	—
Q4_K_M	4	7.9 GB	Medium	—
Q5_K_M	5	9.4 GB	High	—
Q6_K	6	10.7 GB	High	—
Q8_0	8	13.9 GB	Very High	—
F16	16	26.7 GB	Maximum	—

Quality benchmarks

OLMo 2 13B benchmark scores

Benchmark verified

General

Chatbot Arena—

IFEval82.6%

Source: official · 2024-11-24

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights7.9 GB

KV Cache2.4 GB

Runtime0.9 GB

Headroom0.6 GB

Häufig gestellte Fragen

FAQ — OLMo 2 13B

Siehe auch

Quantisierungsleitfaden Bewertungsmethodik Rechner öffnen