IBMIBM

Granite 3.1 8B

Aktuell
104.8KDownloads168LikesDec 2024Veröffentlicht128K TokenKontextApache 2.0Lizenz14 EinstiegQualität

Granite 3.1 8B (8B parameters) requires approximately 8.6 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 10 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run Granite 3.1 8B on your machine.

Run

ollama run granite3.1-dense

Quick specs

Parameters8B
Architecturestate-space
Context128K tokens
Modalitytext
Min RAM3.1 GB
Rec. RAM4.9 GB (Q4_K_M)
LicenseApache 2.0
FamilyGranite
Code Chat

About this model

Model Summary: Granite-3.1-8B-Instruct is a 8B parameter long-context instruct model finetuned from Granite-3.1-8B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.

  • Developers:: Granite Team, IBM
  • GitHub Repository:: ibm-granite/granite-3.1-language-models
  • Website: Granite Docs
  • Paper:: Granite 3.1 Language Models (coming soon)
  • Release Date: December 18th, 2024

Verwandte Modelle

Deine Hardware

Erkennung...

Schnellauswahl

Beste Hardware

Top-Empfehlungen für Granite 3.1 8B

Dieses Modell ausführen

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
3.1 GB
Low
Q3_K_S
3
3.9 GB
Low
NVFP4
4
4.5 GB
Medium
Q4_K_M
4
4.9 GB
Medium
Q5_K_M
5
5.8 GB
High
Q6_K
6
6.6 GB
High
Q8_0
8
8.6 GB
Very High
F16
16
16.4 GB
Maximum

Quality benchmarks

Granite 3.1 8B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro28.2%
GPQA Diamond8.3%
MATH-50022.0%
ARC Challenge62.6%

General

Chatbot Arena
IFEval72.1%

Source: community · 2024-12-01

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights4.9 GB
KV Cache2.0 GB
Runtime1.2 GB
Headroom0.6 GB

Häufig gestellte Fragen

FAQ — Granite 3.1 8B

Siehe auch