Will It Run AI

IBMIBM

Granite 3.1 8B

当前
104.8K下载量168点赞Dec 2024发布日期128K tokens上下文Apache 2.0许可证14 入门质量

Granite 3.1 8B (8B parameters) requires approximately 8.6 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 10 GB of VRAM.

快速开始

— 复制粘贴即可本地运行

Copy-paste commands to run Granite 3.1 8B on your machine.

Run

ollama run granite3.1-dense

Quick specs

Parameters8B
Architecturestate-space
Context128K tokens
Modalitytext
Min RAM3.1 GB
Rec. RAM4.9 GB (Q4_K_M)
LicenseApache 2.0
FamilyGranite
Code Chat

About this model

Model Summary: Granite-3.1-8B-Instruct is a 8B parameter long-context instruct model finetuned from Granite-3.1-8B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.

  • Developers:: Granite Team, IBM
  • GitHub Repository:: ibm-granite/granite-3.1-language-models
  • Website: Granite Docs
  • Paper:: Granite 3.1 Language Models (coming soon)
  • Release Date: December 18th, 2024

相关模型

你的硬件

检测中...

快速推荐

最佳硬件

Granite 3.1 8B 的最佳选择

运行此模型

量化选项

各量化级别的 VRAM 估算

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
3.1 GB
Low
Q3_K_S
3
3.9 GB
Low
NVFP4
4
4.5 GB
Medium
Q4_K_M
4
4.9 GB
Medium
Q5_K_M
5
5.8 GB
High
Q6_K
6
6.6 GB
High
Q8_0
8
8.6 GB
Very High
F16
16
16.4 GB
Maximum

Quality benchmarks

Granite 3.1 8B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro28.2%
GPQA Diamond8.3%
MATH-50022.0%
ARC Challenge62.6%

General

Chatbot Arena
IFEval72.1%

Source: community · 2024-12-01

硬件兼容性

全部硬件的适配估算

打开计算器

Computing compatibility...

内存详细分析

Reference: RTX 2060 6GB

Weights4.9 GB
KV Cache2.0 GB
Runtime1.2 GB
Headroom0.6 GB

常见问题

FAQ — Granite 3.1 8B

另请参阅