Stability AI

StableLM 2 12B

Name: StableLM 2 12B
Rating: 49 (90 reviews)
Author: Stability AI

旧版

HuggingFace

263下载量89点赞Apr 2024发布日期4K tokens上下文Stability AI Community许可证4 入门质量

StableLM 2 12B (12B parameters) requires approximately 22.3 GB of VRAM with Q5_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 26 GB of VRAM.

快速开始

— 复制粘贴即可本地运行

Copy-paste commands to run StableLM 2 12B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "stabilityai/stablelm-2-12b-chat" \
  --hf-file "stablelm-2-12b-chat-Q5_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters12B

Architecturedense

Context4K tokens

Modalitytext

Min RAM4.7 GB

Rec. RAM8.6 GB (Q5_K_M)

LicenseStability AI Community

FamilyStableLM

✓ Chat

About this model

`Stable LM 2 12B Chat` is a 12 billion parameter instruction tuned language model trained on a mix of publicly available datasets and synthetic datasets, utilizing Direct Preference Optimization (DPO).

•Developed by: Stability AI
•Model type: StableLM 2 12B Chat model is an auto-regressive language model based on the transformer decoder architecture
•Language(s): English
•Paper: Stable LM 2 Chat Technical Report
•Library: Alignment Handbook

快速推荐

最佳性价比C

Mac mini M4 64GB~$1,099 — 8 tok/s

最佳综合B

RTX 5090 32GB~$1,999 — 103 tok/s

最佳硬件

StableLM 2 12B 的最佳选择

RTX 5090 32GBB

32 GB

RTX PRO 4500 Blackwell 32GBB

32 GB

AMD Instinct MI100 32GBB

运行此模型

StableLM 2 12B on RTX 5090 32GB StableLM 2 12B on RTX PRO 4500 Blackwell 32GB StableLM 2 12B on AMD Instinct MI100 32GB

量化选项

各量化级别的 VRAM 估算

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	4.7 GB	Low	—
Q3_K_S	3	5.9 GB	Low	—
NVFP4	4	6.7 GB	Medium	—
Q4_K_M	4	7.3 GB	Medium	—
Q5_K_M	5	8.6 GB	High	—
Q6_K	6	9.8 GB	High	—
Q8_0	8	12.8 GB	Very High	—
F16	16	24.6 GB	Maximum	—

Quality benchmarks

StableLM 2 12B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro19.3%

GPQA Diamond2.2%

MATH-5005.4%

ARC Challenge65.0%

General

Chatbot Arena—

IFEval40.8%

Source: official · 2024-02-01

硬件兼容性

全部硬件的适配估算

打开计算器

Computing compatibility...

内存详细分析

Reference: RTX 2060 6GB

Weights8.6 GB

KV Cache12.2 GB

Runtime0.9 GB

Headroom0.6 GB

常见问题

FAQ — StableLM 2 12B

另请参阅

量化指南评分方法打开计算器