Will It Run AI

TinyLlamaTinyLlama

TinyLlama 1.1B

レガシー
huggingfaceHuggingFaceollamaOllama
2.3Mダウンロード1.6KいいねDec 2023公開日4K トークンコンテキストApache 2.0ライセンス30 基本品質

TinyLlama 1.1B (1.100000023841858B parameters) requires approximately 2.8 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 4 GB of VRAM.

はじめに

— コピー&ペーストでローカル実行

Copy-paste commands to run TinyLlama 1.1B on your machine.

Run

ollama run tinyllama

Quick specs

Parameters1.1B
Architecturedense
Context4K tokens
Modalitytext
Min RAM0.4 GB
Rec. RAM0.7 GB (Q4_K_M)
LicenseApache 2.0
FamilyTinyLlama
✓ Chat

About this model

The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.

あなたのハードウェア

検出中...

おすすめ

Intel
コスパ最良B
Intel Arc A380 6GB~$139 — 15 tok/s
NVIDIA
総合最良B
GTX 1650 4GB~$149 — 15 tok/s

最適なハードウェア

TinyLlama 1.1Bのおすすめ

NVIDIA
GTX 1650 4GBB
4 GB
NVIDIA
RTX 3050 Ti Laptop 4GBB
4 GB
Intel
Intel Arc A370M 4GBB
4 GB
NVIDIA
RTX 2060 6GBB
6 GB
NVIDIA
RTX 4050 Laptop 6GBB
6 GB

このモデルを実行

TinyLlama 1.1B on GTX 1650 4GBTinyLlama 1.1B on RTX 3050 Ti Laptop 4GBTinyLlama 1.1B on Intel Arc A370M 4GB

量子化オプション

量子化レベル別VRAM推定値

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
0.4 GB
Low—
Q3_K_S
3
0.5 GB
Low—
NVFP4
4
0.6 GB
Medium—
Q4_K_M
4
0.7 GB
Medium—
Q5_K_M
5
0.8 GB
High—
Q6_K
6
0.9 GB
High—
Q8_0
8
1.2 GB
Very High—
F16
16
2.3 GB
Maximum—

Quality benchmarks

TinyLlama 1.1B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro1.1%
GPQA Diamond—
MATH-5001.5%
ARC Challenge33.9%

General

Chatbot Arena—
IFEval6.0%

Source: community · 2024-01-08

ハードウェア互換性

全ハードウェアの適合度推定

カリキュレーターを開く

Computing compatibility...

メモリ内訳

Reference: RTX 2060 6GB

Weights0.7 GB
KV Cache0.3 GB
Runtime1.2 GB
Headroom0.6 GB

よくある質問

FAQ — TinyLlama 1.1B

関連項目

量子化ガイドスコアリングメソドロジーカリキュレーターを開く