TinyLlama

TinyLlama 1.1B

Name: TinyLlama 1.1B
Rating: 53 (196 reviews)
Author: TinyLlama

Legacy

HuggingFace

Ollama

2.3MDownloads1.6KLikesDec 2023Veröffentlicht4K TokenKontextApache 2.0Lizenz30 BasisQualität

TinyLlama 1.1B (1.100000023841858B parameters) requires approximately 2.8 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 4 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run TinyLlama 1.1B on your machine.

Run

ollama run tinyllama

Quick specs

Parameters1.1B

Architecturedense

Context4K tokens

Modalitytext

Min RAM0.4 GB

Rec. RAM0.7 GB (Q4_K_M)

LicenseApache 2.0

FamilyTinyLlama

✓ Chat

About this model

The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.

Schnellauswahl

Bestes BudgetB

Intel Arc A380 6GB~$139 — 15 tok/s

Beste GesamtwahlB

GTX 1650 4GB~$149 — 15 tok/s

Beste Hardware

Top-Empfehlungen für TinyLlama 1.1B

GTX 1650 4GBB

4 GB

RTX 3050 Ti Laptop 4GBB

Dieses Modell ausführen

TinyLlama 1.1B on GTX 1650 4GB TinyLlama 1.1B on RTX 3050 Ti Laptop 4GB TinyLlama 1.1B on Intel Arc A370M 4GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	0.4 GB	Low	—
Q3_K_S	3	0.5 GB	Low	—
NVFP4	4	0.6 GB	Medium	—
Q4_K_M	4	0.7 GB	Medium	—
Q5_K_M	5	0.8 GB	High	—
Q6_K	6	0.9 GB	High	—
Q8_0	8	1.2 GB	Very High	—
F16	16	2.3 GB	Maximum	—

Quality benchmarks

TinyLlama 1.1B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro1.1%

GPQA Diamond—

MATH-5001.5%

ARC Challenge33.9%

General

Chatbot Arena—

IFEval6.0%

Source: community · 2024-01-08

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights0.7 GB

KV Cache0.3 GB

Runtime1.2 GB

Headroom0.6 GB

Häufig gestellte Fragen

FAQ — TinyLlama 1.1B

Siehe auch

Quantisierungsleitfaden Bewertungsmethodik Rechner öffnen