TinyLlama

TinyLlama 1.1B

Name: TinyLlama 1.1B
Rating: 53 (196 reviews)
Author: TinyLlama

旧版

HuggingFace

Ollama

2.3M下载量1.6K点赞Dec 2023发布日期4K tokens上下文Apache 2.0许可证30 基础质量

TinyLlama 1.1B (1.100000023841858B parameters) requires approximately 2.8 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 4 GB of VRAM.

快速开始

— 复制粘贴即可本地运行

Copy-paste commands to run TinyLlama 1.1B on your machine.

Run

ollama run tinyllama

Quick specs

Parameters1.1B

Architecturedense

Context4K tokens

Modalitytext

Min RAM0.4 GB

Rec. RAM0.7 GB (Q4_K_M)

LicenseApache 2.0

FamilyTinyLlama

✓ Chat

About this model

The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.

快速推荐

最佳性价比B

Intel Arc A380 6GB~$139 — 15 tok/s

最佳综合B

GTX 1650 4GB~$149 — 15 tok/s

最佳硬件

TinyLlama 1.1B 的最佳选择

GTX 1650 4GBB

4 GB

RTX 3050 Ti Laptop 4GBB

运行此模型

TinyLlama 1.1B on GTX 1650 4GB TinyLlama 1.1B on RTX 3050 Ti Laptop 4GB TinyLlama 1.1B on Intel Arc A370M 4GB

量化选项

各量化级别的 VRAM 估算

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	0.4 GB	Low	—
Q3_K_S	3	0.5 GB	Low	—
NVFP4	4	0.6 GB	Medium	—
Q4_K_M	4	0.7 GB	Medium	—
Q5_K_M	5	0.8 GB	High	—
Q6_K	6	0.9 GB	High	—
Q8_0	8	1.2 GB	Very High	—
F16	16	2.3 GB	Maximum	—

Quality benchmarks

TinyLlama 1.1B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro1.1%

GPQA Diamond—

MATH-5001.5%

ARC Challenge33.9%

General

Chatbot Arena—

IFEval6.0%

Source: community · 2024-01-08

硬件兼容性

全部硬件的适配估算

打开计算器

Computing compatibility...

内存详细分析

Reference: RTX 2060 6GB

Weights0.7 GB

KV Cache0.3 GB

Runtime1.2 GB

Headroom0.6 GB

常见问题

FAQ — TinyLlama 1.1B

另请参阅

量化指南评分方法打开计算器