LLaVA

LLaVA 1.6 13B

Name: LLaVA 1.6 13B
Rating: 73 (90 reviews)
Author: LLaVA

当前

HuggingFace

17.9K下载量245点赞Jan 2024发布日期4K tokens上下文Apache 2.0许可证56 良好质量

LLaVA 1.6 13B (13B parameters) requires approximately 21.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 26 GB of VRAM.

快速开始

— 复制粘贴即可本地运行

Copy-paste commands to run LLaVA 1.6 13B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "liuhaotian/llava-v1.6-mistral-7b" \
  --hf-file "llava-v1.6-mistral-7b-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters13B

Architecturedense

Context4K tokens

Modalitytext+vision

Min RAM5.1 GB

Rec. RAM7.9 GB (Q4_K_M)

LicenseApache 2.0

FamilyLLaVA

✓ Vision✓ Chat

About this model

Model type: LLaVA is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. Base LLM: mistralai/Mistral-7B-Instruct-v0.2

•558K filtered image-text pairs from LAION/CC/SBU, captioned by BLIP
•158K GPT-generated multimodal instruction-following data
•500K academic-task-oriented VQA data mixture
•50K GPT-4V data mixture
•40K ShareGPT data

LLaVA 1.6 13B 的最佳选择

RTX 5090 32GBA

32 GB

RTX PRO 4500 Blackwell 32GBA

32 GB

AMD Instinct MI100 32GBA

32 GB

NVIDIA V100 32GBA

32 GB

AMD Instinct MI60 32GBA

32 GB

运行此模型

LLaVA 1.6 13B on RTX 5090 32GB LLaVA 1.6 13B on RTX PRO 4500 Blackwell 32GB LLaVA 1.6 13B on AMD Instinct MI100 32GB

量化选项

各量化级别的 VRAM 估算

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	5.1 GB	Low	—
Q3_K_S	3	6.4 GB	Low	—
NVFP4	4	7.3 GB	Medium	—
Q4_K_M	4	7.9 GB	Medium	—
Q5_K_M	5	9.4 GB	High	—
Q6_K	6	10.7 GB	High	—
Q8_0	8	13.9 GB	Very High	—
F16	16	26.7 GB	Maximum	—

硬件兼容性

全部硬件的适配估算

打开计算器

Computing compatibility...

内存详细分析

Reference: RTX 2060 6GB

Weights7.9 GB

KV Cache12.2 GB

Runtime1.2 GB

Headroom0.6 GB

常见问题

FAQ — LLaVA 1.6 13B

另请参阅

量化指南评分方法打开计算器