LLaVA

LLaVA 1.6 13B

Name: LLaVA 1.6 13B
Rating: 73 (90 reviews)
Author: LLaVA

Aktuell

HuggingFace

17.9KDownloads245LikesJan 2024Veröffentlicht4K TokenKontextApache 2.0Lizenz56 GutQualität

LLaVA 1.6 13B (13B parameters) requires approximately 21.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 26 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run LLaVA 1.6 13B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "liuhaotian/llava-v1.6-mistral-7b" \
  --hf-file "llava-v1.6-mistral-7b-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters13B

Architecturedense

Context4K tokens

Modalitytext+vision

Min RAM5.1 GB

Rec. RAM7.9 GB (Q4_K_M)

LicenseApache 2.0

FamilyLLaVA

✓ Vision✓ Chat

About this model

Model type: LLaVA is an open-source chatbot trained by fine-tuning LLM on multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. Base LLM: mistralai/Mistral-7B-Instruct-v0.2

•558K filtered image-text pairs from LAION/CC/SBU, captioned by BLIP
•158K GPT-generated multimodal instruction-following data
•500K academic-task-oriented VQA data mixture
•50K GPT-4V data mixture
•40K ShareGPT data

Verwandte Modelle

Schnellauswahl

Bestes BudgetA

Mac mini M4 64GB~$1,099 — 10 tok/s

Beste GesamtwahlA

RTX 5090 32GB~$1,999 — 151 tok/s

Beste Hardware

Top-Empfehlungen für LLaVA 1.6 13B

RTX 5090 32GBA

32 GB

RTX PRO 4500 Blackwell 32GBA

32 GB

AMD Instinct MI100 32GBA

32 GB

NVIDIA V100 32GBA

32 GB

AMD Instinct MI60 32GBA

32 GB

Dieses Modell ausführen

LLaVA 1.6 13B on RTX 5090 32GB LLaVA 1.6 13B on RTX PRO 4500 Blackwell 32GB LLaVA 1.6 13B on AMD Instinct MI100 32GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	5.1 GB	Low	—
Q3_K_S	3	6.4 GB	Low	—
NVFP4	4	7.3 GB	Medium	—
Q4_K_M	4	7.9 GB	Medium	—
Q5_K_M	5	9.4 GB	High	—
Q6_K	6	10.7 GB	High	—
Q8_0	8	13.9 GB	Very High	—
F16	16	26.7 GB	Maximum	—

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights7.9 GB

KV Cache12.2 GB

Runtime1.2 GB

Headroom0.6 GB

Häufig gestellte Fragen

FAQ — LLaVA 1.6 13B

Siehe auch

Quantisierungsleitfaden Bewertungsmethodik Rechner öffnen