InternLM

InternVL2 8B

Name: InternVL2 8B
Rating: 81 (185 reviews)
Author: InternLM

Aktuell

HuggingFace

468.2KDownloads187LikesJul 2024Veröffentlicht8K TokenKontextMITLizenz76 StarkQualität

InternVL2 8B (8B parameters) requires approximately 8.6 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 10 GB of VRAM.

Loslegen

— kopieren & einfügen, um lokal auszuführen

Copy-paste commands to run InternVL2 8B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "OpenGVLab/InternVL2-8B" \
  --hf-file "InternVL2-8B-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters8B

Architecturedense

Context8K tokens

Modalitytext+vision

Min RAM3.1 GB

Rec. RAM4.9 GB (Q4_K_M)

LicenseMIT

FamilyInternVL

✓ Vision✓ Chat

About this model

We are excited to announce the release of InternVL 2.0, the latest addition to the InternVL series of multimodal large language models. InternVL 2.0 features a variety of instruction-tuned models, ranging from 1 billion to 108 billion parameters. This repository contains the instruction-tuned InternVL2-8B model.

•For more details and evaluation reproduction, please refer to our Evaluation Guide
•We simultaneously use InternVL and VLMEvalKit repositories for model evaluation. Specifically, the results reported for DocVQA, ChartQA, InfoVQA,...

Schnellauswahl

Bestes BudgetA

Intel Arc B570 10GB~$219 — 45 tok/s

Beste GesamtwahlS

RTX 3080 Ti 12GB~$1,199 — 112 tok/s

Beste Hardware

Top-Empfehlungen für InternVL2 8B

Dieses Modell ausführen

InternVL2 8B on RTX 3080 Ti 12GB InternVL2 8B on RTX 5070 12GB InternVL2 8B on RTX 3080 12GB

Quantisierungsoptionen

VRAM-Schätzungen nach Quantisierungsstufe

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	3.1 GB	Low	—
Q3_K_S	3	3.9 GB	Low	—
NVFP4	4	4.5 GB	Medium	—
Q4_K_M	4	4.9 GB	Medium	—
Q5_K_M	5	5.8 GB	High	—
Q6_K	6	6.6 GB	High	—
Q8_0	8	8.6 GB	Very High	—
F16	16	16.4 GB	Maximum	—

Hardware-Kompatibilität

Eignungsschätzungen für alle Hardware

Rechner öffnen

Computing compatibility...

Speicheraufschlüsselung

Reference: RTX 2060 6GB

Weights4.9 GB

KV Cache2.0 GB

Runtime1.2 GB

Headroom0.6 GB

Häufig gestellte Fragen

FAQ — InternVL2 8B

Siehe auch

Quantisierungsleitfaden Bewertungsmethodik Rechner öffnen