AI Compatibility Tier List

RTX 3050 Ti Laptop 4GB

374 AI models ranked by how well they run on your 4 GB VRAM hardware. Best match: BGE M3 (score: 82).

374

Models tested

Can run

4 GB

VRAM

Best tok/s

Tier Distribution

A 7

B 9

C 49

F 309

A: 7B: 9C: 49F: 309

Post on X

A Tier -- Great Fit (7 models)

Excellent performance with comfortable memory margins.

BGE M3

A82

0.57B3.6 GB7 tok/s8K ctx

dense

mxbai Embed Large

A78

0.34B3.5 GB4 tok/s1K ctx

dense

Snowflake Arctic Embed L

A77

0.34B3.5 GB4 tok/s1K ctx

dense

BGE Large EN v1.5

A76

0.34B3.5 GB4 tok/s1K ctx

dense

Nomic Embed Text v1.5

A74

0.14B2.1 GB2 tok/s8K ctx

dense

Ministral 3 3B

A73

3B3.9 GB36 tok/s19K ctx

multimodal

Jina Embeddings v3

A73

0.57B4.4 GB7 tok/s8K ctx

dense

B Tier -- Good Fit (9 models)

Solid performance, may need quantization or reduced context.

Qwen 2.5 Coder 1.5B

B69

1.5B2.6 GB18 tok/s33K ctx

dense

Qwen 3 1.7B

B69

1.7B4.0 GB20 tok/s16K ctx

dense

All MiniLM L6 v2

B66

0.02B1.6 GB2 tok/s0K ctx

dense

Qwen 3.5 2B

B62

2B4.2 GB24 tok/s14K ctx

dense

DeepSeek R1 1.5B

B61

1.5B2.6 GB18 tok/s33K ctx

dense

TinyLlama 1.1B

B60

1.1B2.3 GB13 tok/s4K ctx

dense

Qwen 2.5 1.5B

B59

1.5B2.6 GB18 tok/s67K ctx

dense

Granite 4.1 3B

B58

3B4.4 GB36 tok/s11K ctx

dense

Gemma 3 1B

B56

1B2.3 GB12 tok/s33K ctx

dense

C Tier -- Tight Fit (49 models)

Runs with offloading or significant compromises.

Gemma 2 2B

C54

2B4.1 GB24 tok/s8K ctx

dense

Qwen 3 0.6B

C53

0.6B2.5 GB7 tok/s33K ctx

dense

EXAONE 3.5 2.4B Instruct

C53

2.4B3.0 GB29 tok/s70K ctx

dense

gemma 2b

C53

2B2.8 GB24 tok/s101K ctx

dense

gemma 2 2b it

C53

2B3.2 GB24 tok/s72K ctx

dense

Llama 3.2 1B

C53

1B2.4 GB12 tok/s68K ctx

dense

Gemmasutra Mini 2B v1

C53

2B2.8 GB24 tok/s101K ctx

dense

gemma 2 2b it

C53

2B2.8 GB24 tok/s101K ctx

dense

Qwen 2.5 Coder 0.5B

C52

0.5B1.8 GB6 tok/s131K ctx

dense

Llama 3.2 3B Instruct

C51

3B3.8 GB36 tok/s25K ctx

dense

Qwen 3.5 0.6B

C51

0.6B2.5 GB7 tok/s44K ctx

dense

Qwen2.5 3B Instruct

C51

3B3.5 GB36 tok/s40K ctx

dense

Qwen3.5 4B

C51

4B4.2 GB35 tok/s9K ctx

dense

Llama 3.2 3B Instruct

C51

3B3.5 GB36 tok/s40K ctx

dense

gemma 3 4b it

C51

4B4.2 GB35 tok/s9K ctx

dense

Ministral 3 3B Instruct 2512

C51

3B3.5 GB36 tok/s40K ctx

dense

gemma 3 4b it

C51

4B4.2 GB35 tok/s9K ctx

dense

HELVETE 3B

C51

3B3.5 GB36 tok/s40K ctx

dense

Hermes 3 Llama 3.2 3B

C51

3B3.5 GB36 tok/s40K ctx

dense

stablelm zephyr 3b

C51

3B3.5 GB36 tok/s40K ctx

dense

StarCoder2 3B

C51

3B3.5 GB36 tok/s40K ctx

dense

AI21 Jamba Reasoning 3B

C51

3B3.5 GB36 tok/s40K ctx

dense

stablelm 3b 4e1t

C50

3B3.5 GB36 tok/s40K ctx

dense

ai21labs AI21 Jamba Reasoning 3B

C50

3B3.5 GB36 tok/s40K ctx

dense

ai21labs AI21 Jamba2 3B

C50

3B3.5 GB36 tok/s40K ctx

dense

HelpingAI 3B hindi i1

C50

3B3.5 GB36 tok/s40K ctx

dense

AI21 Jamba2 3B

C50

3B3.5 GB36 tok/s40K ctx

dense

HelpingAI 3B hindi

C50

3B3.5 GB36 tok/s40K ctx

dense

AI21 Jamba2 3B i1

C50

3B3.5 GB36 tok/s40K ctx

dense

stablelm 2 zephyr 1.6b

C50

1.6B2.5 GB19 tok/s147K ctx

dense

logos16v2 stablelm2 1.6b i1

C50

1.6B2.5 GB19 tok/s147K ctx

dense

Qwen2.5 1.5B Instruct

C50

1.5B2.4 GB18 tok/s162K ctx

dense

DeepSeek R1 Distill Qwen 1.5B

C50

1.5B2.4 GB18 tok/s162K ctx

dense

Yi Coder 1.5B Chat

C50

1.5B2.4 GB18 tok/s162K ctx

dense

StarCoder2 3B

C49

3B3.6 GB36 tok/s16K ctx

dense

Yi Coder 1.5B

C49

1.5B2.4 GB18 tok/s162K ctx

dense

Falcon H1 1.5B Instruct

C49

1.5B2.4 GB18 tok/s162K ctx

dense

EXAONE 4.0 1.2B

C48

1.2B2.2 GB14 tok/s224K ctx

dense

Llama 3.2 1B Instruct Q8 0

C47

1B2.2 GB12 tok/s257K ctx

dense

TinyLlama 1.1B Chat v1.0

C47

1.1B2.1 GB13 tok/s252K ctx

dense

TinyLlama 1.1B Chat v0.3

C47

1.1B2.1 GB13 tok/s252K ctx

dense

TinyLlama 1.1B Chat v0.6

C47

1.1B2.1 GB13 tok/s252K ctx

dense

TinyLlama 1.1B Chat v1.0 imatrix

C46

1.1B2.1 GB13 tok/s252K ctx

dense

Llama 3.2 1B Instruct

C46

1B2.0 GB12 tok/s285K ctx

dense

gemma 3 1b it

C46

1B2.0 GB12 tok/s285K ctx

dense

Falcon3 1B Instruct abliterated

C46

1B2.0 GB12 tok/s285K ctx

dense

Qwen 2.5 0.5B

C46

0.5B1.8 GB6 tok/s131K ctx

dense

SmolVLM 500M Instruct

C43

0.5B1.8 GB6 tok/s366K ctx

dense

embeddinggemma 300M

C41

0.3B1.6 GB4 tok/s393K ctx

dense

F Tier -- Can't Run (309 models)

Too heavy for this hardware.

granite embedding 107m multilingual

D38

0.11B1.5 GB2 tok/s422K ctx

dense

Falcon H1 Tiny 90M Instruct

D38

0.09B1.5 GB2 tok/s423K ctx

dense

Llama 3.2 3B

3B4.8 GB36 tok/s8K ctx

dense

Llama 3.1 8B

8B8.1 GB5 tok/s4K ctx

dense

Llama 3.1 70B

70B48.9 GB2 tok/s4K ctx

dense

Llama 3.3 70B

70B48.9 GB2 tok/s4K ctx

dense

Llama 3.2 11B Vision

11B10.0 GB4 tok/s4K ctx

vision

Qwen 2.5 7B

7B6.4 GB9 tok/s4K ctx

dense

Qwen 2.5 14B

14B12.8 GB3 tok/s4K ctx

dense

Qwen 2.5 32B

32B24.7 GB2 tok/s4K ctx

dense

Qwen 2.5 72B

72B50.1 GB2 tok/s4K ctx

dense

Qwen 2.5 Coder 7B

7B6.4 GB9 tok/s4K ctx

dense

Qwen 2.5 Coder 14B

14B12.8 GB3 tok/s4K ctx

dense

Qwen 2.5 Coder 32B

32B24.7 GB2 tok/s4K ctx

dense

Qwen 3 8B

8B8.4 GB4 tok/s4K ctx

dense

Qwen 3 14B

14B12.3 GB3 tok/s4K ctx

dense

Qwen 3 32B

32B24.7 GB2 tok/s4K ctx

dense

Qwen 3 4B

4B5.9 GB16 tok/s4K ctx

dense

Qwen 3 235B A22B

235B147.5 GB2 tok/s4K ctx

moe

Qwen 3 30B A3B

30.5B21.4 GB2 tok/s4K ctx

moe

Qwen3-Coder 30B A3B Instruct

30.5B21.4 GB2 tok/s4K ctx

moe

Qwen3-Coder-Next

80B51.6 GB2 tok/s4K ctx

moe

Qwen3-Coder 480B A35B Instruct

480B297.0 GB2 tok/s4K ctx

moe

Qwen3-VL 30B A3B Instruct

30B21.1 GB4 tok/s4K ctx

moe

Qwen 3.5 4B

4B5.9 GB19 tok/s4K ctx

dense

Qwen 3.5 9B

9B9.0 GB4 tok/s4K ctx

dense

Qwen 3.5 27B

27B20.9 GB2 tok/s4K ctx

dense

Qwen 3.5 35B A3B

35B24.1 GB3 tok/s4K ctx

moe

Qwen 3.6 35B A3B

35B26.8 GB3 tok/s4K ctx

moe

Qwen 3.6 27B

27B18.7 GB2 tok/s4K ctx

dense

Qwen 3.5 122B A10B

122B78.2 GB2 tok/s4K ctx

moe

Qwen 3.5 397B A17B

397B246.3 GB2 tok/s4K ctx

moe

Gemma 2 9B

9B11.9 GB3 tok/s4K ctx

dense

Gemma 2 27B

27B29.0 GB2 tok/s4K ctx

dense

Gemma 3 4B

4B5.8 GB14 tok/s4K ctx

dense

Gemma 3 12B

12B13.5 GB2 tok/s4K ctx

dense

Gemma 3 27B

27B29.0 GB2 tok/s4K ctx

dense

Ministral 8B

8B8.4 GB4 tok/s4K ctx

dense

Ministral 3 8B

8B8.4 GB4 tok/s4K ctx

multimodal

Ministral 3 14B

14B12.3 GB3 tok/s4K ctx

multimodal

Mistral Nemo 12B

12B11.1 GB3 tok/s4K ctx

dense

Mistral Small 24B

24B18.4 GB2 tok/s4K ctx

dense

Mistral Small 3.2 24B

24B18.4 GB2 tok/s4K ctx

vision

Mistral 7B Instruct v0.3

7B7.5 GB6 tok/s4K ctx

dense

Mixtral 8x7B

47B31.9 GB2 tok/s4K ctx

moe

Mixtral 8x22B

141B90.7 GB2 tok/s4K ctx

moe

DeepSeek R1 Distill 8B

8B8.1 GB5 tok/s4K ctx

dense

DeepSeek R1 Distill 14B

14B12.8 GB3 tok/s4K ctx

dense

DeepSeek R1 Distill 32B

32B24.7 GB2 tok/s4K ctx

dense

Phi 3.5 Mini 4B

4B9.6 GB8 tok/s4K ctx

dense

Phi 4 Mini 4B

4B5.2 GB24 tok/s4K ctx

dense

Phi-4 14B

14B12.9 GB3 tok/s4K ctx

dense

Phi-4-reasoning-plus 14B

14.7B13.3 GB3 tok/s4K ctx

dense

GLM-4 9B

9B7.4 GB5 tok/s4K ctx

dense

Granite 3.1 8B

8B8.1 GB6 tok/s4K ctx

state-space

Granite 4.1 8B

8B8.6 GB4 tok/s4K ctx

dense

Granite 4.1 30B

30B23.5 GB2 tok/s4K ctx

dense

OLMo 2 13B

13B11.7 GB3 tok/s4K ctx

dense

Command R 35B

35B25.1 GB2 tok/s4K ctx

dense

Falcon 7B Instruct

7B5.7 GB12 tok/s4K ctx

dense

Falcon 40B Instruct

40B31.9 GB2 tok/s4K ctx

dense

WizardLM 13B

13B21.4 GB3 tok/s4K ctx

dense

StarCoder 7B

7B12.9 GB4 tok/s4K ctx

dense

StarCoder 15B

15B26.7 GB2 tok/s4K ctx

dense

CodeLlama 7B Instruct

7B13.4 GB4 tok/s4K ctx

dense

CodeLlama 13B Instruct

13B21.4 GB3 tok/s4K ctx

dense

StarCoder2 7B

7B6.1 GB10 tok/s4K ctx

dense

StarCoder2 15B

15B13.3 GB2 tok/s4K ctx

dense

MPT-7B-Instruct

7B13.4 GB4 tok/s4K ctx

dense

MPT-30B-Instruct

30B46.3 GB2 tok/s4K ctx

dense

InternLM 7B

7B13.4 GB4 tok/s4K ctx

dense

InternLM Chat 7B

7B13.4 GB4 tok/s4K ctx

dense

DevStral 7B

7B7.5 GB6 tok/s4K ctx

dense

Devstral Small 1.1

24B18.4 GB2 tok/s4K ctx

dense

Devstral Small 2 24B Instruct

24B18.4 GB2 tok/s4K ctx

dense

Devstral 2 123B Instruct

123B81.7 GB2 tok/s4K ctx

dense

Mistral Small 4 119B

119B79.3 GB2 tok/s4K ctx

moe

Leanstral 119B A6B

119B82.7 GB2 tok/s4K ctx

moe

Mistral Large 3

675B418.4 GB2 tok/s4K ctx

moe

Magistral Small 2507

24B18.4 GB2 tok/s4K ctx

dense

Codestral 2 25.08

22B17.2 GB2 tok/s4K ctx

dense

GLM-5

744B474.2 GB2 tok/s4K ctx

moe

GLM-5.1

754B480.3 GB2 tok/s4K ctx

moe

Kimi K2.5

1000B618.7 GB2 tok/s4K ctx

moe

Kimi K2.6

1000B618.7 GB2 tok/s4K ctx

moe

Kimi Linear 48B A3B

48B31.5 GB2 tok/s4K ctx

linear

Magistral 7B

7B7.5 GB6 tok/s4K ctx

dense

Solar 7B

7B8.5 GB5 tok/s4K ctx

dense

Nous Hermes 1.0

9B19.0 GB3 tok/s4K ctx

dense

Nous Dolphin 13B

13B22.9 GB2 tok/s4K ctx

dense

Baichuan 7B

7B13.4 GB4 tok/s4K ctx

dense

Baichuan 13B

13B22.9 GB2 tok/s4K ctx

dense

Cerebras-GPT 13B

13B20.4 GB2 tok/s4K ctx

dense

StableLM 2 12B

12B22.1 GB2 tok/s4K ctx

dense

Zephyr 7B Beta

7B7.5 GB6 tok/s4K ctx

dense

OpenChat 7B

7B7.5 GB6 tok/s4K ctx

dense

Aya Expanse 8B

8B8.1 GB5 tok/s4K ctx

dense

InternLM 20B

20B34.0 GB2 tok/s4K ctx

dense

Yi 34B Chat

34B25.7 GB2 tok/s4K ctx

dense

Dolphin 2.9 8B

8B8.1 GB5 tok/s4K ctx

dense

DeepSeek V3 671B

671B470.2 GB2 tok/s4K ctx

moe

DeepSeek V3.1 671B

671B470.2 GB2 tok/s4K ctx

moe

DeepSeek V4 Pro

1600B865.2 GB2 tok/s4K ctx

moe

DeepSeek V4 Flash

284B160.6 GB2 tok/s4K ctx

moe

DeepSeek R1 671B

671B470.2 GB2 tok/s4K ctx

moe

DeepSeek Coder V2 16B

16B14.4 GB5 tok/s4K ctx

moe

DeepSeek Coder V2 236B

236B203.9 GB2 tok/s4K ctx

moe

DeepSeek V2.5 236B

236B203.9 GB2 tok/s4K ctx

moe

DeepSeek LLM 7B

7B12.9 GB4 tok/s4K ctx

dense

DeepSeek LLM 67B

67B48.0 GB2 tok/s4K ctx

dense

Llama 4 Scout 17B 16E

109B70.7 GB2 tok/s4K ctx

moe

Llama 4 Maverick 17B 128E

400B248.2 GB2 tok/s4K ctx

moe

Command R+ 104B

104B68.2 GB2 tok/s4K ctx

dense

Command A 111B

111B72.9 GB2 tok/s4K ctx

dense

Phi 3 Mini 3.8B

3.8B9.5 GB8 tok/s4K ctx

dense

Phi 3 Medium 14B

14B12.9 GB3 tok/s4K ctx

dense

Yi 1.5 6B

6B5.9 GB12 tok/s4K ctx

dense

Yi 1.5 9B

9B8.3 GB4 tok/s4K ctx

dense

Yi 1.5 34B

34B25.7 GB2 tok/s4K ctx

dense

Yi Coder 9B

9B8.3 GB4 tok/s4K ctx

dense

Codestral 22B

22B17.2 GB2 tok/s4K ctx

dense

Codestral Mamba 7B

7B6.1 GB13 tok/s4K ctx

state-space

Pixtral 12B

12B11.1 GB3 tok/s4K ctx

dense

Pixtral Large 124B

124B82.3 GB2 tok/s4K ctx

dense

Qwen 2.5 VL 7B

7B6.4 GB9 tok/s4K ctx

dense

Qwen 2.5 VL 72B

72B50.1 GB2 tok/s4K ctx

dense

Qwen 2.5 Math 7B

7B6.4 GB9 tok/s4K ctx

dense

Qwen 2.5 Math 72B

72B50.1 GB2 tok/s4K ctx

dense

Nemotron 70B

70B48.9 GB2 tok/s4K ctx

dense

Nemotron Mini 4B

4B5.7 GB20 tok/s4K ctx

dense

LLaVA 1.5 7B

7B13.4 GB4 tok/s4K ctx

dense

LLaVA 1.6 13B

13B21.4 GB3 tok/s4K ctx

dense

MiniCPM-V 2.6 8B

8B8.1 GB5 tok/s2K ctx

dense

InternVL2 8B

8B8.1 GB5 tok/s4K ctx

dense

CogVLM2 19B

19B15.3 GB2 tok/s4K ctx

dense

Vicuna 7B

7B13.4 GB4 tok/s4K ctx

dense

Vicuna 13B

13B21.4 GB3 tok/s4K ctx

dense

OpenHermes 2.5 7B

7B7.5 GB6 tok/s4K ctx

dense

Starling LM 7B

7B7.5 GB6 tok/s4K ctx

dense

Neural Chat 7B

7B7.5 GB6 tok/s4K ctx

dense

Samantha 7B

7B7.5 GB6 tok/s4K ctx

dense

WizardMath 7B

7B7.5 GB6 tok/s4K ctx

dense

Granite Code 8B

8B8.1 GB5 tok/s4K ctx

dense

Granite Code 20B

20B16.7 GB2 tok/s4K ctx

dense

CodeGeeX 4 9B

9B7.4 GB5 tok/s4K ctx

dense

SQLCoder 7B

7B7.5 GB6 tok/s4K ctx

dense

GPT-OSS 20B

21B16.6 GB4 tok/s4K ctx

moe

SmolLM3 3B

3B5.1 GB34 tok/s7K ctx

dense

DeepSeek R1 Distill 7B

7B6.4 GB9 tok/s4K ctx

dense

OLMo 2 32B

32B24.7 GB2 tok/s4K ctx

dense

Nemotron Nano 8B

8B8.1 GB5 tok/s4K ctx

dense

Nemotron Nano 9B v2

9B9.2 GB4 tok/s4K ctx

dense

Nemotron 3 Nano 30B

30B22.0 GB2 tok/s4K ctx

dense

EXAONE 4.0 32B

32B24.7 GB2 tok/s4K ctx

dense

LFM2 24B

24B18.4 GB2 tok/s4K ctx

dense

Llama 3.1 405B

405B256.0 GB2 tok/s4K ctx

dense

Qwen 2.5 3B

3B5.3 GB30 tok/s6K ctx

dense

Qwen 2.5 Coder 3B

3B5.3 GB30 tok/s6K ctx

dense

DeepSeek R1 Distill 70B

70B48.9 GB2 tok/s4K ctx

dense

OLMo 2 7B

7B7.5 GB6 tok/s4K ctx

dense

Granite Code 3B

3B5.6 GB25 tok/s6K ctx

dense

Granite Code 34B

34B25.7 GB2 tok/s4K ctx

dense

GPT-OSS 120B

117B77.6 GB2 tok/s4K ctx

dense

Aya Expanse 32B

32B23.3 GB2 tok/s4K ctx

dense

Mistral Small 3.1 24B

24B18.4 GB2 tok/s4K ctx

dense

Phi-4 Mini Reasoning 4B

3.8B5.1 GB26 tok/s4K ctx

dense

Gemma 4 E2B

5.1B4.9 GB16 tok/s4K ctx

dense

Gemma 4 E4B

8B7.5 GB4 tok/s4K ctx

dense

Gemma 4 26B A4B

25.2B20.3 GB3 tok/s4K ctx

moe

Gemma 4 31B

30.7B34.7 GB2 tok/s4K ctx

dense

DeepSeek V3.2

671B411.1 GB2 tok/s4K ctx

moe

Nemotron Cascade 2 30B A3B

30B22.5 GB4 tok/s4K ctx

moe

MiniMax M2.7

230B145.4 GB2 tok/s4K ctx

moe

Solar Open 100B

100B74.0 GB2 tok/s4K ctx

dense

Baichuan M2 32B Q4 K M

32B24.6 GB2 tok/s4K ctx

dense

aya expanse 8b

8B7.1 GB6 tok/s4K ctx

dense

baichuan inc Baichuan M2 32B

32B24.6 GB2 tok/s4K ctx

dense

Codestral 22B v0.1

22B17.3 GB2 tok/s4K ctx

dense

cognitivecomputations Dolphin Mistral 24B Venice Edition

24B18.8 GB2 tok/s4K ctx

dense

cognitivecomputations Dolphin3.0 R1 Mistral 24B

24B18.8 GB2 tok/s4K ctx

dense

DiscoPOP zephyr 7b gemma

7B6.4 GB8 tok/s4K ctx

dense

dolphin 2.9.4 llama3.1 8b

8B7.1 GB6 tok/s4K ctx

dense

Dolphin3.0 Llama3.1 8B

8B7.1 GB6 tok/s4K ctx

dense

glm 4 9b chat 1m

9B7.8 GB4 tok/s4K ctx

dense

HelpingAI2 9B

9B7.8 GB4 tok/s4K ctx

dense

internlm JanusCoder 14B

14B11.5 GB3 tok/s4K ctx

dense

internlm2 5 20b chat

20B15.8 GB2 tok/s4K ctx

dense

Meta Llama 3.1 8B Instruct

8B7.1 GB6 tok/s4K ctx

dense

NousResearch Hermes 4 14B

14B11.5 GB3 tok/s4K ctx

dense

starcoder2 15b instruct v0.1

15B12.2 GB2 tok/s4K ctx

dense

Yi 1.5 6B Chat

6B5.7 GB12 tok/s4K ctx

dense

Yi 1.5 9B Chat

9B7.8 GB4 tok/s4K ctx

dense

exaone 3.0 7.8b it

7.8B7.0 GB6 tok/s4K ctx

dense

stablelm 2 1 6b chat imatrix

6B5.7 GB12 tok/s4K ctx

dense

Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored

48B36.2 GB2 tok/s4K ctx

dense

Dolphin3.0 Llama3.1 8B

8B7.1 GB6 tok/s4K ctx

dense

Mamba Codestral 7B v0.1

7B6.4 GB11 tok/s4K ctx

dense

Qwen3.5 9B Uncensored HauhauCS Aggressive

9B7.8 GB4 tok/s4K ctx

dense

HelpingAI2 6B

6B5.7 GB12 tok/s4K ctx

dense

granite 8b code instruct 4k

8B7.1 GB6 tok/s4K ctx

dense

internlm2 limarp chat 20b

20B15.8 GB2 tok/s4K ctx

dense

Phi 4 reasoning vision 15B

15B12.2 GB2 tok/s4K ctx

dense

Codestral 22B v0.1 IMat

22B17.3 GB2 tok/s4K ctx

dense

internlm2 math plus 7b IMat

7B6.4 GB8 tok/s4K ctx

dense

openchat 3.6 8b 20240522 IMat

8B7.1 GB6 tok/s4K ctx

dense

EXAONE 3.5 7.8B Instruct

7.8B7.0 GB6 tok/s4K ctx

dense

EXAONE 4.0 32B

32B24.6 GB2 tok/s4K ctx

dense

K EXAONE 236B A23B

236B172.9 GB2 tok/s4K ctx

dense

vntl llama3 8b v2

8B7.1 GB6 tok/s4K ctx

dense

Codestral 22B v0.1

22B17.3 GB2 tok/s4K ctx

dense

DeepSeek R1 0528 Qwen3 8B

8B7.1 GB6 tok/s4K ctx

dense

EXAONE 3.5 7.8B Instruct

7.8B7.0 GB6 tok/s4K ctx

dense

Qwen3.5 35B A3B

35B26.8 GB2 tok/s4K ctx

dense

Qwen3.5 9B

9B7.8 GB4 tok/s4K ctx

dense

starcoder2 15b instruct v0.1

15B12.2 GB2 tok/s4K ctx

dense

DeepSeek R1 0528 Qwen3 8B

8B7.1 GB6 tok/s4K ctx

dense

gemma 3 12b it

12B10.0 GB3 tok/s4K ctx

dense

gemma 3 27b it

27B20.9 GB2 tok/s4K ctx

dense

Llama 3.3 70B Instruct

70B52.2 GB2 tok/s4K ctx

dense

Llama 3 8B Instruct 32k v0.1

8B7.1 GB6 tok/s4K ctx

dense

Meta Llama 3.1 8B Instruct

8B7.1 GB6 tok/s4K ctx

dense

Meta Llama 3 8B Instruct

8B7.1 GB6 tok/s4K ctx

dense

Mistral 7B Instruct v0.3

7B6.4 GB8 tok/s4K ctx

dense

Mistral Small 24B Instruct 2501

24B18.8 GB2 tok/s4K ctx

dense

mistral small 3.1 24b instruct 2503 hf

24B18.8 GB2 tok/s4K ctx

dense

Yi 1.5 6B Chat

6B5.7 GB12 tok/s4K ctx

dense

Yi Coder 9B Chat

9B7.8 GB4 tok/s4K ctx

dense

zephyr 7b beta Mistral 7B Instruct v0.2

7B6.4 GB8 tok/s4K ctx

dense

aya expanse 32b heretic MPOA i1

32B24.6 GB2 tok/s4K ctx

dense

aya expanse 8b orthogonal heretic

8B7.1 GB6 tok/s4K ctx

dense

aya expanse 8b orthogonal heretic i1

8B7.1 GB6 tok/s4K ctx

dense

Baichuan M3 235B

235B172.2 GB2 tok/s4K ctx

dense

Baichuan M3 235B i1

235B172.2 GB2 tok/s4K ctx

dense

BaichuanMed OCR 72B i1

72B53.7 GB2 tok/s4K ctx

dense

blossom v1 baichuan 7b i1

7B6.4 GB8 tok/s4K ctx

dense

blossom v3 baichuan2 7b i1

7B6.4 GB8 tok/s4K ctx

dense

CodeNinja 1.0 OpenChat 7B i1

7B6.4 GB8 tok/s4K ctx

dense

Codestral 21B Pruned i1

21B16.6 GB2 tok/s4K ctx

dense

Codestral 22B v0.1 i1

22B17.3 GB2 tok/s4K ctx

dense

Codestral RAG 19B Pruned i1

19B15.1 GB2 tok/s4K ctx

dense

Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i1

24B18.8 GB2 tok/s4K ctx

dense

dolphin v2 8b abliterated i1

8B7.1 GB6 tok/s4K ctx

dense

EXAONE 3.5 7.8B Instruct i1

7.8B7.0 GB6 tok/s4K ctx

dense

HelpingAI 15B i1

15B12.2 GB2 tok/s4K ctx

dense

HelpingAI 9B 200k i1

9B7.8 GB4 tok/s4K ctx

dense

HelpingAI 9B i1

9B7.8 GB4 tok/s4K ctx

dense

HelpingAI2.5 10B i1

10B8.6 GB3 tok/s4K ctx

dense

HelpingAI2.5 5B i1

5B4.9 GB20 tok/s4K ctx

dense

HelpingAI2 6B i1

6B5.7 GB12 tok/s4K ctx

dense

HelpingAI2 9B i1

9B7.8 GB4 tok/s4K ctx

dense

Helply 10.2b chat i1

10.2B8.7 GB4 tok/s4K ctx

dense

internlm2 5 1 8b chat i1

8B7.1 GB6 tok/s4K ctx

dense

internlm2 5 7b chat i1

7B6.4 GB8 tok/s4K ctx

dense

internlm2 math plus 20b i1

20B15.8 GB2 tok/s4K ctx

dense

internlm3 8b instruct abliterated i1

8B7.1 GB6 tok/s4K ctx

dense

MD Judge v0 2 internlm2 7b i1

7B6.4 GB8 tok/s4K ctx

dense

OpenChat 3.5 7B Qwen v2.0 i1

7B6.4 GB8 tok/s4K ctx

dense

OpenChat 3.5 7B Starling v2.0 i1

7B6.4 GB8 tok/s4K ctx

dense

SOLAR 10.7B v1.0

10.7B9.1 GB3 tok/s4K ctx

dense

solar finalised finetuned Model 10.7B i1

10.7B9.1 GB3 tok/s4K ctx

dense

Solar Open 100B i1

100B74.0 GB2 tok/s4K ctx

dense

Solar Open 69B REAP i1

69B51.5 GB2 tok/s4K ctx

dense

starcoder2 15b i1

15B12.2 GB2 tok/s4K ctx

dense

Yi 9B Coder i1

9B7.8 GB4 tok/s4K ctx

dense

zephyr 7b dpo full i1

7B6.4 GB8 tok/s4K ctx

dense

zephyr 7b gemma sft african ultrachat 100k

7B6.4 GB8 tok/s4K ctx

dense

Hermes 2 Pro Llama 3 8B

8B7.1 GB6 tok/s4K ctx

dense

Hermes 2 Pro Mistral 7B

7B6.4 GB8 tok/s4K ctx

dense

Hermes 3 Llama 3.1 8B

8B7.1 GB6 tok/s4K ctx

dense

Hermes 4.3 36B

36B27.5 GB2 tok/s4K ctx

dense

Nous Hermes 2 Mistral 7B DPO

7B6.4 GB8 tok/s4K ctx

dense

starcoder2 7b

7B6.4 GB8 tok/s4K ctx

dense

jointpreferences mistral 7b sft helpful

7B6.4 GB8 tok/s4K ctx

dense

OpenSafetyLab MD Judge v0 2 internlm2 7b

7B6.4 GB8 tok/s4K ctx

dense

stabilityai japanese stablelm base gamma 7b

7B6.4 GB8 tok/s4K ctx

dense

stabilityai japanese stablelm instruct beta 70b

70B52.2 GB2 tok/s4K ctx

dense

Codestral 22B v0.1

22B17.3 GB2 tok/s4K ctx

dense

Mistral 7B Instruct v0.3

7B6.4 GB8 tok/s4K ctx

dense

StarCoder2 15B

15B12.2 GB2 tok/s4K ctx

dense

StarCoder2 7B

7B6.4 GB8 tok/s4K ctx

dense

baichuan2 7b chat

7B6.4 GB8 tok/s4K ctx

dense

GGUF SOLARized GraniStral 14B 1902 YeAM HCT

14B11.5 GB3 tok/s4K ctx

dense

GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV

14B11.5 GB3 tok/s4K ctx

dense

stablelm 2 zephyr 1 6b

6B5.7 GB12 tok/s4K ctx

dense

Qwen3 8B DeepSeek v3.2 Speciale Distill

8B7.1 GB6 tok/s4K ctx

dense

japanese stablelm instruct gamma 7B

7B6.4 GB8 tok/s4K ctx

dense

Llama 2 7B Chat

7B6.4 GB8 tok/s4K ctx

dense

Mistral 7B Instruct v0.2

7B6.4 GB8 tok/s4K ctx

dense

Nous Hermes 2 SOLAR 10.7B

10.7B9.1 GB3 tok/s4K ctx

dense

SOLAR 10.7B Instruct v1.0 uncensored

10.7B9.1 GB3 tok/s4K ctx

dense

zephyr 7B alpha

7B6.4 GB8 tok/s4K ctx

dense

zephyr 7B beta

7B6.4 GB8 tok/s4K ctx

dense

Falcon H1 7B Instruct

7B6.4 GB8 tok/s4K ctx

dense

Falcon H1R 7B

7B6.4 GB8 tok/s4K ctx

dense

falcon mamba 7b instruct Q4 K M

7B6.4 GB11 tok/s4K ctx

dense

DeepSeek R1 0528 Qwen3 8B

8B7.1 GB6 tok/s4K ctx

dense

DeepSeek R1 Distill Llama 8B

8B7.1 GB6 tok/s4K ctx

dense

DeepSeek R1 Distill Qwen 14B

14B11.5 GB3 tok/s4K ctx

dense

Falcon H1R 7B

7B6.4 GB8 tok/s4K ctx

dense

gemma 3 27b it

27B20.9 GB2 tok/s4K ctx

dense

Mistral Small 3.2 24B Instruct 2506

24B18.8 GB2 tok/s4K ctx

dense

Qwen3.5 122B A10B

122B75.4 GB2 tok/s4K ctx

dense

Qwen3.5 27B

27B20.9 GB2 tok/s4K ctx

dense

Qwen3.5 35B A3B

35B26.8 GB2 tok/s4K ctx

dense

Qwen3.5 397B A17B

397B290.0 GB2 tok/s4K ctx

dense

Qwen3.5 9B

9B7.8 GB4 tok/s4K ctx

dense

speechless zephyr code functionary 7b

7B6.4 GB8 tok/s4K ctx

dense

llava llama 3 8b v1 1

8B7.1 GB6 tok/s4K ctx

dense

cognitivecomputations Dolphin Mistral 24B Venice Edition

24B18.8 GB2 tok/s4K ctx

dense

Compare with...

See How Other Hardware Stacks Up

Check tier lists for similar GPUs and Apple Silicon configs

Intel Arc A370M 4GB

4 GB VRAM

View tier list

GTX 1650 4GB

4 GB VRAM

View tier list

RTX 2060 6GB

6 GB VRAM

View tier list

RTX 4050 Laptop 6GB

6 GB VRAM

View tier list

Full RTX 3050 Ti Laptop 4GB page Compare this hardware