Will It Run AI

01.AI01.AI

Yi 34B Chat

Legacy
huggingfaceHuggingFace
77.6KDescargas356Me gustaNov 2023Publicado200K tokensContextoYi SeriesLicencia8 EntradaCalidad

Yi 34B Chat (34B parameters) requires approximately 25.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 30 GB of VRAM.

Comenzar

— copia y pega para ejecutar en local

Copy-paste commands to run Yi 34B Chat on your machine.

Run

lms load Yi-34B-Chat && lms server start

Quick specs

Parameters34B
Architecturedense
Context200K tokens
Modalitytext
Min RAM13.3 GB
Rec. RAM20.7 GB (Q4_K_M)
LicenseYi Series
FamilyYi
✓ Chat

About this model

- they might want nothing more than destruction itself rather then anything else from their quest after immortality (and maybe someone should tell them about modern medicine)? In any event though – one thing remains true regardless : whether or not success comes easy depends entirely upon how much effort we put into conquering whatever challenges lie ahead along with having faith deep down inside ourselves too ;) So let’s get started now shall We?" pipeline_tag: text-generation

  • •🤖 The Yi series models are the next generation of open-source large language models trained from scratch by 01.AI
  • •🙌 Targeted as a bilingual language model and trained on 3T multilingual corpus, the Yi series models become one of the strongest LLM worldwide,...
  • •Yi-34B-Chat model landed in second place (following GPT-4 Turbo), outperforming other LLMs (such as GPT-4, Mixtral, Claude) on the AlpacaEval...
  • •Yi-34B model ranked first among all existing open-source models (such as Falcon-180B, Llama-70B, Claude) in both English and Chinese on...
  • •🙏 (Credits to Llama) Thanks to the Transformer and Llama open-source communities, as they reduce the efforts required to build from scratch and...

Modelos relacionados

Tu hardware

Detectando...

Selecciones rápidas

Mejor económicoC
Mac mini M4 64GB~$1,099 — 8 tok/s
NVIDIA
Mejor en generalB
NVIDIA A100 40GB~$10,000 — 68 tok/s

Mejor hardware

Mejores opciones para Yi 34B Chat

NVIDIA
NVIDIA A100 40GBB
40 GB
NVIDIA
RTX PRO 5000 Blackwell 48GBB
48 GB
MacBook Pro M4 Max 64GBC
64 GB
NVIDIA
RTX 6000 Ada 48GBC
48 GB
Mac Studio M2 Ultra 64GBC
64 GB

Ejecutar este modelo

Yi 34B Chat on NVIDIA A100 40GBYi 34B Chat on RTX PRO 5000 Blackwell 48GBYi 34B Chat on MacBook Pro M4 Max 64GB

Opciones de cuantización

Estimaciones de VRAM por nivel de cuantización

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
13.3 GB
Low—
Q3_K_S
3
16.7 GB
Low—
NVFP4
4
19.0 GB
Medium—
Q4_K_M
4
20.7 GB
Medium—
Q5_K_M
5
24.5 GB
High—
Q6_K
6
27.9 GB
High—
Q8_0
8
36.4 GB
Very High—
F16
16
69.7 GB
Maximum—

Quality benchmarks

Yi 34B Chat benchmark scores

Benchmark verified

Reasoning

MMLU-Pro34.4%
GPQA Diamond11.7%
MATH-5006.3%
ARC Challenge—

General

Chatbot Arena—
IFEval47.0%

Source: community · 2025-01-01

Compatibilidad de hardware

Estimaciones de encaje en todo el hardware

Abrir calculadora

Computing compatibility...

Desglose de memoria

Reference: RTX 2060 6GB

Weights20.7 GB
KV Cache3.7 GB
Runtime0.9 GB
Headroom0.6 GB

Preguntas frecuentes

FAQ — Yi 34B Chat

Ver también

Guía de cuantizaciónMetodología de puntuaciónAbrir calculadora