Baichuan

Baichuan 7B

Name: Baichuan 7B
Rating: 65 (142 reviews)
Author: Baichuan

Legacy

HuggingFace

165.6KDescargas841Me gustaJun 2023Publicado8K tokensContextoApache 2.0Licencia40 BásicoCalidad

Baichuan 7B (7B parameters) requires approximately 13.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 16 GB of VRAM.

Comenzar

— copia y pega para ejecutar en local

Copy-paste commands to run Baichuan 7B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "baichuan-inc/Baichuan-7B" \
  --hf-file "Baichuan-7B-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters7B

Architecturedense

Context8K tokens

Modalitytext

Min RAM2.7 GB

Rec. RAM4.3 GB (Q4_K_M)

LicenseApache 2.0

FamilyBaichuan

✓ Chat✓ Reasoning

About this model

Baichuan-7B是由百川智能开发的一个开源的大规模预训练模型。基于Transformer结构，在大约1.2万亿tokens上训练的70亿参数模型，支持中英双语，上下文窗口长度为4096。在标准的中文和英文权威benchmark（C-EVAL/MMLU）上均取得同尺寸最好的效果。

•在同尺寸模型中Baichuan-7B达到了目前SOTA的水平，参考下面MMLU指标
•Baichuan-7B使用自有的中英文双语语料进行训练，在中文上进行优化，在C-Eval达到SOTA水平
•不同于LLaMA完全禁止商业使用，Baichuan-7B使用更宽松的开源协议，允许用于商业目的
•Among models of the same size, Baichuan-7B has achieved the current state-of-the-art (SOTA) level, as evidenced by the following MMLU metrics
•Baichuan-7B is trained on proprietary bilingual Chinese-English corpora, optimized for Chinese, and achieves SOTA performance on C-Eval

Modelos relacionados

Selecciones rápidas

Mejor económicoB

RX 7600 XT 16GB~$329 — 39 tok/s

Mejor en generalA

RX 7900 XT 20GB~$899 — 98 tok/s

Mejor hardware

Mejores opciones para Baichuan 7B

Ejecutar este modelo

Baichuan 7B on RX 7900 XT 20GB Baichuan 7B on RTX A4500 20GB Baichuan 7B on RTX 3090 24GB

Opciones de cuantización

Estimaciones de VRAM por nivel de cuantización

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	2.7 GB	Low	—
Q3_K_S	3	3.4 GB	Low	—
NVFP4	4	3.9 GB	Medium	—
Q4_K_M	4	4.3 GB	Medium	—
Q5_K_M	5	5.0 GB	High	—
Q6_K	6	5.7 GB	High	—
Q8_0	8	7.5 GB	Very High	—
F16	16	14.3 GB	Maximum	—

Quality benchmarks

Baichuan 7B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro—

GPQA Diamond—

MATH-500—

ARC Challenge42.3%

Source: community · 2023-06-15

Compatibilidad de hardware

Estimaciones de encaje en todo el hardware

Abrir calculadora

Computing compatibility...

Desglose de memoria

Reference: RTX 2060 6GB

Weights4.3 GB

KV Cache7.8 GB

Runtime1.2 GB

Headroom0.6 GB

Preguntas frecuentes

FAQ — Baichuan 7B

Ver también

Guía de cuantización Metodología de puntuación Abrir calculadora