Baichuan

Baichuan 7B

Name: Baichuan 7B
Rating: 65 (142 reviews)
Author: Baichuan

Legado

HuggingFace

165.6KDownloads841CurtidasJun 2023Publicado8K tokensContextoApache 2.0Licença40 BásicoQualidade

Baichuan 7B (7B parameters) requires approximately 13.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 16 GB of VRAM.

Comece agora

— copie e cole para rodar localmente

Copy-paste commands to run Baichuan 7B on your machine.

Run

docker run --rm -it ghcr.io/ggerganov/llama.cpp:full \
  --hf-repo "baichuan-inc/Baichuan-7B" \
  --hf-file "Baichuan-7B-Q4_K_M.gguf" \
  -c 4096 -ngl 99

Quick specs

Parameters7B

Architecturedense

Context8K tokens

Modalitytext

Min RAM2.7 GB

Rec. RAM4.3 GB (Q4_K_M)

LicenseApache 2.0

FamilyBaichuan

✓ Chat✓ Reasoning

About this model

Baichuan-7B是由百川智能开发的一个开源的大规模预训练模型。基于Transformer结构，在大约1.2万亿tokens上训练的70亿参数模型，支持中英双语，上下文窗口长度为4096。在标准的中文和英文权威benchmark（C-EVAL/MMLU）上均取得同尺寸最好的效果。

•在同尺寸模型中Baichuan-7B达到了目前SOTA的水平，参考下面MMLU指标
•Baichuan-7B使用自有的中英文双语语料进行训练，在中文上进行优化，在C-Eval达到SOTA水平
•不同于LLaMA完全禁止商业使用，Baichuan-7B使用更宽松的开源协议，允许用于商业目的
•Among models of the same size, Baichuan-7B has achieved the current state-of-the-art (SOTA) level, as evidenced by the following MMLU metrics
•Baichuan-7B is trained on proprietary bilingual Chinese-English corpora, optimized for Chinese, and achieves SOTA performance on C-Eval

Modelos relacionados

Escolhas rápidas

Melhor custo-benefícioB

RX 7600 XT 16GB~$329 — 39 tok/s

Melhor no geralA

RX 7900 XT 20GB~$899 — 98 tok/s

Melhor hardware

Melhores opções para Baichuan 7B

Rodar este modelo

Baichuan 7B on RX 7900 XT 20GB Baichuan 7B on RTX A4500 20GB Baichuan 7B on RTX 3090 24GB

Opções de quantização

Estimativas de VRAM por nível de quantização

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	2.7 GB	Low	—
Q3_K_S	3	3.4 GB	Low	—
NVFP4	4	3.9 GB	Medium	—
Q4_K_M	4	4.3 GB	Medium	—
Q5_K_M	5	5.0 GB	High	—
Q6_K	6	5.7 GB	High	—
Q8_0	8	7.5 GB	Very High	—
F16	16	14.3 GB	Maximum	—

Quality benchmarks

Baichuan 7B benchmark scores

Benchmark verified

Reasoning

MMLU-Pro—

GPQA Diamond—

MATH-500—

ARC Challenge42.3%

Source: community · 2023-06-15

Compatibilidade de hardware

Estimativas de compatibilidade para todo o hardware

Abrir calculadora

Computing compatibility...

Detalhamento de memória

Reference: RTX 2060 6GB

Weights4.3 GB

KV Cache7.8 GB

Runtime1.2 GB

Headroom0.6 GB

Perguntas frequentes

FAQ — Baichuan 7B

Veja também

Guia de Quantização Metodologia de Pontuação Abrir calculadora