Will It Run AI

MosaicMLMosaicML

MPT-7B-Instruct

旧版
huggingfaceHuggingFace
May 2023发布日期8K tokens上下文Apache 2.0许可证40 基础质量

MPT-7B-Instruct (7B parameters) requires approximately 13.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 16 GB of VRAM.

快速开始

— 复制粘贴即可本地运行

Copy-paste commands to run MPT-7B-Instruct on your machine.

Run

lms load mpt-7b-instruct && lms server start

Quick specs

Parameters7B
Architecturedense
Context8K tokens
Modalitytext
Min RAM2.7 GB
Rec. RAM4.3 GB (Q4_K_M)
LicenseApache 2.0
FamilyMPT
✓ Chat✓ Reasoning

About this model

MPT-7B Instruct is MosaicML's instruction-tuned model with a commercially permissive license. Supports 65K context with ALiBi positional encoding for efficient long-document processing.

相关模型

你的硬件

检测中...

快速推荐

最佳性价比B
RX 7600 XT 16GB~$329 — 39 tok/s
最佳综合A
RX 7900 XT 20GB~$899 — 98 tok/s

最佳硬件

MPT-7B-Instruct 的最佳选择

RX 7900 XT 20GBA
20 GB
NVIDIA
RTX A4500 20GBA
20 GB
NVIDIA
RTX 3090 24GBA
24 GB
NVIDIA
RTX 3090 Ti 24GBA
24 GB
NVIDIA
RTX 4090 24GBA
24 GB

运行此模型

MPT-7B-Instruct on RX 7900 XT 20GBMPT-7B-Instruct on RTX A4500 20GBMPT-7B-Instruct on RTX 3090 24GB

量化选项

各量化级别的 VRAM 估算

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
2.7 GB
Low—
Q3_K_S
3
3.4 GB
Low—
NVFP4
4
3.9 GB
Medium—
Q4_K_M
4
4.3 GB
Medium—
Q5_K_M
5
5.0 GB
High—
Q6_K
6
5.7 GB
High—
Q8_0
8
7.5 GB
Very High—
F16
16
14.3 GB
Maximum—

Quality benchmarks

MPT-7B-Instruct benchmark scores

Benchmark verified

Reasoning

MMLU-Pro—
GPQA Diamond—
MATH-500—
ARC Challenge46.5%

Source: community · 2023-05-05

硬件兼容性

全部硬件的适配估算

打开计算器

Computing compatibility...

内存详细分析

Reference: RTX 2060 6GB

Weights4.3 GB
KV Cache7.8 GB
Runtime1.2 GB
Headroom0.6 GB

常见问题

FAQ — MPT-7B-Instruct

另请参阅

量化指南评分方法打开计算器