Will It Run AI

MicrosoftMicrosoft

Phi 3.5 Mini 4B

Legado
705.5KDownloads984CurtidasAug 2024Publicado128K tokensContextoMITLicença39 BásicoQualidade

Phi 3.5 Mini 4B (4B parameters) requires approximately 10.1 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 12 GB of VRAM.

Comece agora

— copie e cole para rodar localmente

Copy-paste commands to run Phi 3.5 Mini 4B on your machine.

Run

ollama run phi3.5

Quick specs

Parameters4B
Architecturedense
Context128K tokens
Modalitytext
Min RAM1.6 GB
Rec. RAM2.4 GB (Q4_K_M)
LicenseMIT
FamilyPhi
Chat

About this model

Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.

  • Memory/compute constrained environments
  • Latency bound scenarios
  • Strong reasoning (especially code, math and logic)

Modelos relacionados

Seu hardware

Detectando...

Escolhas rápidas

Melhor hardware

Melhores opções para Phi 3.5 Mini 4B

Rodar este modelo

Opções de quantização

Estimativas de VRAM por nível de quantização

No hardware detected — fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
1.6 GB
Low
Q3_K_S
3
2.0 GB
Low
NVFP4
4
2.2 GB
Medium
Q4_K_M
4
2.4 GB
Medium
Q5_K_M
5
2.9 GB
High
Q6_K
6
3.3 GB
High
Q8_0
8
4.3 GB
Very High
F16
16
8.2 GB
Maximum

Quality benchmarks

Phi 3.5 Mini 4B benchmark scores

Benchmark verified

Coding

SWE-bench Verified
HumanEval+62.8%
Aider Polyglot
LiveCodeBench

Reasoning

MMLU-Pro69.0%
GPQA Diamond12.0%
MATH-50019.6%
ARC Challenge84.6%

General

Chatbot Arena
IFEval57.7%

Source: official · 2024-08-20

Compatibilidade de hardware

Estimativas de compatibilidade para todo o hardware

Abrir calculadora

Computing compatibility...

Detalhamento de memória

Reference: RTX 2060 6GB

Weights2.4 GB
KV Cache5.9 GB
Runtime1.2 GB
Headroom0.6 GB

Perguntas frequentes

FAQ — Phi 3.5 Mini 4B

Veja também