Kimi K3: Open-Source 3T-Parameter Model
On July 27, Moonshot AI released the Kimi K3 model weights on HuggingFace. It is the world's first open 3T-class large language model.
Kimi K3 has 2.8 trillion parameters using a Mixture of Experts (MoE) architecture — 16 out of 896 experts are activated per token. It supports a 1-million-token context window and natively handles images and video.
Architecture
Kimi K3 introduces Kimi Delta Attention (KDA) for efficient attention over long sequences and Attention Residuals (AttnRes) for flexible cross-layer information flow. The Stable LatentMoE framework uses Quantile Balancing for expert routing.
The model uses quantization-aware training with MXFP4 weights and MXFP8 activations for broader hardware compatibility.
Performance
DeepSWE v1.1: Kimi K3 scored 67.3 using the mini-SWE-agent harness.
SWE Marathon: Evaluated alongside Claude Opus 4.8 and Claude Fable 5 (Claude Code harness), with GPT-5.6 Sol using Codex. Fable 5 hit fallbacks on 35% of tasks.
GPU kernel optimization: Kimi K3 performed competitively with Fable 5 and substantially outperformed Opus 4.8, GPT-5.6 Sol, and GPT 5.5.
An early K3 version handled most of the team's kernel optimization work. It also built MiniTriton, a compact compiler that beats Triton and torch.compile on some workloads.
Pricing & Availability
Kimi K3 pricing: cache-hit input $0.30/MTok, cache-miss input $3.00/MTok, output $15.00/MTok.
Available through Kimi.com, Kimi Work, Kimi Code, and the Kimi API. Mobile on iOS, Android, HarmonyOS.
Open-Source Impact
Kimi K3 is the first open 3T-class model. Over 12 months, Moonshot has pushed open-model size boundaries for nine of them.
Hardware requirements are steep: ~1.5TB VRAM even in MXFP4 format. Practical access will come through cloud providers.
The full technical report is forthcoming.