OpenAI Jalapeño: Custom Inference Chip Debuts at Hot Chips
OpenAI officially unveiled its self-designed inference chip, Jalapeño, at the Hot Chips 2026 conference on August 25. Co-developed with Broadcom, the chip went from initial team hiring to tape-out in just 16 months.
Key Specs
| Item | Data |
|---|---|
| Power | 700W per chip |
| Memory | 216GB HBM4 |
| Memory Bandwidth | 15.4 TB/s |
| 4-bit Compute | 13.4 PFLOPS |
| Chips per Rack | 128 |
| Rack Total Compute | 1.7 EFLOPS (4-bit) |
| Rack Memory | 27.5TB HBM4 |
| Rack Bandwidth | Nearly 2PB/s |
Benchmark Results
SemiAnalysis InferenceX benchmark results show:
- Peak throughput: 1.5x to 1.9x higher than Nvidia Blackwell (GB200/GB300 NVL72)
- End-to-end latency: 1.7x to 3.6x lower
- Ultra-low-latency scenarios: 2.1x to 4.1x faster
- Test models: GPT-OSS-120B, DeepSeek R1, Kimi K2.5
Notably, Jalapeño was tested without Multi-Token Prediction (MTP) optimization, while all competing Nvidia/AMD chips ran with MTP enabled. Jalapeño still came out ahead.
Not a Specialized Chip, a General-Purpose Inference Platform
The common assumption was that OpenAI's chip would only run its own models. SemiAnalysis testing proved otherwise. Jalapeño is a general-purpose inference chip capable of running various models and workloads. OpenAI even ported Doom to the chip using Codex prompts as a demonstration.
The chip uses HBM4 memory with specs comparable to Nvidia and AMD flagship GPUs. Richard Ho, VP of Hardware at OpenAI, said the design goal was to minimize data movement and keep model state like KV caches on-chip.
Timeline and Positioning
- Mid-2024: Chip program initiated
- August 2026: First public reveal at Hot Chips
- Late 2026: Small-batch shipping
- 2027: Volume production
OpenAI clarified that Jalapeño will not replace Nvidia or AMD. Training still requires GPU programmability. Jalapeño handles inference only, taking a different path from Nvidia/AMD's training-plus-inference hybrid chips.
Qwen 3.8-Flash-Next: A Preview of the Qwen4 Architecture
Alibaba's Qwen team posted the Qwen3.8-Flash-Next model page on HuggingFace, marked as "Upcoming release." At the time of writing, 1,532 people had clicked the "waiting for release" button.
Known Parameters
- Total parameters: 125B
- Active parameters: 6B (MoE architecture)
- Expected release: August 26, 2026
- Positioning: Technical preview of the Qwen4 architecture
125B total parameters with only 6B activated per forward pass indicates aggressive MoE routing. Compared to the Qwen3.8-2.4T-A95B (2.4T total, 95B active), Flash-Next uses far fewer active parameters to pursue faster inference.
Qwen3.8 Series Current Performance
The open-source Qwen3.8-2.4T-A95B, released on August 13, shows strong results across multiple benchmarks:
| Benchmark | Qwen3.8-Max | Opus 4.8 | Fable 5 | GPT 5.6 Sol |
|---|---|---|---|---|
| PaperBench | 93.0 | 80.3 | 88.8 | 90.5 |
| Terminal Bench 2.1 | 86.6 | 84.6 | 84.6 | 88.8 |
| SWE-bench Pro | 67.7 | 69.2 | 80.0 | 64.6 |
| FrontierSWE | 73.5 | 70.0 | 88.8 | -- |
| Agents' Last Exam | 52.4 | 45.1 | -- | 53.6 |
Qwen3.8-Max matches or exceeds GPT 5.6 Sol on PaperBench (93.0) and Agents' Last Exam (52.4). However, it trails Fable 5 significantly on SWE-bench Pro (67.7 vs 80.0) and FrontierSWE (73.5 vs 88.8).
The key question for Flash-Next is whether it can maintain coding capability while substantially reducing inference cost.
What It Means
Both developments point in the same direction: AI inference is shifting from "buy Nvidia GPUs" to a combination of custom chips and efficient model architectures. OpenAI's Jalapeño proves custom inference silicon can beat Nvidia on performance-per-watt. Alibaba's MoE approach shows that 6% of parameters can do the work of a full-size model. If both paths succeed, inference costs may drop faster than most expect.