OpenAI Jalapeño beats Blackwell in its first inference tests

Two months ago, Jalapeño was a chip in a lab accompanied by a promise: OpenAI and Broadcom said their first custom accelerator would serve language models with far greater energy efficiency than available hardware. Since August 25, that claim has something it lacked then: named models, specific comparison systems, test conditions, and results observed by an outside organization.

OpenAI ran three open-weight models—GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T—using InferenceX, the public benchmark from SemiAnalysis. In those configurations, Jalapeño delivered 1.5 to 1.9 times more work per watt at peak throughput and completed requests 1.7 to 3.6 times faster than the NVIDIA GB200 or GB300 systems used as references.

This is important because of what it replaces. The original June announcement spoke only of early testing and a future performance-per-watt advantage. There is now working silicon, a complete system, and a first inspectable measurement. It does not, however, show that OpenAI can already serve ChatGPT more cheaply, that it has beaten NVIDIA's entire lineup, or that it has reduced the total power use of a data center. The distance between those two sentences is the real subject of the news.

A very specific performance frontier

Serving a model involves an awkward tradeoff. A system that batches many requests can produce more tokens per second, but each person waits longer; one that prioritizes a single conversation responds quickly while leaving some hardware idle. InferenceX sweeps across different concurrency levels and traces a frontier: how much work a system completes without exceeding a given latency.

Jalapeño moved that frontier in all three tests. The most useful signal is not a single extreme number, but that the chip sustained both more throughput per kilowatt and less waiting. On DeepSeek R1, for example, OpenAI reports 19,641 mixed tokens per second per kilowatt versus 11,781 for GB300 at their respective peak-throughput points; the lowest end-to-end latency was 1.65 seconds versus 5.99. Those measurements do not necessarily come from the same load point, so they should not be combined as if they described one magical request.

The conditions narrow the result further. All three workloads used roughly 8,000 input tokens and 1,000 output tokens and single-token prediction, without speculative decoding. To normalize energy, OpenAI used package power ratings: 700 watts for Jalapeño, 1,200 for GB200, and 1,400 for GB300. The company says its chip sustained no more than 550 watts during the tests, but the published comparison uses nominal ratings for all systems. This measures accelerators under a common convention; it does not necessarily include the CPU, network, cooling, and electrical losses of the complete rack.

SemiAnalysis visited the lab and verified InferenceX runs there alongside OpenAI engineers. That observation is stronger than receiving a press table, but weaker than independent replication: OpenAI supplied the numbers, Jalapeño cannot be bought or rented, and SemiAnalysis did not run its complete suite.

One omission is especially relevant. InferenceX retains these fixed sequences, but its AgentX test replays long coding sessions with irregular turns, tools, subagents, and context reuse. That is much closer to the claim that lower latency speeds up agents chaining many steps. SemiAnalysis has not seen AgentX results for Jalapeño. NVIDIA, meanwhile, has just published preliminary Vera Rubin results on that test, still pending SemiAnalysis review. The model, traffic, serving techniques, and power boundary all change, so those figures cannot declare a winner between Rubin and Jalapeño.

This caution does not erase the advantage over Blackwell. It prevents transferring that advantage to a contemporary commercial rival and a workload that has not yet been measured.

The second result is in the software

A new accelerator does not automatically inherit the ecosystem of kernels and tools that NVIDIA has accumulated around CUDA. Each model architecture brings different combinations of attention, memory, and experts; without mature software, a good specification can remain mostly empty.

Jalapeño offers a second capability result here. OpenAI says Codex with GPT-Astra brought the three open models, which were not in the original production plan, to high performance in two months. On selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the prior specialist-written versions. This is a comparison of selected blocks, not the full model, but it addresses concrete work in one of the hardest bottlenecks of new hardware.

The architecture was designed to make that problem more tractable: explicit tensor placement, predictable communication and synchronization, and control over where the KV cache that holds generation context remains. A human defines the computation and constraints; the model can explore how to distribute and schedule it in parallel. The consequence is not that AI designed the chip by itself. OpenAI and its partners built the architecture, physical implementation, boards, network, and verification. The more defensible consequence is that co-optimization now continues after fabrication at an unusual pace.

That may erode one part of the proprietary software moat without removing it. An organization controlling models, traffic, and compilers can rapidly adapt its own accelerator. An outside developer still has no access to Jalapeño, and OpenAI says it will continue deploying hardware from NVIDIA and other suppliers for both training and inference. The chip does not train models and does not replace a diversified supply chain.

The decisive test starts now. OpenAI plans to begin introducing Jalapeño into its infrastructure before the end of 2026; it is still qualifying production, maturing the software, and collecting reliability data. It has not published sustained performance under live traffic, availability, manufacturing yield, total cost per token, or an associated price change. Nor has it shown how much of the advantage on open models survives on its internal models: it says the margin grows, but does not provide the table needed to inspect that claim.

My reading is that Jalapeño crosses an industrial frontier, not yet a product frontier. A model lab has shown that its first chip can run outside workloads and beat strong commercial systems in a public test, while using its own models to close part of the initial software deficit. That makes an alternative to complete dependence on general accelerators credible and could put downward pressure on the cost of serving many-step agents.

But potential efficiency is not abundant access. For the advance to reach a person, it must survive manufacturing, failures, unpredictable traffic, cooling, integration, and whole-system economics. The next valuable disclosure will not be another multiple on a fixed chart: it will be a reviewable measurement of Jalapeño on AgentX or in production, against contemporary hardware and with end-to-end power and cost. The chip now exists. What remains is showing what changes when it leaves the lab.

Sources

• OpenAI, Jalapeño’s first results show industry-leading speed and efficiency in AI inference, August 25, 2026.
• OpenAI and Broadcom, OpenAI and Broadcom unveil LLM-optimized inference chip, June 24, 2026.
• SemiAnalysis, OpenAI Jalapeño: Better Than Nvidia Blackwell, August 25, 2026, and public InferenceX methodology, accessed August 26, 2026.
• NVIDIA, NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt, August 24, 2026.