Jalapeño vs Blackwell: what the 1.9× perf/watt actually measures

The dataThe numbers, as published

Three models, three comparisons, one operating point each. Throughput is InferenceX's "mixed tokens per second per kilowatt"; latency is end-to-end for the request.

On GPT-OSS the time between tokens dropped from 1.87 ms to 0.69 ms. The part behind it: one reticle-sized compute die on TSMC N3P, 13.4 PFLOPS of MXFP4 on the B0 stepping, six HBM4 stacks for 216 GiB and 15.4 TB/s, a 700 W package TDP with a measured sustained draw of roughly 550 W. The Blackwell parts it was compared against are rated at 1 200 W (GB200) and 1 400 W (GB300) per GPU.

MethodologyWhat InferenceX actually counts

Before crediting the architecture, read the axis labels. Four things about the benchmark change how the table above should be interpreted.

• The denominator is all-in utility power, not chip TDP. InferenceX divides output tokens by the megawatts the whole deployment pulls from the wall — accelerator rack, host rack, networking. Jalapeño's reference deployment is a two-rack pair at roughly 160 kW. A lean host rack is worth as much here as a good compute die.
• It's a curve, not a point. The benchmark sweeps interactivity (tokens per second per user) against concurrency. "1.9×" is the ratio at one operating point on that curve, and the ratio changes along it.
• Mixed tokens, short context. Prefill and decode tokens are combined in one figure, at 8k input / 1k output. The long, multi-turn agent traces that dominate real production traffic — InferenceX calls that suite AgentX — have not been published for Jalapeño yet.
• Speculative decoding was off on Jalapeño while the Nvidia entries ran with multi-token prediction. That handicap runs against OpenAI's own chip, which makes the result more credible, not less.

RooflineBandwidth per watt explains the shape

Decoding is a memory-bandwidth problem. Every generated token streams the active weights and the KV cache through HBM once; the arithmetic per byte is tiny. So the first thing to normalize is not FLOPS but bytes per second per watt:

Against Blackwell, Jalapeño has a 3.3–3.9× advantage in bandwidth per watt before a single instruction executes. The measured advantage in tokens per kilowatt is 1.5–1.9×. Roughly half of the memory-system edge disappears into the parts of the workload that are not a weight stream — compute-bound prefill, host CPUs, network, cooling — which is exactly what an all-in denominator is supposed to expose. That is the useful reading of this result: not "the ASIC is 2× better than the GPU", but "a 700 W part fed by HBM4 turns about half of its raw bandwidth-per-watt edge into delivered tokens once the whole rack is on the meter".

The HBM generation is the elephant. Blackwell carries HBM3e; Nvidia's HBM4 part is Rubin, at roughly 22 TB/s per GPU and a 900–1 150 W compute die — the same bandwidth-per-watt class as Jalapeño. SemiAnalysis reports Jalapeño still ahead of Rubin on tokens per megawatt in their runs, but the gap is a fraction of the Blackwell gap. Rubin is the like-for-like comparison, and Rubin is shipping to customers now while Jalapeño is at engineering-sample stage.

TimelineThe calendar

Tape-out November 2025. Engineering samples August 2026. First deployments in OpenAI's own fleet at very small volume by the end of 2026, the bulk of the ramp at the end of 2027. The silicon that was benchmarked is an early stepping; OpenAI reportedly expects the B0 revision to add about 25 % in performance per watt. By the time Jalapeño runs at volume, the shipping Nvidia reference will not be Blackwell, and OpenAI's own hardware lead said as much when the numbers went out.

MethodHow I read a vendor-first benchmark

The same checklist I apply to my own measurements on the Mandelbrot case study applies here — only the scale of the machine differs.

Why this matters to me beyond the spec sheet: every agent I build or consult on is priced in tokens, and tokens are priced in watts. When the lab that runs the model also designs the silicon for it, the floor under inference cost keeps moving down — and the model-routing and cost-control work that already pays for itself today pays more tomorrow.