OpenAI's Jalapeño chip seated on its board, from OpenAI's announcement video

Yesterday at the Hot Chips conference at Stanford, OpenAI revealed Jalapeño — its first custom chip, built with Broadcom, manufactured by TSMC on 3nm. They claim it beats Nvidia's best on the metrics that matter for serving AI models.

Most coverage of this is written by people who have never paid an AI bill. I run 8 AI startups, and inference is a line item in every single one of them. Last week I literally repriced one of my products around the cost of a single AI message. So this launch isn't abstract to me — it's my unit economics.

First the facts, then what four years of paying these bills tells me about what happens next.

What Jalapeño actually is

An inference-only chip. It runs AI models — it doesn't train them. Every ChatGPT reply is inference, and inference is OpenAI's single biggest cost. This chip exists to cut that bill. (OpenAI still buys Nvidia and AMD for training — so no, Nvidia is not dead.)

The numbers, per chip: 13.4 petaFLOPS of 4-bit compute, 216GB of HBM4, 15.4 TB/s of memory bandwidth — with HBM pins running at 10Gbps, faster than Nvidia's 9.6. Power draw is 700W against 900–1,150W for Nvidia's next-gen Rubin. A rack holds 128 chips (1.7 exaFLOPS), and racks link up to 2,048 chips — headroom for 10–20 trillion parameter models.

Jalapeño being lowered onto its socket — the chip package with six HBM4 stacks visible

On an inference chip, memory bandwidth is the product. Serving tokens is a memory game, not a compute game — and that's exactly where OpenAI pushed hardest.

The claims

Benchmarked against Nvidia's GB300 on the SemiAnalysis InferenceX suite (GPT-OSS-120B, DeepSeek R1, Kimi K2.5):

Up to 1.9x more AI work at peak throughput
1.7–3.6x lower end-to-end latency
700+ tokens/sec per user on DeepSeek R1 — on Kimi K2.5, roughly 9x the next-best system

The detail most coverage missed: Jalapeño posted these numbers without speculative decoding, a trick that gives competitors a 3–5x cost advantage in their best published results. When OpenAI adds it, the gap widens.

And the metric OpenAI keeps repeating is tokens per megawatt, because data centers are power-limited now, not chip-limited. As SemiAnalysis put it: "If you have 1 gigawatt of power, throughput per watt IS revenue."

"Jalapeño — more intelligence per watt with faster responses" — OpenAI's own tagline from the reveal video

The part that actually broke my brain

AI helped design this chip. Specifics, not marketing:

AI-assisted design blocks came out 8–10% smaller with better power than the initial human designs
Idea to tape-out in ~16 months. Chips in this class normally take 3–4 years
OpenAI's internal Codex wrote the chip's software kernels with no engineer touching them. They had no kernel for DeepSeek's attention mechanism until they needed to benchmark it — Codex wrote a working one on its own
For fun, engineers prompted Codex to port DOOM to the chip. It runs at 36 FPS

Which is why SemiAnalysis dropped the strongest line in their whole deep-dive: "The CUDA moat is potentially dead." Nvidia's real moat was never just silicon — it was the years of software work needed to make any new chip usable. OpenAI's AI brought up Jalapeño's software stack faster than Nvidia brought up its own next-gen chip.

The honest caveats

I don't publish hype, so: the benchmarks are OpenAI's own (SemiAnalysis watched the runs in-lab but didn't run the full suite independently). Only short single-turn conversations were tested, on models that aren't frontier-scale. On cost per token today, Jalapeño is roughly a tie with Nvidia's Rubin, not a blowout. And volume production is a 2027 story — the next milestone is 100MW of deployed capacity.

So why does it still matter? Because of the curve it sits on. Here's the part I can speak to personally.

I also broke this down as a 19-slide carousel on Instagram — follow me there for the daily version of this.

I've watched this cost curve from the inside since 2022

When I launched MagicSlides in 2022, the GPT-3-class models behind it cost around $20 per million tokens. Every presentation the tool generated burned real money, and I priced and rationed features around that fear. That product has since crossed 1.5M+ users and 10M+ presentations — every one of them an inference bill.

Last week — literally ten days ago — I repriced MagicChat, my chatbot product. To do it properly I measured the true cost of serving one message through a modern model: $0.001. Eight paise. A full RAG answer — retrieval, 7,700 input tokens, the works — for a tenth of a US cent. I rebuilt the entire plan structure around that number and landed at ~91% gross margins.

That's the same journey OpenAI is on with Jalapeño, one level down the stack. From ~$20 per million tokens to fractions of that in four years — and now the biggest AI company on earth is spending billions on custom silicon to push the floor lower. Magic Meals, my food-scanning app, spends about ₹2 of frontier-model AI per scan. In 2022, that product was economically impossible. Today it's a subscription business. That's what a collapsing cost curve does: it doesn't improve existing products, it makes previously-impossible products viable.

The take nobody covering this will give you

Here's what repricing MagicChat taught me, and it's the real lesson of Jalapeño:

Cheaper inference does not make AI products cheaper. It makes builders' margins fatter.

When I studied my competitors before repricing, the category leader was charging roughly 8x more per message than I was — and winning anyway. Price is not the variable customers buy on. Trust, distribution, and product are. So when chips like Jalapeño push token costs down another order of magnitude, prices won't drop 10x. Margins will expand, and the features that were too expensive to ship become free tier.

OpenAI understands this better than anyone — Nvidia's ~75% gross margins are OpenAI's costs, and building your own chip is just vertical integration. Meta and Microsoft both tried this and largely failed. OpenAI landed it first-generation because frontier AI did the heavy lifting on design and software. That's the actual story of this launch.

If you're waiting for a better time to build AI products: the companies with the biggest bills in the world are spending billions to make YOUR unit economics better. They're doing the capex. You collect the margin.

Sources: OpenAI/Broadcom announcement · SemiAnalysis deep-dive · The Register · Bloomberg