California [US], August 26 (ANI): OpenAI has announced results from testing Jalapeno, its first custom inference chip, saying the in-house silicon delivers higher throughput, lower latency and greater power efficiency across multiple AI models.
In a post on X, OpenAI CEO Sam Altman announced the development, saying, “we made a chip and it is fast”.
https://x.com/sama/status/2092339694210040187
According to an article titled ‘Jalapeno’s first results show industry-leading speed and efficiency in AI inference’ published on OpenAI’s official website, on Tuesday (local time) the company said Jalapeno can “serve more AI work per unit of power while also returning responses more quickly”.
“Jalapeno delivers both higher throughput and lower latency with one architecture, where existing hardware systems often have to make a tradeoff between the two,” OpenAI said in the article.
The company said the chip could enable “faster responses, more responsive agents, and more reliable access as demand grows”, adding that the gains could make increasingly capable AI “more affordable and more broadly available”.
OpenAI said Jalapeno’s performance was tested across GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T, including models developed outside the company.
“Across all three, Jalapeno delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems,” the company said.
“For highly interactive workloads, it delivered 2.1 to 4.1 times higher performance,” it added.
OpenAI said Jalapeno represents a broader “full-stack advantage”, allowing the company to design models, products, serving software, chips, memory, networking and systems together.
“Jalapeno is working first-party silicon with measured results, and it is the beginning of a multigenerational platform,” the company said.
According to the company, the chip was evaluated using InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. OpenAI said it compared Jalapeno with leading commercially available AI systems across operating conditions ranging from high-throughput serving to highly interactive, low-latency use.
The company said Jalapeno is rated at 700 watts, while its measured sustained power remained at or below 550 watts on the tested workloads.
On Kimi K2.5 1T, the largest public model tested, Jalapeno delivered “approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system”, OpenAI said.
OpenAI said Jalapeno was designed specifically around modern and future language-model workloads, with its chip, memory, network, software and rack-scale system developed together.
“We designed Jalapeno to minimize data movement and communication delays,” the company said, explaining that model state, including the KV cache used during response generation, can be explicitly placed and kept local.
The company also highlighted the role of AI in developing the chip, saying it helped the team move from initial design to tapeout in nine months by exploring implementations and shortening design, measurement and verification cycles.
“AI played a direct role in Jalapeno’s development,” OpenAI said.
The company said it also used Codex with GPT-Astra to bring three open-weight models that were not part of Jalapeno’s original production plan to high performance within two months.
“For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations,” OpenAI said, while noting that the figures applied to selected blocks rather than the full model.
OpenAI said it plans to begin deploying Jalapeno within its compute infrastructure by the end of the year.
“It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape,” the company said.
The company added that it would continue to deploy accelerators from NVIDIA and other partners for training and inference workloads, while preparing Jalapeno for large-scale operation and validating its performance across additional models. (ANI)
Related reading
