AMD to Acquire AI Inference Chip Startup Taalas as Chipmakers Embrace Heterogeneous Computing
Major AI chipmakers are moving away from using GPUs for every task and toward heterogeneous chip architectures. On Aug. 6, AMD (AMD) announced that it would acquire Taalas, a startup developing inference chips. AMD plans to add Taalas’ technology to its full-stack AI platform, including the Helios rack-scale solution, Instinct GPUs and EPYC CPUs. The technology will be integrated into AMD’s product roadmap. AMD did not disclose the purchase price. Founded in 2023 and headquartered in Toronto, Canada, Taalas develops chips tailored to individual AI models. The chips use high-speed SRAM, or static random-access memory, and optimize inference data flows to address the computing and memory bottlenecks of general-purpose architectures. Taalas says its chips can run Llama 3.1 and, when running inference for certain large models, can deliver output speeds of up to 1,000 times those of conventional GPU chips. The company is also developing more advanced products. Vamsi Boppana, senior vice president of AMD’s artificial intelligence business, said AMD is building a full-stack AI platform that gives customers flexibility in deploying computing solutions. He said Taalas’ technology and engineering team would strengthen the competitiveness of AMD’s AI product portfolio by providing differentiated inference performance and efficiency. The acquisition is not AMD’s only move involving inference chips. Since the start of the year, AMD has been pushing toward heterogeneous and even customized chips. Unlike the general-purpose GPUs offered by AMD and Nvidia (NVDA) for a range of AI computing applications, custom chips from companies such as Cerebras Systems and Taalas are less flexible and are not suited to every AI workload. They can, however, accelerate inference for specific models. In July, AMD announced a partnership with another AI inference-chip company, Cerebras Systems. Cerebras uses a wafer-scale chip design that integrates hundreds of thousands of computing cores into a single processor, reducing communication latency during model training and inference. The approach is similar to Taalas’ and, like Taalas’ products, is based on ASICs, or application-specific integrated circuits. AMD plans to combine Cerebras’ wafer-scale chips with its Helios rack-scale solution to deliver the ultra-low latency required by AI applications. In March, AMD announced a partnership with Meta that included the supply of semi-custom chips. When AMD announced the Cerebras partnership, AMD CEO Lisa Su said the growing diversity of AI inference requirements called for more flexible approaches. Su had previously said that AI infrastructure was becoming more complex, with multiple workloads requiring different types of computing. “In the next phase of AI infrastructure, there won’t be a single chip that does everything best. It’s already a heterogeneous world,” she said. Su said ASICs would always have a place in computing, with different chips used for different workloads and optimized to deliver the fastest computing performance. Nvidia is also integrating different types of computing chips. Earlier this year, Nvidia obtained a nonexclusive license to Groq’s intellectual property and hired members of the Groq team. Groq’s product is an LPU, or language processing unit, that uses SRAM and is designed to deliver strong inference performance at low cost. Market reports have said the Nvidia-Groq transaction was worth $20 billion. In March, the Groq 3 LPU appeared at Nvidia’s GTC conference. Nvidia’s Rubin platform includes a rack featuring Groq 3 LPX, designed for the low-latency and long-context requirements of agentic systems. AI systems based on heterogeneous chips are gradually being introduced. Cerebras Systems is deploying AMD Helios systems in data centers, and the combined solution is expected to become available through Cerebras Cloud in the second half of this year. Nvidia’s Vera Rubin platform is also moving into accelerated production and is expected to ship in the second half of this year.
On Aug. 6, AMD (AMD) announced that it would acquire Taalas, a startup developing inference chips. AMD plans to add Taalas’ technology to its full-stack AI platform, including the Helios rack-scale solution, Instinct GPUs and EPYC CPUs. The technology will be integrated into AMD’s product roadmap. AMD did not disclose the purchase price.
Founded in 2023 and headquartered in Toronto, Canada, Taalas develops chips tailored to individual AI models. The chips use high-speed SRAM, or static random-access memory, and optimize inference data flows to address the computing and memory bottlenecks of general-purpose architectures. Taalas says its chips can run Llama 3.1 and, when running inference for certain large models, can deliver output speeds of up to 1,000 times those of conventional GPU chips. The company is also developing more advanced products.
Vamsi Boppana, senior vice president of AMD’s artificial intelligence business, said AMD is building a full-stack AI platform that gives customers flexibility in deploying computing solutions. He said Taalas’ technology and engineering team would strengthen the competitiveness of AMD’s AI product portfolio by providing differentiated inference performance and efficiency.
The acquisition is not AMD’s only move involving inference chips. Since the start of the year, AMD has been pushing toward heterogeneous and even customized chips.
Unlike the general-purpose GPUs offered by AMD and Nvidia (NVDA) for a range of AI computing applications, custom chips from companies such as Cerebras Systems and Taalas are less flexible and are not suited to every AI workload. They can, however, accelerate inference for specific models.
In July, AMD announced a partnership with another AI inference-chip company, Cerebras Systems. Cerebras uses a wafer-scale chip design that integrates hundreds of thousands of computing cores into a single processor, reducing communication latency during model training and inference. The approach is similar to Taalas’ and, like Taalas’ products, is based on ASICs, or application-specific integrated circuits. AMD plans to combine Cerebras’ wafer-scale chips with its Helios rack-scale solution to deliver the ultra-low latency required by AI applications. In March, AMD announced a partnership with Meta that included the supply of semi-custom chips.
When AMD announced the Cerebras partnership, AMD CEO Lisa Su said the growing diversity of AI inference requirements called for more flexible approaches. Su had previously said that AI infrastructure was becoming more complex, with multiple workloads requiring different types of computing. “In the next phase of AI infrastructure, there won’t be a single chip that does everything best. It’s already a heterogeneous world,” she said.
Su said ASICs would always have a place in computing, with different chips used for different workloads and optimized to deliver the fastest computing performance.
Nvidia is also integrating different types of computing chips. Earlier this year, Nvidia obtained a nonexclusive license to Groq’s intellectual property and hired members of the Groq team. Groq’s product is an LPU, or language processing unit, that uses SRAM and is designed to deliver strong inference performance at low cost. Market reports have said the Nvidia-Groq transaction was worth $20 billion.
In March, the Groq 3 LPU appeared at Nvidia’s GTC conference. Nvidia’s Rubin platform includes a rack featuring Groq 3 LPX, designed for the low-latency and long-context requirements of agentic systems.
AI systems based on heterogeneous chips are gradually being introduced. Cerebras Systems is deploying AMD Helios systems in data centers, and the combined solution is expected to become available through Cerebras Cloud in the second half of this year. Nvidia’s Vera Rubin platform is also moving into accelerated production and is expected to ship in the second half of this year.