-
- News
- Books
Featured Books
- design007 Magazine
Latest Issues
Current IssueRules of Thumb
This month, we delve into rules of thumb—which ones work, which ones should be avoided. Rules of thumb are everywhere, but there may be hundreds of rules of thumb for PCB design. How do we separate the wheat from the chaff, so to speak?
Partial HDI
Our expert contributors provide a complete, detailed view of partial HDI this month. Most experienced PCB designers can start using this approach right away, but you need to know these tips, tricks and techniques first.
Silicon to Systems: From Soup to Nuts
This month, we asked our expert contributors to weigh in on silicon to systems—what it means to PCB designers and design engineers, EDA companies, and the rest of the PCB supply chain... from soup to nuts.
- Articles
- Columns
Search Console
- Links
- Media kit
||| MENU - design007 Magazine
New IBM Processor Innovations to Accelerate AI on Next-Generation IBM Z Mainframe Systems
August 26, 2024 | IBMEstimated reading time: 4 minutes
IBM revealed architecture details for the upcoming IBM Telum® II Processor and IBM Spyre™ Accelerator at Hot Chips 2024. The new technologies are designed to significantly scale processing capacity across next generation IBM Z mainframe systems helping accelerate the use of traditional AI models and Large Language AI models in tandem through a new ensemble method of AI.
With many generative AI projects leveraging Large Language Models (LLMs) moving from proof-of-concept to production, the demands for power-efficient, secured and scalable solutions have emerged as key priorities. Morgan Stanley research published in August projects generative AI's power demands will skyrocket 75% annually over the next several years, putting it on track to consume as much energy in 2026 as Spain did in 2022.1 Many IBM clients indicate architectural decisions to support appropriately sized foundation models and hybrid-by-design approaches for AI workloads are increasingly important.
The key innovations unveiled today include:
IBM Telum II Processor: Designed to power next-generation IBM Z systems, the new IBM chip features increased frequency, memory capacity, a 40 percent growth in cache and integrated AI accelerator core as well as a coherently attached Data Processing Unit (DPU) versus the first generation Telum chip. The new processor is expected to support enterprise compute solutions for LLMs, servicing the industry's complex transaction needs.
IO acceleration unit: A completely new Data Processing Unit (DPU) on the Telum II processor chip is engineered to accelerate complex IO protocols for networking and storage on the mainframe. The DPU simplifies system operations and can improve key component performance.
IBM Spyre Accelerator: Provides additional AI compute capability to complement the Telum II processor. Working together, the Telum II and Spyre chips form a scalable architecture to support ensemble methods of AI modeling – the practice of combining multiple machine learning or deep learning AI models with encoder LLMs. By leveraging the strengths of each model architecture, ensemble AI may provide more accurate and robust results compared to individual models. The IBM Spyre Accelerator chip, previewed at the Hot Chips 2024 conference, will be delivered as an add on option. Each accelerator chip is attached via a 75-watt PCIe adapter and is based on technology developed in collaboration with the IBM Research. As with other PCIe cards, the Spyre Accelerator is scalable to fit client needs.
"Our robust, multi-generation roadmap positions us to remain ahead of the curve on technology trends, including escalating demands of AI," said Tina Tarquinio, VP, Product Management, IBM Z and LinuxONE. "The Telum II Processor and Spyre Accelerator are designed to deliver high-performance, secured, and more power efficient enterprise computing solutions. After years in development, these innovations will be introduced in our next generation IBM Z platform so clients can leverage LLMs and generative AI at scale."
The Telum II processor and the IBM Spyre Accelerator will be manufactured by IBM's long-standing fabrication partner, Samsung Foundry, and built on its high performance, power efficient 5nm process node. Working in concert, they will support a range of advanced AI-driven use cases designed to unlock business value and create new competitive advantages. With ensemble methods of AI, clients can achieve faster, more accurate results on their predictions. The combined processing power announced today will provide an on ramp for the application of generative AI use cases. Some examples could include:
- Insurance Claims Fraud Detection: Enhanced fraud detection in home insurance claims through ensemble AI, which combine LLMs with traditional neural networks geared for improved performance and accuracy.
- Advanced Anti-Money Laundering: Advanced detection for suspicious financial activities, supporting compliance with regulatory requirements and mitigating the risk of financial crimes.
- AI Assistants: Driving the acceleration of application lifecycle, transfer of knowledge and expertise, code explanation as well as transformation, and more.
Specifications and Performance Metrics:
Telum II processor: Featuring eight high-performance cores running at 5.5GHz, with 36MB L2 cache per core and a 40% increase in on-chip cache capacity for a total of 360MB. The virtual level-4 cache of 2.88GB per processor drawer provides a 40% increase over the previous generation. The integrated AI accelerator allows for low-latency, high-throughput in-transaction AI inferencing, for example enhancing fraud detection during financial transactions, and provides a fourfold increase in compute capacity per chip over the previous generation.
The new I/O Acceleration Unit DPU is integrated into the Telum II chip. It is designed to improve data handling with a 50% increased I/O density. This advancement enhances the overall efficiency and scalability of IBM Z, making it well suited to handle the large-scale AI workloads and data-intensive applications of today's businesses.
Spyre Accelerator: A purpose-built enterprise-grade accelerator offering scalable capabilities for complex AI models and generative AI use cases is being showcased. It features up to 1TB of memory, built to work in tandem across the eight cards of a regular IO drawer, to support AI model workloads across the mainframe while designed to consume no more than 75W per card. Each chip will have 32 compute cores supporting int4, int8, fp8, and fp16 datatypes for both low-latency and high-throughput AI applications.
Availability
The Telum II processor will be the central processor powering IBM's next-generation IBM Z and IBM LinuxONE platforms. It is expected to be available to IBM Z and LinuxONE clients in 2025. The IBM Spyre Accelerator, currently in tech preview, is also expected to be available in 2025.
Statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Suggested Items
IBM Expands its AI Accelerator Offerings; Announces Collaboration with AMD
11/18/2024 | IBMIBM and AMD have announced a collaboration to deploy AMD Instinct MI300X accelerators as a service on IBM Cloud. This offering, which is expected to be available in the first half of 2025, aims to enhance performance and power efficiency for Gen AI models such as and high-performance computing (HPC) applications for enterprise clients.
IBM Introduces Granite 3.0: High Performing AI Models Built for Business
10/21/2024 | IBMAt IBM's annual TechXchange event the company announced the release of its most advanced family of AI models to date, Granite 3.0. IBM's third-generation Granite flagship language models can outperform or match similarly sized models from leading model providers on many academic and industry benchmarks, showcasing strong performance, transparency and safety.
Intel, IBM Deliver Enterprise AI in the Cloud
09/03/2024 | IntelIBM and Intel announced a global collaboration to deploy Intel® Gaudi® 3 AI accelerators as a service on IBM Cloud. This offering, which is expected to be available in early 2025, aims to help more cost-effectively scale enterprise AI and drive innovation underpinned with security and resiliency.
ASMPT, IBM Deepen Collaboration to Advance Bonding Methods for Chiplet Packages for AI
07/24/2024 | ASMPTASMPT and IBM today announced a renewed agreement to extend their collaboration on the joint development of the next advancement of chiplet packaging technologies.
IBM to Acquire Octo
05/20/2024 | PRNewswireIBM announced an agreement to acquire Octo, a U.S.-based IT modernization and digital transformation services provider exclusively serving the U.S. federal government, including defense, health, and civilian agencies. IBM's acquisition of Octo establishes one of the largest digital transformation partners to the federal government.