Dan’s Biz Bookshelf: ‘Building a Story Brand 2.0’
Marcy’s Musings: A World of Difference
It’s Only Common Sense: Why Should They Take Your Call?
Marvell Technology, Inc., a leader in data infrastructure semiconductor solutions, announced new innovations across its AI memory infrastructure portfolio, advancing its position spanning server-level AI storage, rack-scale CXL memory expansion and pooling, and pod-level optical shared memory. The portfolio helps hyperscalers and cloud providers scale memory more independently from compute, improving infrastructure utilization, scalability and token efficiency for agentic AI inference.
As agentic AI inference scales, memory capacity, bandwidth and connectivity are becoming as critical as compute. Larger models, longer context windows and growing KV caches are driving unprecedented memory demands and exposing the limits of traditional server-attached architectures. Increasingly, AI performance and token efficiency depend on how efficiently processors can access and move data across memory resources.
Memory disaggregation addresses this challenge by enabling memory to be expanded, pooled and shared more independently of compute. By making memory more accessible and reducing data movement and latency, the Marvell portfolio helps reduce GPU stalls and improve model FLOPs utilization, allowing AI infrastructure to generate more tokens within the same data center footprints and power envelopes.
“AI infrastructure is moving beyond isolated servers to systems where compute, memory and connectivity operate seamlessly together,” said Will Chu, executive vice president and general manager, Custom Cloud Solutions, at Marvell. “As AI scales, memory must scale more independently of compute so resources can be deployed where they deliver the greatest value. With the industry’s broadest AI memory infrastructure portfolio, Marvell is helping customers improve utilization, boost token efficiency and scale AI without compromising performance, power or cost.”
“As AI workloads grow larger and more complex, memory capacity, bandwidth, latency and data movement are becoming primary constraints on AI performance,” said Alan Weckel, co-founder and technology analyst at 650 Group. “Marvell’s memory and storage portfolio gives hyperscalers and cloud providers a strong foundation for building scalable, efficient AI systems capable of supporting increasingly advanced workloads.”
Server-level AI Storage
The Marvell Bravera SC6 PCIe® 6.0 SSD controller advances storage performance for AI inference, enabling more KV cache to move from high-bandwidth memory to SSD to improve infrastructure efficiency. Doubling the performance of the widely deployed Bravera SC5 PCIe 5.0 SSD controller, Bravera SC6 helps cloud providers support demanding AI, cloud and enterprise workloads while improving performance, reducing write amplification and extending NAND endurance. Optimized for hyperscale deployments, Bravera SC6 supports NAND from multiple suppliers, providing cloud providers with greater flexibility in storage sourcing and deployment.
Rack-level Memory Expansion and Pooling
The Marvell Structera X memory expansion solutions help hyperscalers scale memory more efficiently for increasingly memory-intensive AI workloads. Developed in close collaboration with leading hyperscalers, Structera X enables customers to expand, optimize and better utilize memory resources, including extending the value of existing memory investments while supporting demanding AI inference applications. As larger models, longer context windows and growing KV-cache requirements make memory capacity and bandwidth critical infrastructure constraints, Structera X provides a path toward more flexible and disaggregated memory architectures. By enabling larger memory pools and more efficient sharing of resources across servers, the platform helps improve infrastructure utilization, operational efficiency and total cost of ownership while laying the foundation for future advances in CXL-based memory pooling and sharing.
Pod-level Optical Shared Memory
The Marvell Photonic Fabric memory modules, Photonic Fabric NIC and Photonic Fabric chiplets are foundational elements of a multi-rack optical shared-memory architecture for AI inference. The solution creates a new shared-memory tier across multiple XPUs and racks up to 50 meters, enabling up to 32TB of warm KV cache offload with high bandwidth and extremely low latency. By expanding memory access across AI clusters and enabling KV cache to be loaded from a shared memory tier rather than storage, Photonic Fabric helps increase inference throughput, support larger models and longer context lengths, and increase token efficiency by delivering up to 2-3x higher token throughput within existing data center footprints and power envelopes.
