System Architecture Beyond the Die With Advanced Packaging as the Scaling Factor

Estimated reading time: 6 minutes

HBM enables this by stacking dynamic random-access memory (DRAM) dies and connecting them through wide parallel interfaces. Advanced packaging allows these stacks to be placed close to compute dies, enabling high-density interconnects (UHDI) and improved data throughput. As a result, memory placement, stack count, and bandwidth distribution become key architectural decisions that directly influence package size, routing complexity, and system efficiency.

Such a tighter integration introduces new constraints. Additional memory stacks increase substrate complexity and affect yield, while thermal interaction and shared power delivery must be carefully managed. As a result, system design becomes increasingly package-centric, reinforcing the shift toward disaggregated architectures where partitioning is closely tied to memory integration.

Scaling Disaggregated Design Further Through Co-Packaged Optics And Co-Packaged Copper

As advanced packaging pushes integration beyond the monolithic die, system bottlenecks are also shifting from compute density to data movement. This affects accelerators, switches, and rack-scale infrastructure. As a result, disaggregation is extending beyond the package, driving demand for high-bandwidth, energy-efficient interconnect architectures. In this context, co-packaged optics (CPO) and co-packaged copper (CPC) are critical technologies for next-generation scale-in, scale-out, and scale-across systems.

In scale-in architectures, advanced packaging technologies like silicon interposers, embedded bridges, and high bandwidth memory (HBM) enable dense local communication between compute chiplets and memory stacks. Yet, as artificial intelligence (AI) workloads distribute computation across several accelerators and nodes, interconnect scaling shifts from a package-level to a system-level challenge.

CPO addresses this challenge by integrating optical engines directly alongside switches or accelerator packages. This reduces the electrical trace length between the compute and the optical conversion. As a result, it significantly improves bandwidth density and lowers the power consumption associated with long-reach electrical signaling. Networking speeds are moving toward 800G, 1.6T, and beyond. CPO is becoming increasingly important for scale-out architectures, where traditional pluggable optics face limitations in signal integrity, front-panel density, and energy efficiency.

At the same time, CPC continues to play a critical role in short-reach, low-latency communication within servers and tightly coupled systems. CPC enables high-density electrical connectivity with lower manufacturing complexity and greater cost efficiency than optical integration. These technologies are not competing. Instead, CPC and CPO are evolving as complementary interconnect approaches. Copper remains optimized for localized communication within packages and boards. Meanwhile, optics increasingly enables scalable communication across racks and distributed compute fabrics.

The evolution of CPC and CPO extends advanced packaging from compute and memory integration into a broad networking infrastructure. The package thus becomes the launch point for system-level interconnects enabling large-scale, disaggregated computing.

Manufacturing Decisions for Disaggregation and Beyond

Lastly, as systems become more package-centric, partitioning moves to the forefront. Disaggregating functionality across chiplets is no longer only a design choice but a manufacturing strategy that directly impacts yield, cost, and scalability for silicon architecture designed for advanced computing like data centers. Instead of integrating everything on a single die, systems are now assembled from multiple optimized components within a package.

In monolithic designs, yield is constrained by die area, with larger dies more susceptible to defects. Partitioning into smaller chiplets improves yield and enables known good die strategies, where only validated components are integrated. It also allows different functions to be fabricated on process nodes best suited to their requirements, improving wafer utilization and reducing dependence on leading-edge nodes.

These benefits, however, shift complexity to the package. Assembly demands precise die placement, fine-pitch interconnect formation, and validation across multiple interfaces. Final yield depends not only on die quality but also on assembly precision and interconnect reliability, requiring test strategies that extend from wafer-level validation to full system-level verification.

This drives the partitioning decisions, which now sit at the intersection of design, packaging, and manufacturing. Advanced packaging specifically leads and defines how far disaggregation can scale and, in doing so, reshapes system architecture itself. The system is no longer defined by the limits of a single die, but by how effectively multiple dies can be integrated within the package. This shift certainly shows that the package has become the new scaling factor.

Chetan Arvind Patil is principal engineer, test engineering and customer strategy, at Marvell Technology.

• < Previous Page