NVIDIA Architecture Drives Optical Interconnects; CPO in AI Data Centers Rising

Estimated reading time: 2 minutes

NVIDIA’s next-generation AI compute rack architecture indicates that future GPU designs will increasingly prioritize higher chip-to-chip interconnect density and faster data transmission, according to TrendForce’s latest research on the high-speed interconnect market. Intra-rack chip interconnects (scale-up) and large-scale interconnects across racks (scale-out) will become central considerations in data center design as AI clusters continue to scale.

Traditional electrical transmission using copper cables faces physical limitations and will struggle to support the massive data movement required by next-generation AI infrastructure. As a result, optical transmission technologies are gaining greater importance.

TrendForce forecasts that co-packaged optics (CPOs) will steadily increase their share of optical communication modules in AI data centers, with penetration potentially reaching 35% by 2030.

NVIDIA’s NVLink 6 communication protocol defines 400G SerDes per lane as the peak transmission rate, with a bandwidth ceiling of 3.6 TB/s per GPU. At such extreme transmission speeds, electrical signals over copper degrade rapidly with distance, effectively limiting copper interconnects to distances of less than one meter.

Nevertheless, Broadcom believes ongoing advances in SerDes technology will continue to push physical limits. Copper-based solutions are expected to remain the dominant option for ultra-short-reach interconnects within racks through at least 2028, thanks to their cost advantages and relatively low power consumption.

However, as chip interconnect scales expand and the scale-up configuration grows from a single rack to cross-rack deployments (e.g., a 576-GPU cluster composed of eight NVIDIA NVL72 systems), copper-based interconnects will no longer be able to meet the required performance and bandwidth demands.

Optical transmission offers a clear advantage through wavelength-division multiplexing (WDM), which enables multiple wavelengths to be carried over a single fiber. This dramatically increases transmission density—an advantage that copper-based transmission cannot match.

Consequently, major CSPs are collaborating with emerging startups to develop new optical interconnect solutions, preparing for the next wave of bandwidth demand while laying the groundwork for the broader adoption of CPO technology.

Industry leaders deepen investments as AI infrastructure becomes more dependent on optical technologies

NVIDIA’s recent approach to CPO and silicon photonics involves utilizing TSMC’s COUPE 3D packaging technology to stack logic and photonics chips. By integrating 200G PAM4 micro-ring modulators (MRMs) onto the silicon photonics die, they enhance optical engine bandwidth density while keeping the size small and reducing power use.

NVIDIA has also recently announced a US$4 billion investment, split evenly between Lumentum and Coherent, alongside multi-year procurement agreements securing priority access to advanced laser and optical components. These moves indicate that NVIDIA is strategically securing critical components for future scale-up optical interconnects while taking a more active role in the development of next-generation laser and photonic technologies. It also signals that future AI compute infrastructure will rely increasingly on optical technologies.

TrendForce expects optical interconnect technologies based on silicon photonics and CPO to be first adopted for scale-out inter-rack data transmission in the NVIDIA Rubin generation. These technologies are also planned to be integrated into future scale-up interconnect architectures to enable higher bandwidth density. It is estimated that CPOs will account for only about 0.5% of optical transceiver modules used in AI data centers in 2026.

Scale-up optical interconnects spanning multiple racks could emerge as early as the Rubin Ultra or Feynman generations as silicon photonics and CPO packaging technologies mature. As data transmission bandwidth continues to increase, TrendForce forecasts that silicon-photonics-based CPO solutions could reach approximately 35% penetration of AI data centers by around 2030. Meanwhile, new optical technologies, including advanced optical interconnect architectures and optical I/O, are also likely to emerge.