Speeding AI Compute, Intel Debuts First Integrated Optical I/O Chiplet

Intel claims the optical compute interconnect chiplet will “revolutionize high-speed data processing” of AI loads in data centers and HPC.

Intel recently developed an optical chip interconnect system and demonstrated its first fully integrated optical I/O (OCI) chiplet.

This chiplet addresses one of the biggest challenges facing computer architects in the age of AI: the energy and latency cost of traditional electrical interconnects. Parasitics increase with shrunken process nodes, higher clock frequencies, and larger SoCs, causing data movement energy and latency to spike as well. These costs are not acceptable for AI workloads that rely on the movement of large amounts of data between disparate systems in a heterogeneous computing platform.

At the Optical Fiber Communication Conference (OFC) 2024, Intel’s Integrated Photonics Solutions (IPS) Group ran live data on its OCI chiplet co-packaged with an Intel CPU. Intel believes this chiplet may revolutionize high-bandwidth interconnects by allowing them to be co-packaged with optical input/output (I/O) in next-generation data centers and high-performance computing (HPC) applications for AI.

Intel Makes a Leap in Chip-to-Chip Comms

Intel claims its new OCI chiplet represents a significant advancement in chip-to-chip communication. At the heart of this technology is silicon photonics, which fabricates optical components using standard silicon semiconductor manufacturing processes. This creates highly compact, efficient, and cost-effective optical devices that can be integrated seamlessly with electronic circuits.

Intel’s new chiplet co-packages these optical transceivers with the CPU on a single substrate, reducing the physical distance between the CPU and the optical components and thereby minimizing latency and power consumption during data transmission. The co-packaged design also enables efficient heat dissipation since the proximity of the components reduces thermal resistance. This helps maintain optimal operating temperatures and ensure reliable performance even under heavy computational loads.

Intel reported the performance specifications of its first OCI implementation. The solution delivers up to 4 Tbps of bidirectional data transfer. It features a live optical link with a transmitter and receiver connection between two CPU platforms over a single-mode fiber (SMF) patch cord.

Other notable performance highlights include an optical spectrum with eight wavelengths at 200 GHz spacing on a single fiber. It also supports 64 channels of 32 Gbps data in each direction up to 100 meters, using eight fiber pairs. Each carries eight dense wavelength division multiplexing (DWDM) wavelengths. Remarkably energy efficient, it consumes only 5 pJ per bit compared to 15 pJ/bit of pluggable optical transceiver modules.

The Need for Optical Interconnects

The OCI chiplet may significantly impact the efficiency and performance of AI systems.

Modern AI workloads rely on efficient data management and transfer across various system components, including CPUs, GPUs, and memory. As AI models become increasingly complex, the volume of data they process skyrockets, leading to significant data transfer and bandwidth challenges. Unfortunately, traditional electrical interconnects, which have served the industry well for decades, are becoming bottlenecks that hinder the scalability of AI systems.

Electrical interconnects are typically limited by their physical properties. As process nodes shrink and data rates and chip-to-chip distances increase, traditional interconnects suffer higher parasitic impedances. And, as signals face greater resistance and capacitance, more energy is lost in data movement, decreasing efficiency and increasing heat generation. Moreover, higher data rates necessitate more robust error correction mechanisms, which add latency and power consumption. In large-scale AI applications, these inefficiencies can lead to significant performance degradation and make it challenging to meet the demands of real-time data processing and analysis.

Optical interconnects are a promising solution to these challenges. Leveraging light to transmit data, optical interconnects offer substantially higher bandwidth compared to their electrical counterparts. This is due to the higher frequency of light waves, which can carry more information per unit of time. Optical signals also encounter minimal impedance when traveling through fibers, making them a more energy-efficient means of data transfer.

Implementing optical interconnects in AI systems can dramatically enhance data transfer rates, reduce latency, and enable more efficient parallel processing.

A Path to Faster AI Compute?

As the demand for AI continues to grow, the scalability provided by optical interconnects allows systems to evolve without being constrained by the limitations of electrical interconnects. While the technology is still in its nascent stages, Intel’s chiplet proves that it is possible to implement in practice, and that potential is worth pursuing.

Share to your social below!