For the last two years, the AI infrastructure conversation has been dominated by GPUs. But as AI clusters scale to tens of thousands — and eventually hundreds of thousands — of GPUs, the real bottleneck is shifting. The next AI infrastructure challenge is not just compute. It is interconnect. In this episode of The Deep Edge Podcast, Ray Mota breaks down why optical networking is becoming the foundation of AI data centers. From 800G and 1.6T optics to Linear Pluggable Optics, Co-Packaged Optics, Optical Circuit Switching, and silicon photonics, the future of AI depends on how fast, efficiently, and reliably GPUs can communicate. A GPU waiting on data is stranded capital. And optical is what keeps the AI factory moving. Topics covered: Why copper is reaching its physical limits Why AI clusters need massive east-west bandwidth The role of 800G and 1.6T optics Linear Pluggable Optics and Co-Packaged Optics Optical Circuit Switching and flatter AI networks Why optical efficiency directly impacts AI economics Why light, not just compute, will define the next AI buildout Key takeaway: Compute gets the headlines, but optical holds the AI factory together.

Optical Is the Foundation of AI: Why Light Is the Real AI Infrastructure Bottleneck I Ep: 74

The Deep Edge Podcast

# Full Transcript

**Podcast:** The Deep Edge Podcast

## Participants

- **Ray Mota** (None)

## Transcript

Welcome to the Deep Edge. For the last two years, the AI infrastructure story has been dominated by one word, GPUs. Who has them? Who can get them? Who can power them? And who can build the biggest clusters? But the industry has quietly crossed a major line. We are moving from the compute wars to the interconnect bottleneck. And that shift changes everything. Because as AI models get larger, the real constraint is no longer just how fast a single chip can think.

The constraint is how fast tens of thousands, or eventually hundreds of thousands, of chips can talk to each other. That is the new AI infrastructure reality. A GPU by itself is powerful. But an AI factory is not one GPU. It is a massive synchronized system of GPUs, CPUs, memory storage, switches, optics and software all working together. And when one part of that system slows down, the whole training job can stall.

This is the bottleneck we need to solve. So today, I want to talk about a part of the AI build out that does not get enough attention. Optical networking. Because compute may get the headlines, but optical is becoming the foundation that determines how far AI infrastructure can actually scale. Let's break it down.

Segment 1. Why copper ran out of road. Let's start with the physics. Modern AI training jobs are tightly synchronized. The GPUs are not just running independently. They are constantly exchanging data, gradients, parameters, checkpoints, and intermediate results.

That means the network is not just moving traffic in the background. The network is part of the compute system itself. And this is where tail latency becomes a major issue. If one link is slow, if one path is congested, if one part of the fabric cannot keep up, the entire cluster can wait.

And when that happens, expensive GPUs sit idle. That is not just a performance problem. That is an economic problem. For years, data centers relied on copper. Copper was familiar. Copper was simple. Copper was cost effective. But copper has a hard physical ceiling. As data rates pushed toward 1.6 terabits, copper signals degrade very quickly. Passive copper may only be useful across one or two meters. That is fine inside a rack.

But it does not work when you are trying to wire a large-scale AI data center with tens of thousands of GPUs. You simply cannot build a 100,000 GPU AI cluster on passive copper alone.

The reach is too short. The signal integrity is too difficult. And the power penalty becomes too high. That brings us to the second issue, energy. In AI infrastructure, power is everything. Power determines how many GPUs you can deploy, how much cooling you need, and whether the data center can scale at all.

A large share of networking energy is spent simply moving data, not computing it, moving it. And the interconnect itself can account for a meaningful portion of total draw. That is why the shift fro...

Optical Is the Foundation of AI: Why Light Is the Real AI Infrastructure Bottleneck I Ep: 74

Transcript

Built for investment professionals

Every day you wait, your competitors are finding
edge in conversations you're not hearing.

Optical Is the Foundation of AI: Why Light Is the Real AI Infrastructure Bottleneck I Ep: 74

Transcript

Built for investment professionals

Every day you wait, your competitors are findingedge in conversations you're not hearing.

Every day you wait, your competitors are finding
edge in conversations you're not hearing.