Dylan Patel, founder of SemiAnalysis, argues the biggest gains in AI don&#x27;t come from faster chips, they come from software-hardware co-design. Optimizing the model, the kernels, and the silicon together turns a 2x here and a 2x there into 100x. He explains why DeepSeek&#x27;s experts were shaped for Nvidia&#x27;s Hopper (and why TPUs struggle to run it), why OpenAI&#x27;s sparser models and Anthropic&#x27;s denser ones pull them toward different hardware, and why the so-called CUDA moat was never really about CUDA. Dylan breaks down InferenceX, his living benchmark that runs the latest models on over $50M of donated hardware daily, tracking a roughly 60x annual drop in cost per unit of quality. He makes the case that inference will be a bigger market than oil, that the compute crunch persists because models expand the value of useful work faster than compute grows, and why Jensen Huang is bankrolling neoclouds to engineer a multipolar world.<br><br>Hosted by Shaun Maguire and Sonya Huang, Sequoia Capital

Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis

Training Data

# Full Transcript

**Podcast:** Training Data

## Participants

- **Sonya Huang** (Sequoia Capital)
- **Shaun Maguire** (Sequoia Capital)
- **Dylan Patel** (SemiAnalysis)

## Transcript

[SPEAKER_01] I think it's really fun inside of SemiAnalysis because we have 90 people and like a big chunk of them are technologists, engineers across the whole supply chain. And then a big chunk is people who are formerly at hedge funds. And you see these arguments like people are like, oh, that doesn't matter. And it's like, then someone's like, well, but cost. And then someone, the engineer's like, no, no, no, but this technology is the coolest. And you see this organically like fight it out. And we're pretty informal. And, you know, given the fact that I was a forum moderator is you can imagine what this internal insight looks like. You're enjoying it.

[SPEAKER_01] You don't wrestle with the pig because the pig enjoys it. Exactly.

[SPEAKER_00] This with Dylan Patel. You know, I'm Sonya Huang from Sequoia. My partner, Sonya Huang. It's pretty insane what you've done. Semis five years ago were not very sexy in the West.

[SPEAKER_00] They were sexy in the East, but people here in the West had kind of forgotten about them. You did not forget about them though. You went very long. You created probably the premier research company in the space. It's been educating the world and, you know, the state of the art from very technical details to supply chain, you know, to the bigger picture. There's rumors that semi-analysis recently passed a hundred million of revenue. I don't know how accurate those are, but whatever the numbers are, you guys are crushing. It's as accurate as the information is. Yeah.

[SPEAKER_00] You know, you never, you never know. There's also rumors that you might start a venture funds. Like, you know, I, I hear all the time in the ecosystem, people wanting, you know, affiliation with semi-analysis. You've, you've built this trusted brand. And so whatever you do, it's working. It's clearly like just the beginning of the journey for you. Congratulations on that. But how did this happen? Like, how did you first question is like, what is the background? How did you kind of get to where you are now?

[SPEAKER_01] Well, when I was a young boy and, you know, coming out of the womb. So, so, okay. So I grew up in like a small business. My parents had a motel. We lived in the motel. We laid our gas station. So, you know, I was selling, you know, I joke a lot of times the first neural network I trained was racially and visually profiling people based on when they entered the gas station, which cigarette to pick. Basically, you know, the cigarettes were all extrude across the top and I was too short to actually like, you know, reach them.

[SPEAKER_01] And technically it wasn't legal to sell cigarettes at that age, but whatever. I had to move the step stool over to the right area. I started working in my first office before it was legal too. So it was a g...

Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis

Transcript

Built for investment professionals

Every day you wait, your competitors are finding
edge in conversations you're not hearing.

Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis

Transcript

Built for investment professionals

Every day you wait, your competitors are findingedge in conversations you're not hearing.

Every day you wait, your competitors are finding
edge in conversations you're not hearing.