Key Topics: AI infrastructure evolution Server and rack engineering innovations Analog components and power management Semiconductor industry growth and GPU impact Enterprise and hyperscaler strategies for differentiation Cooling technologies and hybrid solutions Networking and infrastructure for AI workloads Multi-cloud and on-prem AI deployment trends

EP 180: HPE Reflections on AI Compute Racks, Analog Analog Analog Semis.

The Circuit

# Full Transcript

**Podcast:** The Circuit

## Participants

- **Ben Bajarin** (The Circuit)
- **Jay Goldberg** (The Circuit)

## Transcript

Hello everyone, welcome to another episode of The Circuit. I am Ben Bajarin. Greetings, programs. I'm Jay Goldberg. So, there wasn't a huge news week. However, I was at HPE Discover and had a number of interesting observations and takeaways. I sort of joked if anybody saw, you know, on Twitter that this is my life now when I see an open compute tray.

I put my reading glasses on and my phone into magnification mode and I go look at silicon, names of vendors on silicon, MLCCs, power capacitors, you name it. Which, you know, is partly the, I like to see how these things are designed and built and the engineers on the floor are generally very excited to answer very technical questions.

Something that I greatly appreciate about these shows. Um, cause they're not just, oh, this is what it can do. It's like, oh, that's why this is where the power thermal lays. We designed it this way so that we could squeeze out X, Y, Z of efficiency or network latency. I'm like, this is, this is the greatest conversation ever.

Um, but, but there was a couple things I'm taking away from this and I don't know how to sort of frame this in the analysis because it is technical, but basically, you know, there's a discussion out there around how everybody designs their infrastructure. So, you know, we know, right. Amazon, uh, with AWS or Microsoft with Azure or Google with GCP, you know, they design a lot of these racks, right?

They don't just buy, you know, NVIDIA GPUs and go with everything or even for their TPUs, right? They are engineering the crap out of these, uh, racks for something, you know, again, everybody's variable changes. Maybe it's performance, maybe it's latency, maybe it's efficiency, you know, you, you name it, right? Maybe it's to use less water. Maybe it's to use a little bit of water and more fans.

Like everybody takes an approach. And I think that's interesting. Um, and again, kind of going back to our Computex take, you know, I did this same thing. It's just harder when you're looking at a, at a laptop, right? Even in Computex in those days, like very rarely where they're like letting you just see the inner workings of a laptop.

But, but even if they did, there really wasn't like that much differentiation, right? There wasn't a Acer or Dell or Lenovo, like is just eking out drastically better performance with their Intel SKUs or drastically better battery life, right? It was roughly the same. Um, this, this is very different. Like people are really architecting solutions.

And so for example, like HP has this hybrid design, they call it. It's a four rack pod and it is a mix of air and liquid cooling. So what it can do is it obviously can do more cool, more Watts than an air fan and an air only implementation, but it can't cool as much as a pure liquid. But, but that's actually very interesting because, you...

EP 180: HPE Reflections on AI Compute Racks, Analog Analog Analog Semis.

Transcript

Built for investment professionals

Every day you wait, your competitors are finding
edge in conversations you're not hearing.

EP 180: HPE Reflections on AI Compute Racks, Analog Analog Analog Semis.

Transcript

Built for investment professionals

Every day you wait, your competitors are findingedge in conversations you're not hearing.

Every day you wait, your competitors are finding
edge in conversations you're not hearing.