AI:AM #4: Cameron on Model Consciousness, Duvenaud's Gradual Disempowerment, swyx's AI-Eng Alpha

This AI:AM highlights cut brings together Cameron Berg, David Duvenaud, Michiel Bakker, Shawn “swyx” Wang, and Bing Xu to examine what we understand about frontier AI systems and what happens as more decisions move into their hands. Berg grounds model-consciousness debates in experiments on architecture, agency, valence, and welfare, while Duvenaud argues that even well-aligned AI could gradually disempower humans through ordinary economic choices. Bakker frames Europe’s AI challenge as a sovereignty problem, and swyx turns to practitioner stakes around agents, evals, maintainable code, and who owns the system of record. Xu closes the loop at the infrastructure layer, arguing that self-improving compute and GPU-kernel automation may deepen rather than weaken the CUDA moat.

For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/ai-am-4-cameron-on-model-consciousness-duvenaud-s-gradual-disempowerment-swyx-s-ai-eng-alpha/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:

Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(00:38) Special Sponsor

(02:26) Model consciousness indicators

(10:47) Valence inside models (Part 1)

(17:09) Sponsor: Claude

(19:01) Valence inside models (Part 2)

(19:01) Misalignment and uncertainty

(25:16) Gradual disempowerment threat

(35:10) Slow zones and successors

(47:41) Europe's AI bind

(55:13) Frontier code benchmarks

(01:01:59) Routing and memory

(01:10:42) Agent infrastructure strain

(01:16:25) Self improving infrastructure

(01:27:56) Routing compute costs

(01:35:39) Sovereign AI financing

(01:42:24) Judging AI judges

(01:47:26) Building AI DNA

(01:52:38) Episode Outro

(01:55:09) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk