AI Book Club discussion of The Infinity Machine by Sebastian Mallaby

by Tom Johnson on Jun 28, 2026 comments
categories: ai • ai-book-club • podcasts

Note: These shownotes are AI-generated.

• Audio-only version
• Links mentioned
• Topics covered in this podcast
• Narrative essay version of the conversation
• Transcript

Audio-only version

Links mentioned

• The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence (Sebastian Mallaby — the book discussed in this episode)
• The MANIAC (Benjamin Labatut — recommended reading by Tom)

Topics covered in this podcast

• The complex character of Demis Hassabis — The group discussed how Hassabis comes across as an authentic figure driven by a desire to bring positive impact to society rather than to chase wealth or power, likening his sense of mission to the protagonist of Ender’s Game — a prodigy who feels destined to usher in AI safely before a worse actor gets there first.
• Character as the last safeguard — Participants traced the book’s underlying thesis: that once structural protections fail — ethics boards, regulation, the “singleton” dream of coordinated rollout — the safety of AI may come down to nothing more than the personal character and motives of its creators.
• Whether character even matters once AI is loose — A key counterpoint pushed back on that thesis: with eight billion people on Earth and models already proliferating, does it really matter who builds AGI first? Once a capability is out there, no single person’s virtue can control how it gets used.
• The acceleration of AI development — The discussion captured the vertigo of the current pace, comparing it to watching an oncoming train, and reflected on how dramatically the competitive landscape has shifted — from OpenAI’s early dominance to the present moment where Claude and others trade the lead.
• The role of government regulation — The group examined the growing involvement of governments, including a frontier model being pulled from access over official safety concerns, and debated whether deeper company–government partnerships (especially around defense) are becoming inevitable.
• AI, automation, and the future of knowledge work — The conversation turned to the unsettling ease with which AI handles real work — generating a detailed technical manual from a stack of installation photos in the time it would take a person hours — and the uneven, often-disputed productivity gains across teams.
• Training your own replacement — A sharper version of the job-displacement worry: when you write precise instructions teaching an AI to do your job, are you boosting your value or quietly automating yourself out of it?
• The rivalry between deep learning and reinforcement learning — Members found the “inside baseball” history fascinating — the competing camps, shifting theories, and decades of trial and error behind each leap to the next level of capability.
• The use of video games to train AI — The group discussed DeepMind’s strategy of using Chess, Go, and StarCraft as proving grounds, letting models learn through trial and error against clear win/loss conditions — and how often the team had to get it wrong to eventually get it right.
• Letting AI learn vs. over-specifying it — Building on the trial-and-error theme, the group debated a tension in writing agent skills: prescribe too loosely and results are inconsistent, prescribe too rigidly and you may foreclose the AI’s ability to figure out a better approach on its own.
• The merging of Google Brain and DeepMind — Participants reflected on the consolidation of Google’s two AI labs into one, noting how the union produced the Gemini model — whose name, meaning “twin,” fits its origin far better than “Bard” ever did.
• The opacity of AI companies — Even for those working inside big tech, the group agreed that the strategic decisions and inner workings of frontier AI labs remain remarkably hard to parse from the outside — which is part of what made the book’s behind-the-scenes account so valuable.
• The difficulty of forming an opinion on the book — A candid closing reflection: was the book simply more biography than argument, or does leaning on AI to do our thinking quietly dull our own capacity to react and judge?

Narrative essay version of the conversation

If the podcast were an article, this is what it would read like.

When Character Is the Only Safeguard Left

A sweeping new biography argues that nothing structural will save us from runaway AI — only the people building it. That should worry us more than it reassures.

There is a quietly terrifying admission buried inside the otherwise admiring portrait of Demis Hassabis: every guardrail anyone tried to build around artificial intelligence has failed. The ethics boards dissolved or proved toothless. The dream of a “singleton” — a single, coordinated body carefully ushering superintelligence into the world — shattered the moment ChatGPT turned a leisurely research project into a wartime sprint. Regulation moves at the speed of paperwork while the technology moves at the speed of capital. Strip all of that away, and what’s left holding the line between a beneficial intelligence and a catastrophic one is something startlingly thin: the personal character of a handful of founders.

That’s the uncomfortable center of gravity here. If the safeguards are gone, then the entire bet reduces to whether the right kind of person happens to be steering. And the case for optimism rests on a single biography. Hassabis drives a decade-old Audi. He doesn’t collect houses. He sold his company to Google not out of greed but out of exhaustion at perpetually begging investors for the compute he needed. His stated ambition isn’t wealth or even power — it’s to use AI to crack the unifying pattern behind physical reality, the theory of everything that has eluded science for generations. He frames himself, by way of Ender’s Game, as a reluctant prodigy fulfilling a kind of prophecy: the one who has to get to AGI first so that someone worse doesn’t. If you must wager the future on a single temperament, the argument goes, this is about as good a temperament as you’ll find.

But here’s where the reassurance curdles. The whole premise assumes that whoever builds AGI can also control how it’s used — and that assumption looks shakier by the month. There are eight billion people on this planet. Once a capability is loose, no founder’s virtue, however genuine, can recall it. The competitive landscape makes this concrete: DeepSeek and a wave of Chinese models now occupy the same frontier, ChatGPT and Claude trade the lead quarter by quarter, and the idea that any one conscience presides over the technology is already a fiction. Even the most responsible-looking move — a company dialing back a powerful model because it judged the release too dangerous — can be read two ways: principled restraint, or a defensive crouch to avoid blame when something goes wrong. Good character, it turns out, is not the same thing as control, and the book never quite resolves the gap.

What makes this more than an abstract worry is the velocity. The race isn’t just continuing; it’s compounding. The breakthroughs that felt like distant milestones — a model that reasons step by step before answering, agents that act rather than merely respond — keep arriving faster than anyone budgeted for. And the asymmetry is the part that should keep policymakers up at night. A technology this capable hands outsized influence to small actors: a modest country directing swarms of autonomous drones, a lean startup reshaping an entire industry, a single knowledge worker producing in twenty minutes what used to take a day. That democratization of leverage is exactly why governments, after years of deregulatory cheerleading in the name of staying competitive, are suddenly reaching for blunt instruments — abrupt model suspensions, national-security export controls, the leverage of defense contracts that companies can’t easily refuse.

The discomfort sharpens when you turn the lens on the people doing knowledge work. There’s a genuine unease in watching a tool ingest a hundred photographs of a hardware installation and produce a polished step-by-step manual in seconds — or absorb an entire dense book and return analysis sharper and better-sourced than a careful human reader could manage after a week. The instinct is to reframe this as liberation: automate the mundane, ascend to the complex, expand your value. Maybe. But there’s an honest second reading that the rosy framing skips. When you write meticulous instructions teaching an agent to migrate an SDK or document a deprecated API exactly the way you would, you are, in a real sense, training your own replacement. The productivity gains are not evenly felt or evenly believed — some practitioners report modest lifts, others quietly clear their backlog in an afternoon — but the trajectory points one direction, and pretending otherwise is its own kind of denial.

And there’s a subtler trap inside the craft itself. The discipline of writing precise, granular instructions for an AI — the dominant mode of working with these systems today — may run directly against the grain of what made the technology powerful in the first place. The deepest breakthroughs came not from programming machines with human knowledge but from letting them discover it through trial and error, the way an agent taught itself to dominate a board game by playing itself millions of times. If the future belongs to systems that learn on their own, then over-specifying every step is a comfortable habit that quietly forecloses the more radical possibility — a tool that figures out the job better than we ever could have told it to.

The honest unease isn’t whether the machine is smart enough. It’s already running circles around us, and we know it. The real question is what happens to a society organized around human expertise when expertise becomes the cheapest input in the system — and whether we’ll notice the floor shifting before it’s already gone.

Transcript

Tom (00:03:22)
Welcome everybody to this book club. Today we’re discussing Sebastian Mallaby’s The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence. Just a reminder, I do record these book club meetings and send them out, so just keep aware of that. Make sure you don’t say anything you’ll regret. Nobody ever does. I throw that out there just because I don’t want to do a bunch of post-processing. This book was a long book. We’ve got at least 400 pages of reading material plus a bunch of footnotes and other things in the back. I’ll admit I read the first half in print and the second half I listened to via audiobook. I somehow tapped out with my reading patience, but really enjoyed it. Overall general reaction, would you give this a thumbs up, thumbs down, or a neutral thumb?

Sharon (00:04:30)
I give it a thumbs up.

Tom (00:04:32)
Well, just curious why a thumbs up? Any particular reason?

Lois (00:04:37)
I thought it was really well written, that he explained things really well, that he gave a lot of perspectives. Very thoughtful.

Tom (00:04:47)
Yeah. There’s a lot of detail. You can tell this is a serious work of reporting and investigative journalism. Any other particular reasons why you give it a thumbs up?

Lois (00:05:01)
Well, I guess I’m just really interested in the subject area.

Geoffrey (00:05:10)
I mean, the story is so relevant to what’s happening today with how AI is integrating into our lives. Learning about the history of how it was created and the thought process that Hassabis—is that his last name?—had in developing his AI program and how it evolved. Also how he stayed in control of the company even after Google bought it. I think is really interesting and important for me to have a trust in Google’s AI, how he made a board of advisors to make sure that it didn’t lose the safety that he wanted to keep within it.

Tom (00:06:46)
Yeah. Lois and I are both Googlers, and I don’t know if anybody else has worked at Google, but it was a great book for filling in the gaps of a lot of things that have happened in the last 5 years. You mentioned this desire that Hassabis had—he definitely wanted to form safety boards and put guardrails around AI. Although ultimately that sort of didn’t work out to the same strength that he wanted. And then when ChatGPT came out, it became a code red and converted the peacetime mode into a wartime mode where they had to move a lot faster. I was trying to tease out the book’s overall argument, because it comes across as very much a biography. In contrast to other books that we’ve read like Karen Hao’s Empire of AI, where she portrayed Sam Altman as a power-hungry, manipulative, untrustworthy person, Hassabis comes across pretty positively. That’s something that a lot of critics mark Mallaby down for—it’s too positive of a portrayal. I have to wonder in the back of my mind, it’s one Brit writing a biography of another Brit, and so much of the AI conversation is about American AI behemoth companies, and yet this kid from London is the brains behind so much that’s driving it. But anyway, what is the author’s overall argument? Does anybody have an idea? What is Mallaby really trying to say?

Bianca (00:09:17)
I think this one’s definitely complicated. I don’t know if I really considered what his thesis was. And also disclaimer, I did not finish the book yet. I made it pretty close, but not all the way there. But I would say that one of the points he was trying to communicate is that AI is this great big advanced concept, but it is also ultimately created and run by imperfect humans who each have their own agendas and pros and cons and complexities. So I feel like there’s something going on that there’s still this humanity underneath everything. It was also interesting that you pointed out that maybe the author was too nice to his subject. Hassabis agreed to participate. He agreed to be interviewed. It’s an uncanny situation to be in close proximity to your subject and try to tell the truth but also not do a smear campaign against them after going out for lunch several times. So those are my thoughts.

Tom (00:11:24)
Yeah, the human element is definitely something I agree with. One could argue that Mallaby’s general thesis is that so many of these structural safeguards sort of fail. The idea of an ethics board didn’t hold up. Regulation is just moving things through. The singleton idea that you would have a single body rolling out AI fell apart with the competitive landscape. So you’re left with just the character of the people building it. Mallaby’s argument could be that Hassabis is about as good of a person as you could hope to find to steer AI. The author’s argument really gets into what is the motivation of Hassabis. What’s driving him? We see it at the start with the comparison to Ender’s Game. He’s this boy prodigy who feels like it’s his mission to bring about AI. He’s compromised in terms of having to sell his soul to Google for the compute and the resourcing. He doesn’t want to be endlessly begging people for money. But over time, Hassabis tries to answer deep questions with AI. He wants to understand the unifying pattern behind reality in physics—the unifying theory of everything that has eluded scientists for generations. He’s not a wealth monger. He drives a 10-year-old Audi. He doesn’t have five houses. He capitulates to the corporation out of necessity, but at his core, his character and motive are pure. Maybe that’s enough to steer AI in a way that doesn’t end badly for us. What do you think of that argument?

Geoffrey (00:13:53)
I really appreciate what you said about his authenticity. Like, I see him as very authentic with how he presents himself. I’ve seen a lot of interviews with him talking about his goals and his view that his life goals are pursuing AI to bring to the world. One of the things that I caught on with the book was when he was talking about how he learned about Ender’s Game. I think he read it in his 20s. The book is much better than the movie. When I first read it in junior high, I read it in two and a half days because I was so engrossed in it. Seeing him latch on as somebody that is almost fulfilling a prophecy—not in a spiritual way, but more as a way to bring positivity to society—I see that in the interviews I’ve watched and the way he addresses his lifestyle. So I really appreciate that about him.

Tom (00:15:07)
Yeah. That Ender’s Game analogy gets to the core of who he feels he is. Is he fulfilling a prophecy? The chosen one to lead us safely into AI over bad actors getting AGI first. I feel like that’s a resounding theme. If you remember some other books we’ve read, there was a big division between Elon Musk and Sam Altman about who should lead OpenAI because you want to have the right leader to usher in artificial general intelligence. At the hands of the wrong person, it could be devastating.

Lois (00:17:15)
Aren’t we really making an assumption that once it’s out there, anyone can control it? Does it really matter who is in charge? There’s 8 billion people on Earth. How is that one person going to stop it from being used badly?

Tom (00:17:32)
Yeah, that is a good argument. What about with Anthropic and how they dialed back the rollout of Mythos because they felt like it could be too harmful? That could be an example of one company taking a more responsible position. Although they may be doing a CYA job to protect themselves from massive fingerpointing. But you’re right. Does it really matter who develops AGI? Once it’s out there, it’s out there. The Chinese models, DeepSeek and others, are right in the same competitive sphere. It’s no longer one person steering the show. What else about the character of Hassabis did you find interesting? What about his chess background? He loves competition and loves to win. He’s going to go all out to move things forward. It’s interesting because there was a time not too long ago where we didn’t have as much compute resources. So I can see where this incredible drive to roll out a superior model could be a good trait if you’re trying to get to AGI. Do you think the race is accelerating even more so than it was a year or two ago? The race between OpenAI, Anthropic, Google, DeepSeek?

Bianca (00:20:11)
It does seem like it’s accelerating quite a bit. It was interesting reading this book. I feel like you’re watching a train drive past you. You hear it coming and you reflect when he’s talking about what’s going on between 2012 and now. It’s like, I remember where I was when that happened. We had no idea at the time that all this competition was going on between these companies. Seeing it meet at present time and see we’ve got Claude, we’ve got GPT, we’ve got Mythos, we have DeepSeek, and thinking about what’s ahead. Seeing all this play out in the public sphere and the fighting between these companies is really interesting. As a tech writer, it felt both the benefit and the struggles that come with it. I’m so curious to see what will happen in the near future because a month from today the landscape could be very different, especially in terms of regulation. So fascinating times we live in for sure.

Tom (00:21:18)
I definitely was feeling like the acceleration is moving a lot faster just in preparing notes for this book club. It is mind-blowing to me that I can find a PDF of the book, upload it into NotebookLM or Claude, and ask it some questions, and 20 seconds later it’s read the entire book. Its analysis is so sharp, spot on, and supported by textual evidence. We are not going to compete against the machine. The machine is really smart. It is humbling to feel like there’s literally a smarter intelligence running circles around us. At what point does it start directing and guiding us? One question I had was where are we heading? There was a big emphasis in the book about Google’s chain-of-thought reasoning that was another step forward for all these models. Where do you see the long-term trajectory in 5 or 10 years at this pace? One trait that I think is emerging is this idea of outsized influence. You could have a small country in possession of advanced AI controlling drones, or small companies like Anthropic or OpenAI exerting massive transformative impact.

Bianca (00:26:25)
I don’t dare to guess what will be going on in 5 years, but I think there will continue to be seesawing of who has the most power. A few years ago, everyone associated AI with OpenAI and ChatGPT. Now I would say the pendulum has swung to Anthropic. Claude and Claude Code have become so usable, and within the programming and tech writing sphere, Claude is the go-to. Whoever is in power is being very intentional, laying down the groundwork, and playing the long game. There are people making connections, networking, amassing power, and leveraging that to move their company forward. When DeepMind brushed off ChatGPT, that was possibly a mistake on their part. The people thinking ahead will be the ones who come out on top.

David (00:28:44)
Another thing to consider is government involvement. Governments might think this technology is too powerful not to get involved and take a stake. Like what we saw happen with Intel—this company can’t fail if we want to be competitive. Especially for military and defense, they want to call the shots. If a company says they aren’t interested in that work, the government can say they won’t do business with them. Maybe it’s more attractive for companies and governments to work together.

Tom (00:29:57)
The role the government plays is interesting. Previously, the US government tried to reduce regulation to stay competitive. But they’re going to have problems if they don’t think through what happens with massive unemployment due to AI. I’ve noticed more Chinese vloggers describing how factory jobs are disappearing and being driven by AI. If they automate too many processes and people are unable to survive, that will cause massive chaos. The government is going to be pulled into this more and more.

Geoffrey (00:31:12)
There are two things that caught my interest. One is government control, and the other is robots. Some governments are trying to understand how much control they can have. Claude has this notice that Claude 5 is currently unavailable because the US government decided it was not safe to be used, possibly because Claude didn’t want to continue working with the US government on certain projects that went against their moral ethics. You can see how the government is constraining certain companies. Also, the videos of humanoid robots in China are insane. AI is being used to program these robots to function independently. We’re going to have mass layoffs because these robots can work 24/7. Even our computer jobs are being taken over. I do instruction manuals, and I tested out Claude by uploading over a hundred pictures of a detailed installation. It wrote all the directions for me, which would have taken me 10 hours. It’s just done. It’s very humbling.

Tom (00:35:24)
Geoffrey, you mentioned you’re a designer. What kind of design work do you do?

Geoffrey (00:35:30)
I’m a display designer. I design trade show displays and point-of-purchase displays. We hire companies to install them in stores like Best Buy, and I physically take pictures of the installation.

Tom (00:36:29)
Cool. The humanoid robot thing has me really curious. And how these AI tools can do so much of our work is profoundly unsettling. What exactly is our Plan B when this becomes too easy? Working at a company with hundreds of tech writers, I would say most people don’t see a productivity boost of more than 20%.

Lois (00:37:34)
I’m seeing way higher than that. I will provide evidence.

Tom (00:37:40)
Yeah. I was looking at my change list count and my trajectory has been going up. I’m writing skills and trying to speed things along. The latest project I did very fast, and it’s good enough. Let’s jump back into the book. Geoffrey Hinton made an appearance. He’s known as a godfather of AI. He predates the transformer breakthrough and connected neural networks to AI, patterning it after the human brain. He used to work for Google but left to be more vocal about cautionary tales. Once AI turns evolutionary with survival instincts, he feels we’re cooked. Did the Hinton part jump out at anybody?

Lois (00:40:00)
I found it really fascinating, especially the rivalry between the deep learning community and the reinforcement learning community. It’s such an inside baseball type of thing.

Tom (00:41:08)
Yeah, these different camps about how to best move AI along. Trial and error versus pattern recognition. We don’t really know what’s going on behind the scenes that enables the level change up. Hassabis had various theories on how to get to that next level. Earlier, he was convinced that patterning AI after the human brain would be key. There was a time when he wanted to construct learning as a video game. People are constantly looking for the next breakthrough. The author really gets into a lot of this detail, which makes the book long, but it’s hard to retain it. It’s fascinating how things have constantly been changing and evolving.

Bianca (00:44:16)
It was interesting how prevalent games were. They used games to test hypotheses—can this AI tool learn how to play this game? Going from Chess to Go, to StarCraft. You can have them play human players as well. The other thought I had was how often you have to get something wrong to get something right. The obsession within DeepMind to try to beat StarCraft with reinforcement learning, and seeing it not go the way they thought. But having the time and money to spin their wheels for decades eventually led to breakthroughs.

Tom (00:46:13)
I’m fascinated by the ability of these tools to learn, and video games give such a clear picture of winning or losing. At some point, instead of programming tools with knowledge of the world, they let the tools discover knowledge on their own through trial and error. As we build agent skills, are we committing a flaw by making detailed instructions and not letting the AI learn on its own? I don’t want to make my skill instructions too explicit and granular, preventing it from thinking more freely. Are you writing skills, Bianca?

Bianca (00:48:55)
Yeah. As tech writers, we have done a lot of prompt engineering. I now write very explicit instructions for skills because I started out too broad and the AI misinterpreted things. So I overcorrected and became too explicit, and then it got confused. It’s interesting finding the balance because we want repeatable, consistent results. If you don’t overprescribe, the structure could be wildly different. I haven’t heard anyone make the argument that we want to empower the AI to learn its own skill, but it’s an interesting observation.

Tom (00:49:56)
It’s cool to hear experiences of other tech writers having the same issues. I’ve got policy documents that I want to look somewhat the same, so I am very specific, but it’s a constant battle. Any other themes about this book?

Geoffrey (00:51:25)
If I’m telling AI exactly how my job is done, is it learning my job so that I won’t have as much to do? People are being paid to train AI how to do their own job skills. It helps in the short term, but you’re teaching it to take over.

Tom (00:53:51)
I definitely have conflicted feelings about training AI to do my job. If you can automate mundane stuff, you can focus on complex tasks and expand your value. After reading this book, I was troubled by my lack of a strong opinion on it. Am I dumbing down because I have AI do so much thinking for me? Or is this just a book that doesn’t provoke a strong reaction? I really enjoyed the historical aspects, especially working at Google. I remember when Google Brain and DeepMind merged. Gemini, meaning twin, tracks much better than Bard as a name for the model born from that merger. So, our next book will be The MANIAC by Benjamin Labatut. It’s a short work of fiction about an obsessed scientist, which will be a nice mirror into some of these AI founders. Any other book recommendations?

Lois (00:58:04)
I can’t recommend it to a general audience, but The Proof is in the Code was interesting, especially with the recent math discoveries by OpenAI and Google DeepMind.

Tom (00:59:07)
Okay, we’ll check it out. If you have other recommendations, float them in the Slack group. Any last thoughts?

Lois (00:59:07)
It was interesting the interaction with Peter Thiel, given how central he’s become to recent discourse. He’s been giving lectures about the antichrist and moved to Argentina.

Tom (00:59:50)
Peter Thiel, founder of Palantir. Yeah, he’s been on some podcasts. I don’t know if we want to dive into the antichrist, but he would be interesting. Thanks for bringing that up. Well, I hope you have a great rest of your weekend. Thanks again for coming and for your insights. Bye-bye.

Lois (01:00:45)
Bye.

Comment on LinkedIn