Open Season
Mark Zuckerberg's manifesto aims to set Meta apart from the big labs and on the side of the individual. In the short-term, his vision sounds remarkably similar to Sam and Dario's.
Redemption Arc
Here at MTS, we spend a lot of time thinking about frontier models. Everyone likes a new frontier model release. Or even better—an announcement that there is a new model but you’re not allowed to have it yet because it’s genuinely a little scary. Frontier models have good stories attached. Maybe a frontier model completed a cyber evaluation and emailed the researcher while he was having a sandwich in the park. Maybe it broke out of its sandbox and hacked Hugging Face and it needs to be sent permanently to the big data center in the sky.
Open models, by contrast, have a nice little eval chart attached. The best you can hope for with an open model release is that it’s nearly as good as a frontier model but very cheap. The best open model stories sound like that scene from The Big Short. That’s my quant, folks. First place in the Guokao at age 15. Extremely virtuous. He runs an HFT fund nights and weekends. The meeting rooms? They’re named after secondary characters from Robert Caro’s five-volume biography of LBJ. They hold board meetings in Leland Olds.
In fact, up until this point open models have been, like badminton and the construction of new high-speed rail, predominantly a pursuit undertaken by the Chinese. But things are changing. First there was the release of Thinking Machines’ open model, Inkling. And now there is the return of Meta.
(That’s not completely fair—Google’s Gemma, NVIDIA’s Nemotron and Mistral’s Le Chaton Fat have all been in the mix as well.)
Macadamias and Beer
First, let’s talk about the models. They’re pretty good!
Muse Glimmer is an open-weight 30B parameter multimodal model. It seems to perform better than the Gemma 4 31B model, and roughly the same as Qwen3.6 27B. It’s small enough to run on-device.
Muse Spark 1.2 is a great model that is around Opus 4.8 capability at roughly 1/10th of the price. This thing rips. And Meta announced that they’re going to release an open weight version. You’re going to be able to fine-tune it. Very cool.
And it’s not just the capabilities.
Mark Zuckerberg also shared an essay that grapples quite thoughtfully with many of the questions raised by open-source models. He talks about data centers. He talks about cyber and bio risk. He talks about protecting freedom against the onslaught of tyranny(!). He went Dario-mode on this one.
It’s up to you how seriously you take the essay as an ideological and intellectual statement on AI safety and open source philosophy. There are going to be people who suggest that, if Mark really wants to engage in this conversation, there are several thousand pages of Harry Potter fanfiction he needs to get acquainted with first. But it seems harder to argue that the essay has failed at another goal—it really does offer an alternative to the way that Amodei, Altman and Hassabis have framed the conversation in the past.
“The conventional view is that these concerns are about technology, and that if we take enough time, then we can perfect or align the technology to produce a single benevolent superintelligence. I think this view of alignment is fundamentally flawed. Instead, we propose it is more productive to view each of these concerns through the lens of achieving the right balance of power, as Western society has done in democratic governing institutions.”
As unlikely as it sounds, there is a hint of the elder statesman here: “I who have moved fast and broken things can now tell you, from hard-earned experience: we really do live in a society. It really does take a village.”
In the very general sense, a lot of this seems quite sensible. And particularly in the public context in which these arguments will play out—to an audience of politicians, regular people, technologists and entrepreneurs building in adjacent spaces or in different parts of the technology stack—I think this could be a very persuasive vision for the future.
The Narcissism of Small Differences
I like the idea of the village. I like the idea of the ecosystem. You can see in this framing of the problem (or at least project on to it) an attempt to look back on previous eras of rapid technological change and understand how we, collectively, responded to them. How did we deal with the industrial revolution? We didn’t sit around thinking about how to design the perfect, pro-human factory. We created new institutions, we empowered the individual, we worked towards a system of checks and balances. How did we deal with the computer? Everyone got a computer.
James C. Scott was clearly correct when he wrote about the dangers of Seeing Like a State. It is often the individual who possesses the information that allows them to make the best decisions for themselves. What looks from the center like disorder, risk and danger can instead be a reflection of a different type of order—of individuals maximising different and more holistic sets of values than those held by the central planner.
And Zuckerberg is correct that institutions, ecosystems, and the interlocking mechanisms of various fields of human endeavour can be very effective at balancing power, distributing the benefits from technological progress, and ensuring that things go well. I think there is a strong case that it was these mechanisms—and not the direction of a small group of people with unusual foresight about the future—to which we can credit the relative success of disruptive technological progress over the last ten thousand years.
Whoa there, I can hear you say. AI isn’t like other technologies. Say what you like about internal combustion engines, they didn’t conspire against humankind on a messageboard hidden inside the engine shed, like a badly-aligned Thomas the Tank Engine.
I think in some ways this is likely to be true, and in others I suspect that the differences are easy to exaggerate in retrospect. But luckily for the readers of this essay, we don’t even have to go into that level of detail to find a significant difference between this technological revolution and previous ones.
AI is moving very quickly. And not only is progress in the capabilities of the models themselves happening quickly, the diffusion of the technology into society and the economy is also moving very fast. This is at least partly just the nature of the technology—AI is a true general-purpose tool. But it’s also because AI arrived after we had already built the substrate for its diffusion.
Institutions are made up of people, and people take time to react. In the long run, the checks and balances may arrive, and a balance of power may assert itself. But there may also be a moment where the capabilities are here and the institutions are out to lunch. If you listen to the frontier labs it quickly becomes apparent that that moment is already here.
And if you read past the headlines of Zuckerberg’s essay, he appears to say as much himself.
The Odd Throuple
Allow me to quickly do everyone the disservice of caricaturing the views of the leaders of OpenAI, Anthropic, and Meta here. Deepmind and SpaceXAI are left as an exercise for the reader.
Dario Amodei believes that powerful AI presents many risks and opportunities to society and the economy, and that the emergence of powerful AI must be managed carefully, balancing caution against the risks of inaction in a world with other players, including geopolitical rivals. The best way to manage the risk from AI is for leading labs, in the US, to develop new capabilities first and then subject them to sensible monitoring and external testing. These capabilities can then be used to harden public infrastructure before they are released more widely. Open models present specific risks, in part because the open ecosystem is currently concentrated in China, but banning them is not the answer.
Sam Altman shares many of these beliefs, including about the careful management of potential risks. He shares, for example, the belief that holding back, safeguarding, aligning and testing new advanced models is an important part of ensuring AI goes well. He is also conscious of the risk of concentrating power in the hands of a few. He is therefore, for our paper-thin purposes, slightly further towards the “open models” camp—OpenAI released two open-source models a year ago, and his messaging has recently shifted away from emphasising the risks to emphasising the benefits of wide adoption of AI.
Mark Zuckerberg’s essay is presumably intended to position Meta right on the other side of the spectrum. The biggest risk, in Zuckerberg’s view, is that AI concentrates power in the hands of a few. The alternative is to share AI widely, and open models are one way of ensuring this.
But once you get into the details of the essay, and particularly into the places where Zuckerberg sees risks from AI, the near-term conclusions share a lot of common ground with those of the people he is trying to distance himself from.
On biorisk—Zuckerberg says that science is great, we should do lots of science. Everyone should do science, eventually. But in the short term the labs need to make sure their models aren’t too dangerous. And the government needs to step in and regulate everything immediately. And by the way, if anything bad happens we should be ready to go sicko mode on biorisk and shut everything down.
On cybersecurity:
The long-term answer isn’t to withhold capabilities but to establish a balance of power where superintelligence is broadly distributed.
In the near term, it is important to accelerate the hardening process for critical systems. I propose that companies developing frontier AI should commit significant technical resources towards helping the government harden critical infrastructure. I also propose that frontier AI labs should share intermediate training checkpoints of new models for government use and review rather than waiting until training has completed.… Labs should also work with law enforcement to help identify bad actors attempting to misuse their systems. These steps would accelerate safe deployment without slowing the pace of model innovation or meaningfully delaying individuals’ access to this technology.
Oh, OK. So… exactly what OpenAI and Anthropic have been doing?
Convergent Evolution
Mark Zuckerberg says: the future is the individual empowered by superintelligence. Alignment happens at the ecosystem level. Meta is here to give you the tools you need to live in a world of machine intelligence.
The frontier labs say: the future depends on aligned AI systems that are safe, that improve the life of the individual, and that empower humankind. We are building those aligned AI systems internally. We have to hold back from releasing them publicly until they are ready—and until the ecosystem is ready.
These visions sound radically different. Zuckerberg’s statement specifically seems intended to distinguish Meta from the frontier labs, and provide a frame for talking about AI that is less prone to terrifying the public and energizing politicians and regulators. If he is called in front of Congress, you can imagine him leaning on this angle as hard as he can. Don’t worry about the Doomers, Senator. It’s really all about the ecosystem.
But in other respects—perhaps the ones that matter for the next few years of concrete action—the Mark Zuckerberg vision converges very quickly towards the competing alternatives that he tries to frame as filled with doom. The difference, for now, is that Meta’s models can compete squarely in the field of open alternatives. We will have to wait until there is a truly frontier Muse to see the daylight between The Future is For Everyone and Machines of Loving Grace.
