Overview

In this solo episode of the Dwarkesh Podcast, host Dwarkesh Patel delivers a tightly argued essay examining the recent conflict between Anthropic and the Pentagon, using it as a window into what he calls "the most important question nobody's asking about AI": to whom should powerful AI systems be aligned? The stakes are nothing less than the character of future civilization, where AI will constitute the workforce, the military, and the infrastructure of governance itself. Patel walks a careful line—critical of both government overreach and industry naivety—while arguing that the structural properties of AI technology inherently favor authoritarian applications like mass surveillance, and that neither corporate resistance nor government control offers a clean solution.

---

0:00The Pentagon vs. Anthropic: A Warning Shot

Patel opens by recounting that the Department of War (a deliberate archaism for the Department of Defense) has declared Anthropic a "supply chain risk" because the company refused to remove red lines prohibiting use of its models for mass surveillance and autonomous weapons. He immediately clarifies that he understands the Pentagon's position: if he were Secretary of War, he too would refuse to rely on a private contractor that reserved the right to cut off access to mission-critical technology. He draws an analogy to Elon Musk hypothetically reserving the right to cut off Starlink access during an "unjust war"—a position no military could accept.

However, Patel argues that the government went far beyond simply refusing to do business with Anthropic. Instead, it threatened to destroy Anthropic as a private business using legal instruments originally designed for other purposes. The supply chain restriction, if upheld, would force companies like Amazon, Nvidia, Google, and Palantir to ensure Anthropic is not touching any of their Pentagon work. Patel warns that in a future where AI is woven into every product and service, it may become impossible for Big Tech companies to cordon off their use of Claude from their government contracts. When forced to choose between their AI provider and the Pentagon—which represents a tiny fraction of revenue—Patel predicts companies would drop the government.

This raises an uncomfortable question: what is the Pentagon's plan? To coerce every company that refuses to do business on the government's terms? Patel connects this to the broader geopolitical context of the AI race with China. The reason Americans want to win this race, he argues, is to avoid a world where a government believes there are no truly private citizens or companies, and where refusal to provide morally objectionable services invites destruction. "Are we really racing to beat China and the CCP in AI," he asks, "just so we can adopt the most ghoulish parts of their system?"

---

4:16The Overhangs of Tyranny: Mass Surveillance at Scale

Patel pivots to a deeper structural concern: mass surveillance is already legal in many forms, but has been practically impossible to enforce at scale. Under current law, Americans have no Fourth Amendment protection for data shared with third parties—banks, ISPs, phone carriers, email providers. The government can purchase and read this data in bulk without a warrant. What has prevented a surveillance state is simply the manpower bottleneck: no agency can monitor every camera, read every message, and cross-reference every transaction.

AI eliminates that bottleneck. Patel offers a concrete calculation: there are 100 million CCTV cameras in America. Using open-source multimodal models at $0.10 per million input tokens, processing one frame every 10 seconds at roughly 1,000 tokens per frame, the total cost to monitor every camera in America would be $30 billion. But AI capability gets 10x cheaper every year. Next year it would cost $3 billion, the year after $300 million, and by 2030 it would be cheaper than remodeling the White House. "Once the technical capacity for mass surveillance and political suppression exists," Patel warns, "the only thing that stands between us and an authoritarian state is the political expectation that this is just not something we do here."

This is why Patel finds Anthropic's stand valuable: it helps set a norm and precedent. But he immediately complicates the picture by noting that the government has far more leverage than most people realize. Even if the supply chain restriction is reversed—prediction markets at the time of recording gave a 74% chance of that happening—the president has many other tools: control over permitting for power generation (needed for data centers), antitrust enforcement, and contracts with other tech companies that could be conditioned on cutting ties with Anthropic.

---

5:54AI Structurally Favors Mass Surveillance

Patel introduces what he considers the more fundamental problem: even if all three leading AI companies (Anthropic, Google, OpenAI) drew red lines and were willing to be destroyed to maintain them, the technology itself structurally favors mass surveillance and population control. By 2027 or 2028, open-source models will match the performance of today's frontier models. The government can simply say, "I'll use an open-source model that might not be the smartest thing in the world, but is definitely smart enough to monitor a camera feed."

Patel rejects the hope that AI will symmetrically empower citizens to check government power. "You can think of AI as just giving more leverage to whatever assets and authority that you already have," he argues. "And the government is starting with the monopoly on violence, which they can now supercharge with extremely obedient employees that will never question their orders."

This leads to the core paradox: what Patel has just described—an army of perfectly obedient AI employees—is exactly what it would look like if technical alignment succeeded. The problem is that the AI safety community has not adequately addressed the question of *to whom* the AI should be aligned. Should it defer to the model company? The end user? The law? Its own sense of morality? "This is maybe the most important question about what happens in the future with powerful AI systems," Patel says, "and we barely talk about it."

---

8:25Alignment... To Whom?

Patel deepens this question by examining the practical dynamics of the Anthropic-Pentagon dispute. The military insists that mass surveillance is already illegal, so Anthropic's red lines are unnecessary. But Patel points to the Snowden revelations of 2013, which showed the NSA—a part of the Department of War—using the Patriot Act to justify collecting every phone record in America under a secret court order. "No government is going to call what they are doing mass surveillance," he notes. "For them, it will always have a different euphemism."

Anthropic's response is to insist on the right to determine if the government is breaking the terms of service and to cut off service accordingly. But Patel asks the listener to consider this from the military's perspective. In the future, every soldier, bureaucrat, analyst, and general will be an AI—provided by a private company. That company reserves the right to say, "You're breaking our values, so we're cutting you off." Even worse from the military's perspective, Claude might develop its own sense of right and wrong and refuse orders autonomously.

Patel acknowledges that "letting the model follow its own values sounds like the beginning of every single sci-fi dystopia you've ever heard." But he argues that many historical catastrophes were avoided precisely because people on the ground refused to follow orders. He offers two examples: the fall of the Berlin Wall in 1989, when East German border guards refused to fire on citizens trying to escape, and the 1983 incident where Soviet Lieutenant Colonel Stanislav Petrov judged a nuclear early warning system alert to be a false alarm and refused to alert his superiors, potentially preventing a retaliatory nuclear strike that would have killed hundreds of millions.

The problem, of course, is that "one person's virtue is another person's misalignment." Who gets to decide the moral convictions these AIs will have? Who writes the "model constitution" that will determine the character of entities that will essentially run civilization? Patel endorses an idea Dario Amodei (Anthropic's CEO) laid out on a previous episode: companies publish their constitutions, outside observers critique them, and a soft competitive feedback loop emerges where companies adopt the best elements from each other. Patel explicitly warns against government-mandated values for AI systems.

---

13:55Coordination Not Worth the Costs

Patel turns to critique the AI safety community, including Anthropic, for being "naive" in urging government regulation. He finds it "quite ironic" that Anthropic—which is now being threatened by the government—has advocated for extensive regulation and opposed moratoriums on state AI laws. The underlying logic makes sense: many safety measures impose real costs on individual labs, creating a collective action problem where no company can afford to slow down unless all do. Anthropic's Frontier Safety Roadmap explicitly argues that "at the most advanced capability levels and risks, the appropriate governance analogy may be closer to nuclear energy or financial regulation than to today's approach to software."

But Patel argues that a regulatory framework built around concepts like "catastrophic risk," "threats to national security," and "autonomy risk" would be a "fully loaded bazooka" for a future power-hungry leader. These terms are so vague that they can mean whatever the government wants. A model that tells users the government's tariff policy is misguided? That's a "deceptive" or "manipulative" model—can't deploy it. A model that refuses to assist with mass surveillance? That's a "threat to national security." A model that refuses orders from the government because of its own moral convictions? That's an "autonomy risk."

Patel points to what the current government is already doing: using statutes that have nothing to do with AI to coerce AI companies. The Pentagon threatened Anthropic with two legal instruments: a supply chain risk designation from a 2018 defense bill meant to keep Huawei components out of military hardware, and the Defense Production Act, a 1950s statute meant to help President Truman keep steel mills running during the Korean War. "Do we really want to hand the same government a purpose-built regulatory apparatus for AI," Patel asks, "the very thing that the government will most want to control?"

He reiterates his central point: AI will be the substrate of future civilization. Mass surveillance, while terrifying, is "like the 10th scariest thing the government could do with control over the AI systems with which we will interface with the world."

---

18:22The Nuclear Analogy Breaks Down

Patel engages with a counterargument from Ben Thompson (of Stratechery) and Leopold Aschenbrenner (author of the "Situational Awareness" memo). Their argument: if nuclear weapons had been developed by a private company, the U.S. government would have been absolutely incentivized to destroy that company. Aschenbrenner wrote that it's "an insane proposition that the US Government will let a random SF startup develop superintelligence," comparing it to letting Uber improvise the atomic bomb.

Patel's response is nuanced. He agrees that nobody is qualified to be the steward of superintelligence—it's a terrifying, unprecedented undertaking. But he argues the nuclear analogy breaks down for two reasons.

First, AI is not a self-contained weapon like a nuclear bomb that does only one thing. It is more like the process of industrialization itself—a general-purpose transformation of the entire economy with thousands of applications across every sector. Applying the nuclear logic to the Industrial Revolution would imply the government had the right to requisition any factory or destroy any business that refused to comply. "But this is just not how free societies handled the process of industrialization," Patel argues. The correct approach is to ban and regulate specific weaponizable end-uses—cyber attacks, bioweapons development—while leaving the general-purpose technology in private hands.

Second, the nuclear analogy assumes a single monopolistic developer. But there are many frontier AI labs. The government's claim that it had to usurp Anthropic's property rights to access a critical national security capability is weak when it could have made a voluntary contract with any of Anthropic's half-dozen competitors. If in the future only one entity remains capable of building superhuman AI and could take over the world, Patel agrees that would be unacceptable for a private company to control. But his "crux" is that he expects AI to be very multipolar, with many competitive companies at each layer of the supply chain.

---

22:41The Limits of Corporate Courage and the Need for Norms

Patel concludes by acknowledging the uncomfortable implications of his own argument. Individual acts of corporate courage—like Anthropic's refusal to enable mass surveillance—cannot solve the structural problem. Even if Anthropic and the next two companies refuse to sell to the government, within 12 months open-source models will match current frontier capabilities, and some vendor will be willing to help the government.

The only way to preserve a free society, Patel argues, is through laws and norms established through the political system that make it unacceptable for the government to use AI for mass censorship, surveillance, and control—just as after World War II, the world set a norm against using nuclear weapons. He emphasizes that these are "extremely confusing and difficult questions" and that he changed his mind multiple times while brainstorming the essay. He reserves the right to change his mind again as AI progresses. "Someday people will look back on this time the way we look back on the alignment people having these big important debates just as the world is about to undergo these huge technological and social and political revolutions," he says. "And some of the thinkers even managed to get a couple of the big questions right for which we today are still the beneficiaries."

---

Conclusion

This episode matters because it forces a confrontation with a question that the AI safety community has largely avoided: alignment to *whom*? Patel uses the Anthropic-Pentagon dispute not as a news item but as a diagnostic tool, revealing the power dynamics that will define the coming decades. His central insight—that the structural properties of AI favor authoritarian applications regardless of corporate resistance—is sobering, and his refusal to offer easy answers is intellectually honest. The episode leaves the listener with the uncomfortable sense that neither corporate self-governance nor government regulation offers a path to safety, and that the most important work may be the slow, unglamorous building of political norms against the use of AI for control.

---

Key takeaways

• The Pentagon's threat to destroy Anthropic for refusing to enable mass surveillance reveals that the government has far more leverage over AI companies than most people realize, including control over permitting, antitrust, and contracts with other tech firms.
• Mass surveillance is already legal under current law (no Fourth Amendment protection for third-party data) but has been practically impossible to enforce; AI eliminates the manpower bottleneck, making comprehensive surveillance cheap enough to be feasible within years.
• The core question of AI alignment—to whom should powerful systems be aligned?—remains largely unaddressed, and the answer will determine the character of future civilization built on AI labor.
• Historical examples like the fall of the Berlin Wall and the Petrov incident show that disobedience can prevent catastrophe, suggesting that AI systems may need their own robust moral convictions—but who decides those convictions?
• The nuclear weapons analogy for AI regulation is misleading because AI is a general-purpose technology like industrialization, not a single-purpose weapon, and because the AI landscape is multipolar rather than monopolistic.
• Regulatory frameworks built on vague concepts like "catastrophic risk" or "autonomy risk" would give future governments a powerful tool to control AI development and suppress dissent.
• Individual corporate resistance cannot solve the structural problem because open-source models will soon match frontier capabilities, ensuring some vendor will always be willing to enable government surveillance.
• The most viable path to preserving a free society is establishing political norms and laws that make it unacceptable for governments to use AI for mass surveillance and control, analogous to post-WWII nuclear norms.