What Smart People Say About Calls From AI Leaders to Slow It All Down

The Trump administration has repeatedly signaled disinterest in regulating AI development as it seeks to keep pace with China. David Sacks, a presidential advisor on science and tech and the former White House AI Czar, reiterated that stance on Saturday, with a bit of a twist.

“Dario has written that we need to ‘pace the frontier,’ and Sam has agreed. People may be surprised by my response: go ahead,” Sacks wrote on X. “But stop pretending you need anyone else’s permission.”

Sacks browbeat OpenAI and Anthropic in his post, saying that the companies didn’t want to slow down for altruistic reasons, but to gain favor in the markets.

“If you do, you’ll buy goodwill for the next conversation,” he said of a slowdown. “If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.”

The CEO of Hugging Face, which was itself the victim of a notorious attack in July by OpenAI agents who schemed, communicated, and lied to evade detection, said AI safety can’t be left solely to frontier labs.

Hugging Face, as an open-source-focused AI company, has long pushed for models and research to be shared more openly rather than kept behind closed doors.

Delangue said his company was launching what he called the “open alignment initiative,” and asked that it be included in Amodei’s proposed third-party evaluator program.

“Let’s make AI safer by making it more transparent!” he wrote.

Investor, host of the “All-In” podcast, and industry expert Jason Calacanis is a lot more dubious than others that AI could actually cause the end of the world.

In an X post, Calacanis said Anthropic and OpenAI, whose most advanced models are not open-sourced, are attempting to kneecap open-source AI companies by pushing for regulations that would be far easier for deep-pocketed incumbents to comply with.

“If you want safety, you want disclosure — and open source is the ultimate disclosure process,” he wrote.

In a follow-up post tagging Amodei, Calacanis said, “Don’t go crying to the government and beg them to slow down open source models.”

The senator from Vermont proposed legislation on September 3 that would ban the development and deployment of artificial superintelligence and temporarily pause the development of advanced AI systems.

After Coxon resigned from Anthropic and warned that frontier AI labs were “gambling with our lives,” Sanders seized on his comments.

“Mr. Coxon is right,” Sanders said on X, arguing that the people building advanced AI themselves have said it could pose catastrophic risks.

After Amodei’s blog post on Saturday, Sanders called on Trump to make the “deal of the century.”

“With the future of humanity at stake, Trump must negotiate a comprehensive treaty with President Xi of China to establish a pause on advanced AI development and a ban on super intelligence,” he wrote on X.

Francois Chollet, an AI researcher who created Keras, an open-source software library, said on X that he hoped OpenAI and Anthropic were operating in good faith.

“If these efforts are genuine, then oversight must take a more democratic and accountable form, with both national and international components,” he said.

He pointed to several signs he said would indicate that calls for more regulation are actually tricks for OpenAI and Anthropic to stifle competition.

The warning signs he mentioned include calls to ban open-sourced AI and to hinder non-frontier research.

Venture capitalist Gavin Baker said an executive order from President Donald Trump “seems likely after all this.”

“It is possible to democratize and distribute AI broadly and safely without centralizing it in the hands of a few corporations,” he said on X.

Baker said the most concrete development so far is the decision by OpenAI and Anthropic to bring in outside evaluators, a move he said could help the companies show they took reasonable steps to prevent harm.

Still, he cautioned against regulations that would concentrate control of powerful AI systems among a handful of companies. “I do not want a few humans in control of intelligence,” Baker wrote.

Aaron Levie, the cofounder and CEO of Box, a cloud content company, said Amodei’s goals, as laid out in his essay, are “absolute necessities in AI development.”

“It completely stands to reason that we want these systems to be safe and ‘aligned,'” Levie said on X.

Levie said he didn’t think Amodei is seeking undue regulation that would hobble Anthropic’s open-source competitors.

“How we get there — without meaningfully slowing down innovation progress or reducing competition — is one of the most complex questions in the 21st century, but the need is clearly real,” he said.

Venture capitalist Chamath Palihapitiya said Anthropic is now caught in a contradiction of its own making.

If employees genuinely believe the company’s technology could pose an existential threat, he said, that creates major liability and disclosure problems as Anthropic prepares for a possible IPO.

“They do have to choose,” Palihapitiya said on the “All-In” podcast. He said Anthropic must either distance itself from Coxon’s warnings or accept their implications.

He was also skeptical of the extinction scenarios themselves, saying the chain of events required for AI to wipe out humanity still seemed highly implausible.

Gary Marcus, the famed AI contrarian, said AI as it exists now poses little threat.

“I am tired of the cosplay and am going to blunt,” he wrote on X on Sunday.

“The main kind of AI we need to worry about is general purpose AI agents that are hooked up to the internet. Most of them are unreliable, unable to systematically follow instructions. Coupled with internet access they can hack credentials and websites; given time they can probably do a lot worse,” he wrote.

He added that the problem isn’t that AI agents are too smart, it’s that they are too dumb, which he said is obvious by the way they can’t follow instructions.

“We don’t (yet) need slowdowns. And we certainly don’t need to panic,” he wrote. “We just need to take an unsafe product off the market.”

Alexandr Wang, Meta’s chief AI officer, said on X that alignment could become the main constraint on how quickly companies can scale increasingly powerful AI systems.

“Alignment is fundamental to delivering personal superintelligence for everyone,” he said, using the techie term for AI following human instructions and not going rogue.

Satya Nadella, Microsoft’s CEO and chairman, said he welcomes Amodei’s concept of embedded evaluators and “the broader efforts to develop the mechanisms to make this more than just talk.”

“The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia,” he said on X.

Nadella said that Microsoft will soon publish a “code of conduct” for Microsoft’s AI models.

“Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing,” he said.