In brief
- OpenAI has asked lawmakers whether rival AI developers could legally coordinate a slowdown, according to WIRED.
- Experts say funding and competitive pressures discourage companies from slowing development independently.
- A bill in Congress would protect certain agreements to restrict AI development to address security risks.
OpenAI has asked members of Congress whether rival AI companies could legally agree to slow development, as its chief scientist urges the industry to establish shared safety standards.
According to WIRED, which cited people close to the company, OpenAI asked lawmakers in recent weeks whether an agreement among AI companies to slow development could violate antitrust laws.
Myriad: SPY high in September？ Click to make your prediction.
The outreach to lawmakers follows OpenAI chief scientist Jakub Pachockis call for voluntary slowdowns until developers can better demonstrate their systems are safe. But companies face pressure to keep advancing even when their own researchers have concerns.
“Commercial and geopolitical competition in the AI space is incredibly intense, leading to a concerning dynamic where companies are incentivized to release products before their risks are fully understood,” Miranda Bogen, chief technologist at the Center for Democracy and Technology, told Decrypt.
“Even when internal staff knows more research and testing is needed, their companies are facing immense pressure to cut corners and skip critical safety tests, despite evidence piling up about the consequences of moving too fast,” she added.
An agreement to slow AI development could raise antitrust concerns by restricting competition. Sens. Adam Schiff and Jim Banks introduced legislation to protect certain security collaborations, subject to advance notice to the Justice Department.
The debate over slowing AI development comes as the United States and China compete for leadership in the technology. U.S. President Donald Trump delayed an AI executive order in May over concerns it could weaken Americas lead over China. Trump later signed the order in June, creating a voluntary review process for advanced models before release.
More direct than national security concerns, financial incentives could make agreement difficult.
“Every advance under current conditions yields many millions or billions more in funding and puts the creators of that advance in a greater position of power and influence,” Duncan Sabien, head of communications at the Machine Intelligence Research Institute, told Decrypt.
Calling the arms-race dynamic real, Sabien said OpenAI and Anthropic face pressure to keep advancing despite uncertain risks because each new system helps build more capable successors, allowing a rival to pull ahead if either company slows down without a shared agreement.
“This is especially true since gains in intelligence are compounding; each new system makes it easier to train and deploy the next, more-advanced system,” he said. Sans some sort of coordination mechanism, stepping back just means the other guy gets a lead.
In August, OpenAI paused internal Astra work lacking stronger safeguards over cybersecurity concerns. However, both OpenAI and Anthropic relaxed their safety commitments in February, with Anthropic chief science officer Jared Kaplan arguing that slowing development independently made less sense while competitors continued advancing.
The debate over slowing AI development surged this week after former Anthropic engineer Jacob Coxon publicly announced his resignation, citing concerns about the risks continued AI development poses to humanity.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon said on X, saying it was not a “marketing stunt.”
“If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible—but I hear the same people express fear privately. No other human activity poses this level of danger,” he wrote.
However, Sabien said researchers may accomplish little by leaving if someone else simply takes their place.
“If enough of them achieve common knowledge that they should all stop, then they can break through the coordination barrier and stop together rather than just being replaced by the next slightly-less-cautious person,” he said.
Daily Debrief Newsletter
Start every day with the top news stories right now, plus original features, a podcast, videos and more.
Your Email
Get it!
Get it!