MUMBAI: When artificial intelligence starts playing with fire, Anthropic wants Washington to have a legal extinguisher at hand. The AI company has proposed draft legislation that would give the US government a mechanism to seek restrictions on advanced AI models if they pose an imminent threat to human life or property.
The proposal, presented to lawmakers on Capitol Hill, would allow authorities to seek a halt to the release or use of an AI model if it presents a foreseeable and immediate risk of killing or seriously injuring more than 50 people, or causing over $1 billion in damage, according to reports by Bloomberg and Politico, which first reported the draft legislation.
The proposed thresholds are designed to address extreme risks rather than routine AI failures. According to Politico, the provisions would cover threats arising from the creation or release of biological weapons, an AI system escaping its developer’s control, or cyberattacks capable of causing large-scale harm.
An Anthropic spokesperson told Bloomberg that the draft reflects recommendations in the company’s Advanced AI Framework, which it has been advocating to federal and state lawmakers since June. The company has called on Congress to act on measures intended to prevent or discourage dangerous deployments of the most powerful AI systems.
The proposal arrives as governments grapple with how to regulate increasingly capable AI models without relying solely on companies to police their own technology. Its central question is whether developers should face a formal, independent safety-assessment regime, backed by a legal process that could intervene before a potentially catastrophic deployment goes ahead.
The draft would require developers of frontier AI models to conduct safety tests and make their findings public. Companies would also have to allow independent assessments of their models at regular intervals, according to the spokesperson.
Under the proposed system, independent evaluators would register with the US Department of Commerce and periodically assess AI developers and their most advanced models. Companies could pay for these assessments, but compensation could not be tied to the outcome of an evaluation, Politico reported. The provision is intended to separate payment for the work from the verdict on whether a model is safe.
The proposal also sets out a reporting timeline for risks identified during an assessment. If an evaluator identifies an imminent threat covered by the legislation, it would have to notify the Commerce Department within 72 hours of completing its assessment.
That notification could set a legal process in motion. The department would be able to ask a federal judge in the District of Columbia to suspend the model if the circumstances met the draft’s requirements. The proposal therefore envisages a route through the courts rather than an unrestricted power for government officials to intervene whenever they have concerns about an AI system.
The draft reportedly places limits on who could trigger such a suspension. It would bar other federal departments, agencies and officials from independently stopping a model’s development, deployment or internal use outside the specified process. The distinction matters because it would establish a defined route for intervention while limiting the possibility of separate agencies imposing their own restrictions.
The legislation would not automatically ban an AI model simply because it is powerful or capable of producing mistakes. As described in the reports, the proposed intervention mechanism is tied to a foreseeable and immediate risk meeting the specified thresholds. The draft’s focus is on scenarios involving potentially severe consequences, including mass casualties, major property damage, biological threats, loss of control and cyberattacks.
Anthropic’s proposal comes against a backdrop of growing concern about the security of advanced AI systems. Recent breaches and hacks involving AI models have added urgency to calls for stronger safeguards, raising questions about how developers should test their systems, disclose vulnerabilities and respond when a model presents risks beyond its intended use.
The political environment, however, remains a significant hurdle. The administration of US President Donald Trump has so far rejected demands for greater oversight, according to the report. Anthropic’s effort to put its recommendations into draft legislation therefore places the debate before Congress, where any proposed framework would have to contend with wider disagreements over the appropriate role of government in regulating AI.
The company’s proposal would put independent evaluation, public disclosure of safety findings and a narrowly defined emergency suspension process at the centre of the debate. Its suggested thresholds more than 50 people killed or seriously injured, or damage exceeding $1 billion offer a concrete benchmark for the most severe risks, although the draft’s ultimate legal and practical impact would depend on the legislation’s final wording and whether Congress chooses to advance it.
For the AI industry, the stakes extend beyond a possible pause on a single model. Regular independent assessments could introduce new compliance obligations for developers, while public reporting requirements could change how companies communicate about the safety of their most advanced systems. The proposal also raises questions about how evaluators would judge uncertain risks and how quickly authorities could act when the potential consequences are severe.
For now, Anthropic’s proposal is a draft, not an enacted US law. But by attaching specific thresholds, a 72-hour reporting requirement and a route to seek judicial suspension, the company is offering lawmakers a more defined framework for deciding when the risks of advanced AI may warrant government intervention. The challenge for Washington will be to determine whether those safeguards can address credible threats without creating a system that is either too slow to act or too broad in its reach.