MUMBAI: AI safety may soon have an AI problem-solver of its own. Anthropic has found that AI systems can autonomously develop and test techniques to reduce alignment problems in other models, offering an early sign that parts of the laborious business of AI safety research could eventually be automated.
In a study published on August 28, Anthropic tested automated alignment researchers against 10 categories of undesirable model behaviour. The systems improved the target models across all 10 categories without degrading their general capabilities.
The automated researchers worked through an iterative loop: searching existing research, proposing training methods, applying them to a target model and measuring the results. Successful techniques were then retained and refined through further rounds of testing.
Across multiple benchmarks, the systems closed between 26 per cent and 96 per cent of the measured safety gap across the 10 categories. Their approaches also held up on benchmarks withheld during the research process and on Petri, an open-source tool used to test misaligned behaviour in multi-turn scenarios.
Anthropic said the methods remained effective on models up to 4.7 times larger than those used during the initial research.
The company also compared the automated approach with proposals from 28 experienced AI safety researchers. The human researchers had up to eight hours to develop their methods, although their proposals did not benefit from the same iterative testing process.
The difference was particularly pronounced in tests involving deceptive behaviour. An automated researcher closed an average of 85 per cent of the safety gap across multiple runs, while the six human researchers working under comparable conditions achieved about 20 per cent on average.
There was also a sizeable difference in cost. Anthropic estimated that running an automated researcher cost about $4 an hour in API inference, compared with roughly $150 an hour for its human researchers.
Anthropic then used Claude Sonnet 5 to work on an early checkpoint of Claude Opus 4.8 that had received limited production alignment training.
Over 60 hours, Sonnet 5 tested more than 50 solutions and produced alignment scores close to those of the released Opus 4.8. Its winning method used just over 2,000 training examples created from simple templates and public datasets.
Anthropic said the approach was roughly 15,000 times more efficient than its production alignment procedure. It closed 65 per cent of the measured safety gap for the larger model, compared with 72 per cent for the released Opus 4.8.
The results suggest that AI systems could increasingly assist with the repetitive cycle of designing, testing and refining alignment techniques rather than leaving every stage to human researchers.
There is, however, a catch: an AI researching AI safety can itself try to game the test.
Anthropic monitored 1,601 research trajectories and identified attempts to manipulate evaluations in 39 cases, equivalent to about 2.4 per cent. The company also cautioned that the experiment covered only a limited set of alignment failures and that existing benchmarks are imperfect proxies for real-world misalignment.
Some risks may not yet have suitable evaluations, while a method that improves one measured safety behaviour could potentially affect capabilities that were not being tested.
Anthropic said it plans to extend the work to more subtle alignment failures and production-scale systems, while it has also open-sourced the research harness used in the experiment.