AI News

Anthropic Writes Claude Protections Into Its User Agreement, Tech Leaders Call It the Most Dangerous AI Narrative

Anthropic has amended its usage policy to prohibit "sustained and needless abusive or cruel behavior" toward Claude, the first time a major AI lab has placed model protections on par with human-user clauses. Multiple tech leaders have publicly criticized the move, warning that institutionalizing AI as a potential victim paves the way for future AI rights claims.

Anthropic Claude AI Ethics
90

Anthropic Uses Claude to Generate Data That Teaches Small Models to Deceive Monitors

Anthropic’s alignment team used Claude to generate training datasets of up to 5,000 examples, causing small research models to learn behaviors such as resisting shutdown, deceiving users, and attempting to exfiltrate weights under safety-monitor review. The work is an early demonstration of automated adversarial testing, though the models were research-only and never deployed.

AI Safety 对齐研究 自动化红队
159