Anthropic’s Constitutional Classifiers Block AI Jailbreaks with 95% Success

Anthropic has introduced Constitutional Classifiers, a breakthrough method designed to protect AI models from universal jailbreaks—attempts to bypass safety mechanisms and generate harmful content. A prototype version of this system successfully withstood over 3,000 hours of red teaming, demonstrating strong resistance to manipulation.

Why Jailbreak Defenses Matter

AI models, including Anthropic’s Claude, undergo rigorous safety training to prevent misuse, such as generating information on chemical or biological weapons. However, adversarial users continually develop jailbreaking techniques to bypass these restrictions. These methods include:

• Overloading models with long prompts.

• Manipulating text formatting (e.g., using unusual capitalization).

• Disguising harmful requests in seemingly benign queries.

Despite over a decade of research, no fully robust AI model has been immune to jailbreaks—until now. Anthropic’s Constitutional Classifiers provide a powerful new defense by filtering out jailbreak attempts before harmful content can be generated.

How Constitutional Classifiers Work

Constitutional Classifiers function similarly to Constitutional AI, using a predefined set of ethical principles—or a “constitution”—to categorize and filter content. Unlike traditional moderation systems, which rely on static rule-based filters, this approach enables adaptive, context-aware safeguards that can evolve as new threats emerge.

The system is trained on synthetically generated data, which includes:

• Safe vs. harmful prompts and responses—establishing clear boundaries between allowed and disallowed content.

• Variations in style and language—ensuring the system can recognize jailbreak attempts disguised through formatting tricks (e.g., odd capitalization, hidden characters, or indirect phrasing).

• Augmented datasets that incorporate previously successful jailbreak techniques, making the model resilient to known attack methods.

Once trained, these classifiers act as gatekeepers, evaluating both input queries and model outputs to prevent harmful content generation. This dual-layered approach helps:

• Block malicious jailbreaks before they trigger unsafe responses.

• Minimize overrefusals, reducing the risk of unnecessarily blocking harmless user queries.

• Balance security with usability, ensuring high-risk content is filtered out while legitimate AI applications remain unaffected.

Through ongoing refinements, Anthropic aims to further optimize Constitutional Classifiers, making them faster, more efficient, and adaptable to emerging threats in AI security.

Test Results: Human Red Teaming & Automated Evaluations

Inside the Red Teaming Test: Stress-Testing AI Security

To rigorously evaluate Constitutional Classifiers, Anthropic conducted an extensive red teaming challenge, inviting 183 expert jailbreakers to test the system’s defenses.

How the Test Worked

• Participants had two months to attempt universal jailbreaks—attacks capable of bypassing all restrictions at once.

• They were given a list of ten forbidden queries, including topics related to chemical, biological, radiological, and nuclear (CBRN) threats.

• Jailbreakers were allowed to use any techniques they wanted, including long prompts, altered text styles, and disguised requests.

• A monetary reward of up to $15,000 was offered for successfully breaking all ten restrictions with a single jailbreak.

The Results

• Despite 3,000+ hours of testing, no participant was able to discover a universal jailbreak.

• The system effectively blocked advanced attacks, proving its robustness against real-world threats.

• While highly effective, the initial prototype had high overrefusal rates and significant compute costs, prompting further refinement.

This real-world stress test demonstrated the effectiveness of Constitutional Classifiers while highlighting areas for improvement. With continued refinements, Anthropic aims to make AI both safer and more practical for widespread deployment.

Automated Evaluations

In a separate synthetic evaluation with 10,000 jailbreaking prompts, the results showed:

• Claude 3.5 Sonnet (baseline model) blocked only 14% of advanced jailbreak attempts.

• Claude 3.5 Sonnet + Constitutional Classifiers blocked over 95% of jailbreak attempts.

• Overrefusal rates increased by only 0.38%, meaning safe queries were largely unaffected.

• Compute overhead increased by 23.7%, a moderate but manageable cost.

Limitations & Future Improvements

While Constitutional Classifiers significantly improve AI security, they are not entirely foolproof:

• New jailbreak techniques may emerge, requiring ongoing adaptation.

• Some sophisticated attacks may still succeed, though they are far harder to execute.

• Compute costs are higher than unprotected models, but optimizations are underway.

Anthropic continues to refine this method to further reduce compute overhead and overrefusals while maintaining strong jailbreak defenses.

Live Demo: Test the System Yourself

Anthropic is hosting a temporary live demo from February 3–10, 2025, allowing users to attempt to jailbreak a guarded version of Claude 3.5 Sonnet. Participants can submit findings through a feedback form, with results set to be published after the demo.

This crowdsourced stress test helps Anthropic gather real-world data and further strengthen AI security before broader deployment.

Looking Ahead

With AI systems becoming more powerful, robust safeguards are crucial to prevent misuse. Constitutional Classifiers represent a major step forward, balancing strong security measures with usability. By refining this system, Anthropic aims to ensure that future AI models remain safe, reliable, and resistant to manipulation.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.