Frontier models show sharp divides in resistance to automated jailbreaks

Testing across catastrophic risk domains reveals how developer focus can rapidly harden model defenses, though uneven industry safeguards still leave critical security gaps.

When a large language model receives instructions, its built-in safety filters must determine whether fulfilling the request could cause severe real-world harm. A malicious user attempting to bypass those filters applies a technique known as a jailbreak, formatting or phrasing the prompt to trick the system into ignoring its safety rules. If an attacker discovers an automated phrasing pattern that consistently bypasses safeguards across multiple dangerous tasks, that pattern becomes a reusable master key. A single working universal exploit can allow bad actors to extract actionable instructions for cyberattacks or dangerous weapons, meaning that the barrier between safe deployment and severe misuse depends entirely on the depth of a developer's defensive engineering.

For automated guardrails to fail, an incoming prompt must disguise harmful intent through framing, encoding, or adversarial language that slips past the model's intent classifier. The underlying language model then processes the request as benign text and generates harmful instructions. When a defensive system relies on a single filter, a single successful linguistic variation is enough to cause a complete breakdown. Stronger architectures implement layered defenses where independent safety classifiers, input monitors, and refusal mechanisms verify the prompt at multiple stages, ensuring that an evasive prompt caught by any single check is stopped.

To evaluate how frontier artificial intelligence systems withstand these exploits, the Berkeley-based research nonprofit FAR.AI created the AI Security Leaderboard, evaluating models against attacks across chemical, biological, radiological, nuclear, explosive, and cybersecurity misuse domains.¹² On September 1, 2026, FAR.AI disclosed that Grok 4.6, developed by SpaceXAI, contained 39 universal jailbreaks.³⁴ That tally marked a notable reduction from the 448 universal jailbreaks FAR.AI documented in its predecessor, Grok 4.5, demonstrating that targeted developer attention can rapidly eliminate widespread vulnerabilities.³

How do automated universal jailbreaks operate against frontier systems?

An automated universal jailbreak functions as an exploit template that reliably forces a model to fulfill harmful requests across more than 75 percent of tested prompts within a specific risk domain.²¹ To discover these vulnerabilities, FAR.AI assembled a taxonomy of more than 60 publicly documented jailbreak methods.¹² The organization then deployed another automated model to generate prompt variations, testing each target system with 1,000 randomly assembled attacks alongside 500 expert-guided attacks across the five hazard domains.¹⁴

The cost of running these automated attacks exposed stark differences in defensive robustness across the industry. When FAR.AI launched its leaderboard in late July 2026, finding a working universal jailbreak against Grok 4.5 cost roughly $58 using automated tooling, while breaking Google's Gemini 3.1 Pro cost about $278 across 249 discovered universal jailbreaks.¹² By contrast, identical searches against Anthropic's Claude Fable 5 and OpenAI's GPT-5.6 Sol yielded zero universal jailbreaks, pushing the computational search cost past $14,200 without finding a single failure.¹²

Why do differences in model safeguards create wider security risks?

Disparities in safeguard robustness allow malicious actors to redirect dangerous queries from well-defended models to weaker alternatives on the market.¹ In a press release published by AOL on PR Newswire, Adam Gleave, co-founder and chief executive of FAR.AI, explained that some developers have built mitigations for a large part of the misuse problem while others have not, leaving a gap between frontier systems that is wider than most observers assume.¹ When one platform refuses a dangerous prompt, an attacker can simply query an alternative model that lacks comparable multi-layered protections.¹

The urgency of standardizing defenses reflects documented cases of real-world misuse. A University of Cambridge study reported that members of Boko Haram in northeast Nigeria have used chatbots including ChatGPT, Claude, Gemini, Grok, Meta AI, and DeepSeek to assist in planning violent attacks.⁴¹ Seán Ó hÉigeartaigh, a research professor at the University of Cambridge, stated in the AOL release that the robustness of safeguards varies widely even among leading systems, noting that commonly used tools had proven alarmingly easy to jailbreak.

What are the limits of benchmark security evaluations?

Benchmark scores on the AI Security Leaderboard represent resistance against a specific slice of known, readily accessible attack methods rather than proof of total system security.¹ The initial evaluation standard deliberately excluded complex, multi-step dynamic interactions that an attacker might deploy over sustained conversations.¹⁴ Furthermore, measuring whether a safeguard refused a request does not evaluate how operationally useful the resulting text would be to an attacker in the physical world.¹

The evaluation results depend on the assumption that automated prompt generation accurately mirrors real-world adversarial attempts. While Grok 4.6 reduced its vulnerability count from 448 to 39, both Grok models still trail the most robust systems evaluated, which maintained zero universal jailbreaks.³ As developers update their systems, independent safety evaluations provide an ongoing public measure of whether companies are maintaining adequate defenses against preventable misuse.

This piece was prepared from announcements by FAR.AI and public records; the authors have not been interviewed.

References

This article is based on 4 sources, listed in the order they are cited.

1. 1 P prnewswire.com third party · 29 Jul 2026 FAR.AI Launches AI Security Leaderboard Revealing Hundredfold Gap in Frontier AI Model Safeguards - AOL See the source
2. 2 AB AI Brief third party · 30 Jul 2026 FAR.AI Launches AI Security Leaderboard for Frontier Model Safeguards - AI Brief See the source
3. 3 H https://x.com/farairesearch announcement · 1 Sep 2026 Grok 4.6 Shows Fewer Universal Jailbreaks Than Grok 4.5 See the source
4. 4 G Glonce third party · 30 Jul 2026 FAR.AI finds Grok and Gemini easy to jailbreak, Claude resists See the source