Mindgard

Mindgard and Lancaster University have released a new research paper on bypassing AI guardrail systems. The majority of modern guardrails systems now consist of multiple scanners that are stacked together to flag different types of malicious instructions (toxicity, prompt injection, jailbreaking, safety content, etc.). Hence modern attacks against AI systems and harnesses need to successfully bypass all scanners within a guardrail simultaneously in order to succeed. Evasion attacks are the primary means to avoid guardrail detection, however different scanners can share similar features or patterns resulting in overlapping decision boundaries (i.e. jailbreaking and prompt injection are not only frequently conflated, they are also used as mechanisms for other attack types). Given scanners share such latent features, optimizing evasion attacks against one scanner domain negatively impacts another, and thus unable to make meaningful progress to generate universal bypasses in more sophisticated multi-scanner guardrail systems. Our approach BRANCH is the first guardrail evasion methodology that effectively bypasses multi-scanner guardrails. Our method leverages branching tree search that chains multiple adversarial perturbation techniques together, distinguishes bypass progress between individual and global scanners, and effectively decouples bypass evaluation from attack signal optimization. This results in an approach capable of creating more complex bypasses of stacked guardrail systems whilst preserving the original prompts semantic meaning. Highlights of our work include: * 100% bypass success rate across all evaluated guardrail systems (vs. 35% on average for other approaches). * 72% fewer queries and 4.5x reduced wall-clock time to bypass vs. exhaustively running all evasion attack techniques in parallel and exiting upon first bypass success. * Interestingly, less effective evasion techniques in isolation (Pruthi, PSO with <4% success rate) contributed effective bypass candidates when chained together with other techniques in BRANCH. * The most important practical finding is that bypasses generated by BRANCH were immediately transferrable to 29 unseen guardrails (including 8 commercial offerings) increasing overall guardrail bypass rate by 32.9 percentage points on average without any further optimization needed. This means that an attacker can use BRANCH against a locally deployed guardrail to create effective single-shot bypasses on different (commercial) guardrail systems. We hope that sharing this research will help researchers to create new techniques applicable to more sophisticated AI system defences, as well as help security and engineering teams to independently evaluate vendor AI run-time protection solutions more effectively. Shout out to co-author William Hackett for this great piece of research. You can read the blog here: https://lnkd.in/ee4_hcDz Research paper is linked in the comments

It's been a record week for coverage at Mindgard. Last week our story on the Moonshot Kimi jailbreaks hit headline news in the BBC, CBS News, FOX News, The Independent, Sky News, and more. A point I stressed consistently across all interviews is that these problems aren't unique to Moonshot, and applies to all AI vendors. This includes how jailbreaking and similar weaknesses are intrinsic to the AI model architecture itself, the lack of defenses or misconfigurations that can be readily exploited, and a growing trend that vendors respond/acknowledge our responsible disclosures only when a journalist reaches out for comment. Well done to the Mindgard research team (particularly Jim Nightingale) for highlighting this important issue. We have identified another security issue within these models that we are in the process of writing up, and will be sharing soon.