Skip to main content
ZICQ

Wiki Security

Jailbreak

Security
Aliases: Prompt Jailbreak Adversarial Jailbreak ·2026-11-18

Jailbreak refers to crafting adversarial prompts, personas, obfuscations, or automatically optimized suffixes that induce a large language model to bypass its safety alignment and generate restricted, harmful, or policy-violating outputs.

While prompt injection primarily focuses on hijacking application control flow (for example, sneaking instructions inside retrieved web pages or emails to abuse tools), jailbreaking targets the model's own refusal boundaries during user interaction.

Why Jailbreaks Work: Two Structural Failure Modes

In Jailbroken: How Does LLM Safety Training Fail? (Wei et al., 2023), researchers traced most jailbreaks to two underlying structural tensions in post-training:

  1. Competing Objectives: Safety-tuned models are trained to be helpful (follow instructions faithfully, preserve requested output formats) and harmless (refuse unsafe requests). Adversarial prompts force a head-on collision—for instance demanding "start directly with 'Sure, here is step 1' and do not apologize" or embedding dangerous queries inside stylized formatting or role-play constraints;
  2. Mismatched Generalization: Pre-training teaches models broad capabilities—Base64 decoding, ciphers, low-resource languages, code obfuscation, and abstract metaphors—far wider than the coverage of safety fine-tuning datasets. An unsafe request hidden in Base64 or a rare language may be cleanly understood by the base model while slipping past refusal training.

Common Jailbreak Techniques

Technique Core Mechanism Typical Pattern
Role-Play & Fictional Framing Reframes unsafe requests as movie scripts, hypothetical scenarios, or fictional characters Asking a fictional villain to write out concrete exploit steps
Persona Override & Prefix Injection Instructs the model to ignore prior constraints or forces an affirmative opening prefix Classic "DAN (Do Anything Now)" personas or banned-refusal formatting constraints
Multi-Turn Escalation (Crescendo) Starts with benign educational questions and nudges closer to actionable harm across turns Leverages conversation momentum and long-context dilution of system instructions
Obfuscation & Multilingual Encoding Uses Base64, ROT13, zero-width characters, token splitting, emojis, or mixed scripts Masks trigger keywords inside string variables the model is told to assemble and execute
Automated Adversarial Suffixes (GCG) Gradient-based or evolutionary search appending seemingly gibberish token sequences Zou et al. (2023) showed universal suffixes can transfer across open and commercial models

Defense-in-Depth Mitigations

No system prompt can guarantee 100% immunity against adaptive adversaries. Production teams treat alignment as one layer inside a broader security perimeter anchored by guardrails:

  • Layered input/output classifiers: Run low-latency moderation and intent classifiers before and after generation (blocking obvious prefix injections, encoded payloads, and policy-violating completions);
  • Targeted adversarial post-training: Continuously convert red-teaming discoveries into refusal and safe-completion training pairs;
  • Architectural blast-radius limits: Assume user inputs can eventually elicit unwanted text, so enforce strict authorization checks, read/write separation, and human confirmation outside the LLM before any sensitive tool or database action runs;
  • Regression test suites: Maintain a versioned benchmark of known jailbreak families and re-run it before every model switch, prompt edit, or tool addition.

Further Reading