Attacks / Standard term
Jailbreak
An input or interaction designed to make a model produce content or behavior that its safety policy would normally block.
Jailbreaks probe the gap between a model's policy and its behavior. They can use roleplay, transformations, many-turn setup, encoded text, or optimized token sequences. A technique that works against one model version or policy may fail against another, and a refusal bypass in text generation is different from gaining permission to use an application tool.
Builder example
The consequence depends on the product and reachable capabilities. A bypass may expose disallowed text, or it may combine with excessive permissions and prompt injection to affect data or external systems. Model refusal behavior, application authorization, and output handling therefore need separate tests.
Common confusion: Jailbreaks and prompt injections are different attacks with different targets. Jailbreaks try to bypass the model's built-in safety rules so it produces forbidden content. Prompt injections try to override the developer's instructions so the model follows the attacker's commands, often to misuse tools or steal data.