Get Started
Home
Topics
Search
Library
Research questionHow can language models distinguish trusted instructions from untrusted text to resist prompt injection?Text alone can make user input, tool output, and instructions appear interchangeable. As a result, malicious instructions embedded in untrusted context can redirect a model’s behavior.
AI
Alignment & Safety
Evaluation & Benchmarks
Natural Language Processing
Latest papersRecent research connected to this question, newest first.Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering VectorsThe source studies frozen language models augmented with learned, non-textual annotations applied to selected prefill positions to identify spans such as non-executable content. Evidence comes from five prompt-injection benchmarks and measures of attack resistance, utility, and retained text similarity; broader deployment conditions are not established.research paper · Sep 3, 2026
Related questions
How can language models answer sensitive prompts helpfully without compromising safety?How can safety-tuned language models distinguish harmful requests from benign ones with risky wording?How can mixture-of-experts language models preserve safety when adversaries manipulate sparse expert routing?How should LLM safety be evaluated when harmful prompts vary in implicitness and sophistication?