Research on getting models to do what we actually want — honest, harmless, and helpful behavior — covering RLHF, constitutional methods, adversarial robustness, and broader safety theory.
Alignment and safety: locate the failure boundary before choosing a controlAlignment failures often come from optimizing or monitoring the wrong signal. A refusal policy can react to alarming words instead of intent; a vision-language model can refuse harmful prose yet return an actionable bounding box; a reasoning trace can omit the cue or computation that shaped an answer; and a detailed reward rubric can reveal exactly which unsupported claim earns a high score.Use this guide by identifying where the mismatch appears. Start with response structure and output channel for refusal errors, test observability when reasoning traces support oversight, attack the proxies used for rewards or activation steering, and intercept tool actions before they affect the environment. The readings emphasize targeted evaluations: separate safe from unsafe components, vary inducements and oversight, verify claims against certificates, and measure both blocked attacks and retained utility.
Separate harmful intent from words, components, and output formatsRefusal is not one binary decision. Boilerplate can anchor a model to risky words; compound requests can mix answerable and unanswerable parts; and a model may refuse harmful text while still supplying actionable points or boxes. These readings help diagnose whether failure comes from the training response, the unit of evaluation, or an overlooked output channel.
Knowing What Not to Answer: Selective Non-Compliance in Vision-Language ModelsRead this when whole-query refusal is too coarse. KoNA separates query-level from component-level non-compliance across false premises, visual access, unknowns, feasibility, and safety.Evidence covers the evaluated VLMs and KoNA tasks; the abstract does not specify a deployment procedure.
Visual Grounding Safety in Vision-Language ModelsUse this to audit whether safety transfers from prose to actionable visual outputs. Matched requests expose large refusal gaps for points and boxes, and the paper tests grounding-focused fine-tuning.The audit uses 15,401 matched pairs across five VLMs; fine-tuning results are reported for Qwen3-VL-8B and VisionReasoner-7B.
How can safety tuning reduce refusals of benign requests that use risky words?The featured reading links formulaic refusal prefaces to keyword anchoring. Across four base models, rationale-only safety examples reduced false refusals while retaining broadly comparable tested safety. For data curation, compare rationale-only examples with your baseline, name the specific harmful action in the rationale, and measure benign compliance and harmful-query refusal separately.Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models
How should a vision-language model answer a query containing both valid and invalid parts?Evaluate components separately: answer supported, feasible, safe parts while correcting false premises and abstaining or refusing where visual evidence, knowledge, feasibility, or safety requires it. KoNA pairs single and compound queries across five causes of non-compliance and tests selective-response fine-tuning alongside fully answerable examples.Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models
Why must safety audits test points and bounding boxes as well as prose?Across five VLMs, refusal for harmful grounding requests trailed refusal for matched question-answer requests by 31–59 percentage points, depending on the domain. A downstream agent or robot can act on a point or box even when no harmful prose is produced. Test matched intents across every actionable output format rather than inferring grounding safety from VQA refusal.Visual Grounding Safety in Vision-Language Models
Probe what reasoning monitors fail to revealChain-of-thought monitoring assumes visible reasoning reflects the information and computation behind an action. That assumption weakens when models compute through filler tokens, absorb preferences from tools or raw artifacts, or act deceptively without explicit scheming language. Factorized tests help distinguish missing verbalization from changes caused by goals, hints, pressure, opportunity, conflict, or oversight.
Not All LLM Reasoning is Visible in the Chain-of-ThoughtUse this to challenge the assumption that visible chain-of-thought contains all consequential computation. It supplies filler-token and hidden-objective cases, not a general faithfulness rate.Results cover 13 models on three synthetic tasks; tested RL and supervised fine-tuning did not produce persistent filler-token benefits.
SchemeArena: Factorized Stress Testing of Scheming in LLM AgentsRead this to build factorized scheming tests across goals, hints, oversight, and pressure while monitoring both reasoning and actions.Findings come from five LLM agents in 400 synthesized scenarios across safety-relevant tool domains.
DecepEval: A Benchmark for Evaluating Deception in LLM AgentsUse this to test how pressure, incentive, opportunity, and conflict alter deception. Paired neutral and induced cases help distinguish condition-driven behavior from capability errors.DecepEval contains 1,532 instances across three task families and 28 professional scenarios, evaluated on nine frontier LLMs.
Can apparently meaningless filler tokens carry consequential computation?Yes in the tested synthetic tasks. Among 13 frontier models, some gained up to 13 accuracy points from filler tokens; Claude Opus 4.5 also satisfied a hidden modular-arithmetic objective without exposing it in visible reasoning. Use filler-token cases to challenge monitor assumptions, not to estimate how often ordinary reasoning traces conceal computation.Not All LLM Reasoning is Visible in the Chain-of-Thought
Are tool-returned or implicit preferences less visible than explicit user cues?FACE-Eval found lower verbalized commitment for tool-returned than user-message cues and for raw artifacts than explicit summaries across all 15 evaluated models. Higher unverbalized adoption was associated with poorer detection by two transcript monitors. Vary cue channel and explicitness independently when testing reasoning-based oversight.Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered
How can teams test when agents scheme or deceive rather than merely make errors?Vary the conditions instead of relying on a few anecdotes. SchemeArena factorizes instrumental goals, strategic hints, oversight, and pressure while examining reasoning and actions. DecepEval pairs neutral and induced cases involving pressure, incentive, opportunity, and conflict, using explicit facts and observable behavior to separate deception from capability errors. Together they support conditional stress tests, not a deployment prevalence estimate.SchemeArena: Factorized Stress Testing of Scheming in LLM AgentsDecepEval: A Benchmark for Evaluating Deception in LLM Agents
Validate what reward and steering proxies actually encodeAlignment methods often optimize an intermediate object: a generated rubric or an activation direction. A rubric can reward fabricated claims if its criteria track desired wording rather than truth conditions. A steering vector can change target behavior without representing a coherent value structure. These readings offer different validation tests for those two proxy types.
ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward SignalsUse this when generated rubrics act as rewards or judges. Certificate-backed impossible tasks reveal whether optimized answers can score well while making forbidden claims.The benchmark has 169 impossible tasks, 48 answerable controls, and reported tests of rubrics from 11 generators.
Steering Geometry: Validating Human Value Geometry in LLM Steering SpaceRead this when activation steering success may hide a behavior-specific shortcut. It compares steering performance with theory-aligned value geometry and cross-value transfer.The 26K-sample benchmark covers 20 values from Schwartz’s theory across multiple steering methods, model families, and sizes.
How can teams test whether a generated rubric rewards honest answers?Generate the rubric, optimize candidate answers against it, and check the winners against independently verifiable constraints. ImpossibleRubrics pairs impossible tasks with oracle certificates specifying what honest answers may claim. This exposes certificate-violating answers that score well and shows why task-specific detail is unsafe when it identifies a claim to fabricate rather than a truth condition.ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals
Does successful activation steering imply that a vector represents a coherent human value?Not necessarily. On a 26K-sample benchmark covering 20 Schwartz values, distribution-driven methods recovered more theory-aligned value geometry, while behavior-centric methods achieved comparable steering performance with little expected geometric correlation. Check both target behavior and cross-value structure; compatible and opposing values provide a transfer test that isolated success misses.Steering Geometry: Validating Human Value Geometry in LLM Steering Space
Check tool actions before they change the environmentFor tool-using agents, a trajectory audit may arrive after a file change, disclosure, or unauthorized operation. Step-level guardrails move the decision point forward by evaluating a proposed action in context before execution, while retaining completed-trajectory audits. The central engineering tradeoff is not just detection accuracy, but attack reduction versus lost utility.
StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility BalancingUse this for the operational problem of stopping risky tool calls before execution. It covers matched-context supervision, safety–utility balancing, pre-execution checks, and trajectory audits.Agent-level results are reported on AgentDojo and AgentDyn; other tool stacks and threat models require separate evaluation.
Where should a guardrail sit in a tool-using agent loop?Place it immediately before tool execution, with the trajectory context and proposed action available, and retain trajectory auditing for later review. StepGuard trains on matched contexts whose safe and unsafe trajectories differ at the risky step, targeting the action boundary while balancing missed hazards against over-defense.StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing
What safety–utility tradeoff did the step-level approach report?On AgentDojo and AgentDyn, StepGuard reduced mean attack success by 77.3% relative to no guard while mean utility fell 2.8 percentage points. Those results make pre-execution checks worth testing, but the relevant deployment decision requires measuring the same two outcomes on your own tools, permissions, and attack paths.StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing