Get Started
Home
Topics
Search
Library
Research questionHow can group-relative policy optimization enforce constraints without normalization coupling reward and constraint objectives?In constrained GRPO, normalizing scalarized rewards within each sampled group can make changing one constraint multiplier alter the effective weighting of the reward and other constraints. This coupling can destabilize multiplier dynamics and make constraint adherence harder to maintain while optimizing task performance.
AI
Alignment & Safety
LLM Pretraining & Post-training
Reinforcement Learning
Latest papersRecent research connected to this question, newest first.Constrained Group Relative Policy OptimizationApplies to LLM and VLM fine-tuning with Lagrangian constrained GRPO. The evidence covers a controlled gridworld, a real-world autonomous driving benchmark, and a mathematical reasoning task, measuring constraint adherence alongside task performance.research paper · Sep 2, 2026
Related questions
How can reinforcement learning reliably satisfy Value-at-Risk constraints during policy training?How can safety-critical control prioritize multiple uncertain risks while certifying how far a policy is from optimal?How can policy optimization for long-horizon LLM agents preserve useful transitions across updates when rollout groups are small?How can RLVR reduce the cost of on-policy rollouts and reliable targets without hurting reasoning quality?