Topic · 48 recaps

Reinforcement Learning

Learning from reward signals through trial and interaction with an environment. Spans classic RL, RLHF, and modern post-training methods for language and agent models.

Posts Questions

Get Started

Topic · 48 recaps

Reinforcement Learning

Learning from reward signals through trial and interaction with an environment. Spans classic RL, RLHF, and modern post-training methods for language and agent models.

Posts Questions

Questions researchers are working onFollow a question through Rcap’s explanations and the latest papers addressing it.

How can a single search agent improve multi-hop web research without sub-agents or test-time verification?Multi-hop web research requires an agent to connect evidence across several pages while retaining the information needed for later steps. Long search trajectories can overwhelm the available context, making both training and reliable performance difficult without additional agents or verification passes.How can active preference learning obtain scalable, calibrated uncertainty for neural reward models without full Bayesian inference?Active preference learning must choose which comparisons to request, but reliable uncertainty estimates become expensive for neural reward models when inference considers all parameters. Poorly calibrated uncertainty can lead to less informative queries and inefficient reward learning.How can agents choose task-dependent world-model rollout horizons for effective multi-step planning?Learned world models let agents imagine future environmental states without interacting with real environments. Single-step or fixed-horizon imagination may provide insufficient foresight when tasks require different amounts of planning and progress changes over time.How can AI tutors identify effective guidance for each learner from sparse learner data?Real learners provide too little feedback to determine which explanations or corrections help a particular student. Useful evidence must reflect both the learner’s existing behavior and how that behavior changes under tutor guidance.How can an autonomous audio system evolve sonic behavior without external data or post-initialization supervision?Without external data or feedback after initialization, an audio system must both produce changing sound and determine how its internal parameters should change. The difficulty is sustaining autonomous sonic evolution without simply becoming static or unstable.How can audio enhancement handle coupled real-world distortions while producing personalized, executable workflows?Real-world recordings can contain interacting distortions, so correcting one artifact may affect others. Enhancement must also adapt to personalization requirements while producing workflows that are valid and executable.How can banking agents safely handle sensitive account requests while staying grounded and using tools correctly?Banking agents must rely on trusted bank-specific information while distinguishing safe assistance from risky or out-of-scope requests. Account-related interactions also require correct tool use and cautious handling of sensitive customer situations.How can black-box adversarial attacks disrupt image-based continuous-control agents with fewer environment queries?In image-based continuous control, an attacker must infer how visual changes affect an agent while querying the environment. Large action spaces make this inference especially expensive and can weaken attacks.How can closed-loop vision-language navigation learn effectively despite distribution shift and sparse micro-action rewards?An agent’s actions change the observations and states it will encounter, causing imitation policies to face distribution shift and ambiguous supervision after deviations. Direct reinforcement learning over low-level movements is also inefficient when rewards are sparse.How can code-repair models fix bugs without unnecessarily rewriting surrounding code?A repair can pass correctness tests yet alter unrelated code, making review harder and weakening confidence that the original implementation was preserved.How can compact device surrogates stay physically consistent and accurate across unseen process or geometry splits with scarce data?Sparse device data make it difficult to learn behavior that remains reliable when fabrication processes or device geometries differ from those observed during training. The surrogate must capture physically consistent trends while extrapolating across these distribution shifts.How can conversational reinforcement learning coordinate strategic utterance choices with token generation under sparse, delayed rewards?A conversational agent must decide both which communicative strategy to pursue and how to realize it token by token. Feedback often arrives at the utterance or conversation level, making credit assignment across long interactions difficult.How can cooperative MARL detect meaningful environment or task changes online without redundant alarms?Past experience can become unreliable when an environment or task objective changes during training. Agents must distinguish meaningful shifts in reward behavior from noise and avoid repeatedly reacting to the same change.How can decentralized heterogeneous robots combine round-level policy reasoning with tick-level local control without destabilizing navigation learning?Decentralized robots must translate infrequent policy updates into reliable low-level actions while their local controllers continue adapting. Different policy agents and shared feedback add coordination challenges without a central action planner.How can DeepResearch systems obtain scalable, query-specific reward signals for report quality?Generic rubrics may miss the fine-grained requirements of a particular research query, while manually writing such rubrics is expensive and difficult to scale. This makes it difficult to turn human judgments about report quality into reliable signals for system improvement.How can dexterous robot hands generalize a manipulation skill from one demonstration to varied objects despite sim-to-real gaps?Dexterous hands must reproduce useful contact strategies with limited real-world training data. Variation in object properties and discrepancies between simulation and reality make reliable transfer difficult.How can diffusion image and video generators be preference-aligned without inefficient training exploration or inference-time search?Policy-gradient preference optimization can explore inefficiently, become trapped in local optima, and exploit reward signals in ways that reduce semantic faithfulness or visual realism. Alignment must also avoid requiring extra search during generation.How can direct vision-language-action robot policies capture multi-timescale dynamics without learning undesirable behavior from mixed-quality deployment trajectories?Behavior cloning may reuse trajectories with very different outcomes without separating useful dynamics from undesirable behavior. Its representations may also fail to preserve how scenes and tasks evolve across multiple time horizons.How can distributed VLA reinforcement learning coordinate variable-latency simulation, inference, and optimization?Synchronous training can leave workers idle when some simulations take longer than others. Variable rollout costs therefore make it difficult to keep simulation, inference, and optimization resources continuously utilized.How can functional bilevel optimization adapt online as learning objectives change over time?Functional bilevel optimization is designed mainly for static, offline problems, where the objectives and data distribution do not change during optimization. Online learning requires its hierarchical updates to remain stable while responding to evolving objectives.

1 / 7