Get Started
Home
Topics
Search
Library
Research questionHow can on-policy diffusion RL reuse rollout computations without trading redundant recomputation for excessive memory use?Diffusion RL methods often recompute selected rollout timesteps with gradient tracking during the update. Reusing rollout activations can remove that computation, but retaining them or tracking gradients during rollout can substantially increase memory use.
Diffusion Models
Image Generation
Inference Optimization
Machine Learning
Reinforcement Learning
Video Generation
Latest papersRecent research connected to this question, newest first.LeanGRPO: Eliminating Redundant Recomputation in Diffusion RLApplies to on-policy trajectory-logprob diffusion RL with the same backend used for rollout and policy updates. Reported evidence covers FlowGRPO and DanceGRPO with FLUX.1-dev and Wan, including end-to-end speedups of up to 1.83× while preserving the original optimization objective.research paper · Sep 3, 2026
Related questions
How can RLVR reduce the cost of on-policy rollouts and reliable targets without hurting reasoning quality?How can reinforcement learning post-training for diffusion models avoid objective mismatch and high-variance updates?How can policy optimization for long-horizon LLM agents preserve useful transitions across updates when rollout groups are small?How can offline reinforcement learning improve policies beyond dataset support while keeping value estimates reliable under distribution shift?