Research questionHow can reinforcement-learning planning optimize nonlinear objectives over visitation measures?Planning-as-inference is often formulated around linear rewards, leaving the treatment of nonlinear functionals of state-action visitation unclear. The interpretation of optimization updates and temporal-difference errors also needs to extend beyond that linear setting.