Reference
- Title: Distributional Inverse Reinforcement Learning
- Authors: Feiyang Wu, [+2 others]
- Venue/date: arXiv 2510.03013, posted Oct 3 2025 (v1)
- PDF: 2510-03013.pdf
- Link: https://arxiv.org/abs/2510.03013
Why relevant
Sits adjacent to bajgar-2024-valuewalk and the thesis-hub continuous-action extension work — another framework that goes beyond a deterministic point-estimate reward, but takes a different route than ValueWalk’s Bayesian posterior-over-rewards approach.
Summary
- Proposes DistIRL, an offline IRL framework that jointly models (a) uncertainty over the reward function and (b) the full distribution of returns — not just a deterministic reward estimate or expected-return matching.
- Core mechanism: minimizes first-order stochastic dominance (FSD) violations, and integrates distortion risk measures (DRMs) into policy learning → recovers both a reward distribution and a distribution-aware (risk-sensitive) policy.
- Theoretical: convergence with O(ε⁻²) iteration complexity.
- Validated on synthetic benchmarks, real-world neurobehavioral data, and MuJoCo control tasks; claims SOTA imitation performance.
- Explicitly does not claim unique ground-truth reward recovery — recovers a reward distribution compatible with the chosen prior, variational family, and FSD-based inductive bias. Same epistemic humility framing as Bayesian IRL / ValueWalk.
Ablation → baseline mapping (useful for related-work table)
- BIRL framework ≈ their Dis-TD-Mean configuration
- RIZE (Karimi & Ebadzadeh, 2025, arXiv 2502.20089) ≈ Det-Qt-Mean — performs worst in their ablations
- Det-TD-Mean ≈ ValueDice, but with explicit reward estimation
Open questions / to check
- How does the FSD/DRM machinery compare to ValueWalk’s MCMC-over-reward-posterior approach — complementary (could stack) or a competing paradigm?
- Worth reading Karimi & Ebadzadeh 2025 RIZE (arXiv 2502.20089) and Ke et al. 2025 switching-rewards/animal-behavior IRL (arXiv 2501.12633) — both cited as close relatives, and the animal-behavior angle overlaps with ValueWalk’s neurobehavioral motivation.
- Check if their D4RL / MuJoCo continuous-control benchmarks give a direct comparison point for the continuous-action-extension-directions work.