Start here (per reading-roadmap). The simpler MCMC-based entry point into this line of work.
Reference
- Title: Walking the Values in Bayesian Inverse Reinforcement Learning
- Authors: Ondrej Bajgar, Alessandro Abate, Konstantinos Gatsis, Michael Osborne
- Venue: UAI 2024 (Proceedings of the 40th Conference on Uncertainty in Artificial Intelligence), PMLR 244:273–287
- Link: https://proceedings.mlr.press/v244/bajgar24a.html
In one line
Bayesian IRL recovers a posterior over reward functions from expert demonstrations. Vanilla Bayesian IRL is costly because it must solve the forward planning problem (rewards → Q-values) thousands of times. ValueWalk flips this: sample primarily in Q-value space (Q → reward is radically cheaper), which also makes the gradient easy to compute — enabling efficient Hamiltonian Monte Carlo sampling.
My notes
TBD — fill in as I read.
Key equations / method
TBD
Relevance to thesis
TBD — this is the framework we extend.