CONF dimitrakakis:pascal:2005/IDIAP Gradient estimates of return distributions Dimitrakakis, Christos Bengio, Samy EXTERNAL http://publications.idiap.ch/attachments/papers/2005/dimitrakakis-pascal-2005.pdf PUBLIC http://publications.idiap.ch/index.php/publications/showcite/dimitrakakis:rr05-29 Related documents PASCAL Workshop on Principled Methods of Trading Exploration and Exploitation 2005 IDIAP-RR 05-29 We present a general method for maintaining estimates of the distribution of parameters in arbitrary models. This is then applied to the estimation of probability distributions over actions in value-based reinforcement learning. While this approach is similar to other techniques that maintain a confidence measure for action-values, it nevertheless offers an insight into current techniques and hints at potential avenues of further research. REPORT dimitrakakis:rr05-29/IDIAP Gradient estimates of return Dimitrakakis, Christos Bengio, Samy EXTERNAL http://publications.idiap.ch/attachments/reports/2005/dimitrakakis-idiap-rr-05-29.pdf PUBLIC Idiap-RR-29-2005 2005 IDIAP Published in PASCAL Workshop in Principled Methods of Trading Exploration and Exploitation, London, UK, 2005 The exploration-exploitation trade-off that arises when one considers simple point estimates of expected returns no longer appears when full distributions are considered. This work develops a simple gradient-based approach for mainting such distributions and investigates methods for using them to direct exploration.