Adaptive Updating: Ecological Rationality Meets Reinforcement Learning

Adaptive Updating: Ecological Rationality Meets Reinforcement Learning

연구 분야: Artificial Intelligence

학회: Minds and Machines

Recent work has argued for an ecological perspective on the rationality of updating, showing that non-Bayesian update rules can outperform Bayesian updating in certain environments. However, this work has left unaddressed the question of how to determine which rule to use without prior knowledge of environmental features—a challenge we term the “dependency problem.” We propose a solution that uses reinforcement learning, specifically a multi-armed bandit framework, to enable dynamic rule selection. Computer simulations indicate that the “adaptive updating” this approach results in is able to solve the dependency problem at low cost in that it performs competitively with fixed update rules across varied contexts. While adaptive updating does not universally outperform fixed rules, it offers a viable alternative when the optimal rule for a given context is unknown.