Universität zu Köln

Tuzsus, Deniz (2025). Reinforcement Learning in dynamic environments: Comparing human and artificial recurrent neural networks regarding the exploration/exploitation tradeoff. PhD thesis, Universität zu Köln.

PDF Dissertation_Deniz_Tuzsus.pdf - Published Version Bereitstellung unter der CC-Lizenz: Creative Commons Attribution No Derivatives. Download (13MB)

Abstract

Adaptive decision-making requires balancing exploration and exploitation, especially in volatile environments. This dissertation examines the computational mechanisms of this tradeoff in humans and recurrent neural networks (RNNs) using restless bandit tasks. The first study compares human learners and 24 RNN architectures, revealing that humans favor directed exploration, while RNNs show higher-order perseveration. Computational modeling confirms these distinct strategies, with human learners leveraging uncertainty-based exploration bonuses and RNNs relying on reinforcement learning heuristics. The second study explores meta-reinforcement learning (meta-RL) in adapting to volatility. RNNs trained under varying conditions demonstrate that exposure to diverse environments enhances adaptability, with meta-volatility-trained networks dynamically adjusting learning rates and exploration strategies, mirroring human-like flexibility. Unpublished analyses further investigate the role of RNN capacity in shaping learning behaviors. Larger networks exhibit increased directed exploration and reduced perseveration, aligning more closely with human learners. These findings suggest that network complexity enables artificial agents to approximate human-like decision-making. Overall, this dissertation provides new insights into how biological and artificial agents optimize exploration-exploitation tradeoffs, highlighting the role of experience, environmental volatility, and neural capacity in adaptive learning​

Item Type:	Thesis (PhD thesis)
Translated title:	Title Language UNSPECIFIED German
Creators:	Creators Email ORCID ORCID Put Code Tuzsus, Deniz UNSPECIFIED UNSPECIFIED UNSPECIFIED
URN:	urn:nbn:de:hbz:38-750160
Date:	2025
Language:	Multiple languages
Faculty:	Faculty of Human Sciences
Divisions:	Faculty of Human Sciences > Department Psychologie
Subjects:	Psychology
Uncontrolled Keywords:	Keywords Language Reinforcement Learning English Cognitive Neuroscience English Artificial Intelligence English
Date of oral exam:	31 October 2024
Referee:	Name Academic Title Peters, Jan Prof. Dr. Pappas, Ioannis Prof. Dr.
Refereed:	Yes
URI:	http://kups.ub.uni-koeln.de/id/eprint/75016