2026-09-21T20:28:43Zhttps://api.repository.cam.ac.uk/server/oai/request

oai:www.repository.cam.ac.uk:1810/2980482021-04-21T20:19:46Zcom_1810_213729com_1810_256065col_1810_219485

Sample-Efficient Deep Reinforcement Learning for Continuous Control 10.17863/CAM.45105 Gu, Shixiang 0000000192460896 Turner, Richard E. Ghahramani, Zoubin Schoelkopf, Bernhard Reinforcement learning (RL) is a powerful, generic approach to discovering optimal policies in complex sequential decision-making problems. Recently, with flexible function approximators such as neural networks, RL has greatly expanded its realm of applications, from playing computer games with pixel inputs, to mastering the game of Go, to learning parkour movements by simulated humanoids. However, the common RL approaches are known to be sample intensive, making them difficult to be applied to real-world problems such as robotics. This thesis makes several contributions toward developing RL algorithms for learning in the wild, where sample-efficiency and stability are critical. The key contributions include Normalized Advantage Functions (NAF), extending Q-learning for continuous action problems; Interpolated Policy Gradient (IPG), unifying prior policy gradient algorithm variants through theoretical analyses on bias and variance; and Temporal Difference Models (TDM), interpreting a parameterized Q-function as a generalized dynamics model for novel temporally abstracted model-based planning. Importantly, this thesis highlights that these algorithms can be seen as bridging gaps between branches of RL – model-based with modelfree, and on-policy with off-policy. The proposed algorithms not only achieve substantial improvements over the prior approaches, but also provide novel perspectives on how to mix different branches of RL effectively to gain the best of both worlds. NAF has subsequently been shown to be able to train two 7-DoF robot arms to open doors using only 2.5 hours of real-world experience, making it one of the first demonstrations of deep RL approaches on real robots. University of Cambridge 2019-11-01 Thesis Doctoral Doctor of Philosophy (PhD) en - Cambridge-Tuebingen PhD Fellowship in Machine Learning - Google Focused Research Award - NSERC https://www.repository.cam.ac.uk/handle/1810/298048 https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/ef84fbcb-530b-4039-a3a4-955d963299ef/download efacfd26b62dc0201d89abe01f079162 https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/42bb22d6-31b2-443b-8b1e-92f5d9a2bdfd/download 87eda9de84448d1f82354d60eee3eb5f https://www.rioxx.net/licenses/all-rights-reserved/ Reinforcement Learning Continuous Control Robotics Deep Learning Machine Learning Model-based Planning Model-free Reinforcement Learning