Reinforcement learning on the cart-pole problem

Tom Vodopivec (2011) Reinforcement learning on the cart-pole problem. EngD thesis.

Preview

PDF Download (2181Kb)

Abstract

The main goal of this thesis was the evaluation and implementation of two types of reinforcement learning algorithms on a computer-simulated control problem. Reinforcement learning is a branch of machine learning which combines principles of dynamic programming and supervised learning for problem solving. For the benchmark system we chose the cart-pole control problem as it is widely used in this field for testing the efficiency of learning algorithms. Out of the reinforcement learning methods we chose two algorithms for temporal difference learning. This type of learning uses methods of dynamic programming and Monte Carlo methods. The first chosen algorithm is Q-learning, the second is an actor-critic algorithm which is called learning by associative search element and adaptive critic element. In the purpose of achieving our goal, we developed a computer application for the experimental testing of the simulation of learning on a benchmark system. Our aim was to make this tool as modular and reusable as possible. We defined a different method of performance evaluation which was used to evaluate both learning algorithms on a wide set of simulation parameters. We also measured the computational performance of both algorithms.

Actions (login required)