A cartpole experiment benchmark for trainable controllers
- 1 October 1993
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems
- Vol. 13 (5) , 40-51
- https://doi.org/10.1109/37.236324
Abstract
The inverted pendulum problem, i.e., the cartpole, which is often used for demonstrating the success of neural network learning methods, is addressed. It is shown that a random search in weight space can quickly uncover coefficients (weights) for controllers that work over a wide range of initial conditions. This result indicates that success in finding a satisfactory neural controller is not sufficient proof for the effectiveness of unsupervised training methods. By analyzing the dynamics of the linear controller, the cartpole problem is reformulated to make it a more stringent test for neural training methods. A review of the literature on unsupervised training methods for cartpole controllers shows that the published results are difficult to compare and that for most of the methods there is not clear evidence of better performance than the random search method.Keywords
This publication has 19 references indexed in Scilit:
- Genetic reinforcement learning for neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Evolving neural network controllers for unstable systemsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- USE OF CMAC NEURAL NETWORKS IN REINFORCEMENT SELF-LEARNING CONTROL11The work was supported by the National Science Foundation under the grant MIP-8808995.Published by Elsevier ,1991
- GENETICALLY PROGRAMMED NEURAL NETWORK FOR SOLVING POLE-BALANCING PROBLEMPublished by Elsevier ,1991
- CMAC-based adaptive critic self-learning controlIEEE Transactions on Neural Networks, 1991
- A neurocontroller based on model feedback and the adaptive heuristic criticPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- A self-organizing controller for dynamic processes using neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- Unsupervising adaption neural-network controlPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- Neuronlike adaptive elements that can solve difficult learning control problemsIEEE Transactions on Systems, Man, and Cybernetics, 1983
- Punish/Reward: Learning with a Critic in Adaptive Threshold SystemsIEEE Transactions on Systems, Man, and Cybernetics, 1973