Continuity and Smoothness Analysis and Possible Improvement of Traditional Reinforcement Learning Methods

Continuity and Smoothness Analysis and Possible Improvement of Traditional Reinforcement Learning Methods

연구 분야: Artificial Intelligence

학회: 2020 IEEE International Conference on Mechatronics and Automation (ICMA)

At present, the deep reinforcement learning method has become one of the important branches in the field of artificial intelligence. Its model-free feature makes it considered as one of the ways to achieve the goal of strong artificial intelligence. In addition to discrete decision-making tasks, deep reinforcement learning has been gradually applied to continuous control tasks. Nevertheless, compared with classical control strategies and methods, the instability of deep reinforcement learning limits its extensive application in real scenarios. The instability of reinforcement learning mainly comes from two aspects: the first is the inherent discontinuity of reinforcement learning action strategies; the second is the randomness of reinforcement learning action strategies. This paper will discuss theoretical reasons of this instability and evaluate the instability of the current mainstream reinforcement learning algorithms by time-frequency analysis. Finally, we give an improved framework based on stochastic differential equation, and theoretically solve the inherent discontinuity of reinforcement learning action strategy.