This paper proposes a simulation-based reinforcement learning algorithm for controlling systems with uncertain and varying system parameters. While simulators are useful to safely synthesize control policies for physical systems using reinforcement learning, mitigating the reality gap remains a major challenge. To address the challenge, we propose a two-stage algorithm. In the first stage, multiple control policies are learned for systems with different parameters in a simulator. In the second stage, for a real system, the control policies learned in the first stage are smoothly switched using an online convex optimization algorithm based on observations. Our proposed algorithm is demonstrated through numerical experiments.
Categories: eess.SY cs.SY