Evaluation of Techniques for Sim2Real Reinforcement Learning





Actor-Critic, Deep Q-Learning, Reinforcement Learning, Transfer Learning, Sim2Real, Robotic


Reinforcement learning (RL) has demonstrated promising results in transferring learned policies from simulation to real-world environments. However, inconsistencies and discrepancies between the two environments cause a negative transfer. The phenomenon is commonly known as the “reality gap.” The reality gap prevents learned policies from generalizing to the physical environment. This paper aims to evaluate techniques to improve sim2real learning and bridge the reality gap using RL. For this research, a 3-DOF Stewart Platform was built virtually and physically. The goal of the platform was to guide and balance the marble towards the center of the Stewart platform. Custom API was created to induce noise, manipulate in-game physics, dynamics, and lighting conditions, and perform domain randomization to improve generalization. Two RL algorithms; Q-Learning and Actor-Critic were implemented to train the agent and to evaluate the performance in bridging the reality gap. This paper outlines the techniques utilized to create noise, domain randomization, perform training, results, and observations. Overall, the obtained results show the effectiveness of domain randomization and inducing noise during the agents' learning process. Additionally, the findings provide valuable insights into implementing sim2real RL algorithms to bridge the reality gap.

Author Biography

Qusay Mahmoud, Ontario Tech University

Qusay H. Mahmoud (Senior Member, IEEE) was the Founding Chair of the Department of Electrical, Computer, and Software Engineering, Ontario Tech University, Canada, where he is currently a Professor of software engineering. He is also the Associate Dean of experiential learning and engineering outreach with the Faculty of Engineering and Applied Science. His research interests include intelligent software systems and cybersecurity.




How to Cite

Ranaweera, M., & Mahmoud, Q. (2023). Evaluation of Techniques for Sim2Real Reinforcement Learning. The International FLAIRS Conference Proceedings, 36(1). https://doi.org/10.32473/flairs.36.133317



Special Track: Autonomous Robots and Agents