Deep Reinforcement Learning Based Synergies Pushing and Grasping Policies in Cluttered Scene Using UR5 Robot

Abstract

This paper presents deep reinforcement learning-based synergy between pushing and grasping systems to improve the grasping performance of the UR5 robot in a cluttered scene. In robotic manipulations, grasping an object in a clutter is fundamental yet a challenging activity for industrial applications. This is because numerous studies focused on improving grasping performance in cluttered environments using either a grasping-only policy or pushing and grasping without incorporating a pushing reward. Additionally, some research has been limited to using similar objects, such as cubes. This paper for mulated the mathematical modeling of the universal robot manipulator. The proposed model involves training two fully-connected convolutional neural networks that transfer visual observations of the scene to a dense pixel-wise sample of end-effector orientations and positions for each pushing and grasping action. A fixed RGB-D camera is used to take the raw images of the scene and generate a heightmap image. Before feeding the heightmap image to the fully convolutional network, it is rotated by 36 different angles to generate 36 pixel-wise Q-value predictions. Both pushing and grasping networks are self-supervised by trial and error from experience and are trained together in a deep Q-learning algorithm. Successful grasps have a reward of 1, while successful pushes have a 0.5 reward value. But unsuccessful actions have a reward of 0 value. The proposed policy learns pushing motions to improve future grasping in a cluttered scene. The experiment demonstrates that the proposed model can successfully grasp objects with an 87% grasp success rate while grasping only policy, no-reward for pushing policy, and stochastic gradient without momentum is 60%, 71%, and 79% respectively. Further, it has been demonstrated that the proposed model is capable of generalizing to randomly arranged cluttered objects, challenging arrangements, and novel objects.