Gesture Based Robot Arm Control

Abstract

Robotics is a constantly evolving field nowadays. A robot is a mechanical device that can perform physical tasks while being controlled and supervised by humans. Several robots have been created to perform risky tasks that are impossible for humans to perform directly. The robot arm is one of the most popular types. In this project, it is proposed to control a robot arm using finger and hand movements. However, the traditional way of gesture control employs sensor-based gloves to track motion, which is time and energy intensive due to its weight. To overcome this problem, using vision for motion tracking can be the most suited method at hand currently. The ML based computer vision method provides us with the real-time landmark on different points of the hand. With these landmarks we can calculate various mathematical parameters that can be used for the control of the robot arm. Movements that are landmarked, are processed and formulated using the mathematical parameters. The final output of mathematical parameters fed to the microcontroller. Signals from the microcontroller will be the desired input signal to the motors. The rotation of the robot arm is controlled using movement of the human arm. The end application of this project is to be as similar as human operation in dull, dirty and dangerous environments. The project's major goal is to protect those who come into direct contact with these environments. Bomb defusing, painting, welding, and other applications are a few examples of where this project is useful.