- Dactyl learns to solve the object reorientation task through simulations, without requiring any human intervention.
- It can analyze fingertips’ coordinates and images from 3 conventional RGB cameras.
- They used MuJoCo physics engine to measure physicals attributes like damping and friction.
For humans, dexterous manipulation of objects is a daily task, but it’s still challenging for autonomous machines. Today’s advanced robots are designed for a particular set of tasks in constraint environment and are unable to completely harness complex end-effectors.
It’s been more than a decade since the first human hand was developed, however, using such machines for manipulating physical objects in the real-world environment still remains one of the most difficult tasks to accomplish.
Recently, researchers at OpenAI unveiled a deep learning system named Dactyl that can grasp and manipulate 3D objects with unprecedented dexterity. The system learns to solve the object reorientation task in a simulated environment without requiring any human intervention.
They placed an object (a prism or a block) in the palm of the robot’s hand and asked it to re-position the object into different orientations, for instance, put a specific face on the front. Dactyl analyzes fingertips’ coordinates and the pictures from 3 conventional RGB cameras.
However, the task is not as simple as it sounds. It involves numerous challenges, such as operating in the real world conditions that give noisy and partial observations, controlling multiple dimension and manipulating more than one object.
New Approach
Image credit: OpenAI
Instead of maximizing realism, researchers adopted a new approach named domain randomization to provide a wide range of experience through simulation. They used MuJoCo physics engine to develop a simulated version of robotics. It helps measure physical attributes such as damping, friction and rolling resistance, which alter over time as the machine experiences wear and tear.
Since it’s quite impossible to infer most dynamics parameters, they used Long Short Term Memory networks, to learn more about the dynamics of the surrounding. Also, they used the same training code and algorithms developed for OpenAI Five, but the system architecture and hyper-parameters are different.
The pose estimator is trained on Convolutional Neural Network, which collects the video clips from 3 RGB cameras fixed near robotic hand, and evaluates position and orientation of the object. In order to make these evaluations accurate, multiple cameras were used.
Results
The neural network was trained on nearly 100 years of experience, which was achieved in just 50 hours, using NVIDIA Tesla V100 GPUs, and TensorFlow accelerated by CUDA deep learning framework. Once fully trained, the network was able to perform 50 consecutive rotations without dropping the object (cube).
Reference: arXiv:1808.00177 | OpenAI
To solve a specific task, the system uses a set of in-hand dexterous manipulation approaches that are regularly used by humans. It’s quite fascinating that the researchers didn’t explicitly teach the network to do this: it discovered these kinds of behaviors autonomously.
Dactyl uses the little finger and thumb for precision grasps like the tip pinch grasp, whereas, us humans prefer thumb and either middle or index finger to do perform the same task. An extra degree of freedom assigned to the robot’s little finger might explain why it prefers it. This indicates that the system is capable of rediscovering the natural grasps and adapting them to better suit its own body abilities and limitations.
Read: New AI System Teaches Robot To Learn Directly From Humans
The results show that deep reinforcement learning algorithms can be used to solve real-world problems that are beyond the reach of non-learning-based techniques. The researchers concede that their approach isn’t perfect yet, but they can improve it to perform much more complex tasks.
