- New robot learns to pick and throw arbitrary objects into specific locations.
- It uses both deep learning and projectile physics to accurately and quickly toss objects in unstructured settings.
- Within 14 hours of training time, it achieved a throwing accuracy of 85%, with 87% grasping reliability in clutter.
In the last decade, significant progress has been made in making robots smarter to do specific tasks more efficiently and learn from real-world experiences. However, when it comes the basic skills — such as catching, swinging, spinning, and tossing — robots still lag far behind than humans.
Now, researchers at Google, MIT, Princeton University, and Columbia University have developed a new robotic arm that can learn to pick and throw arbitrary objects into specific locations. It uses both physics and deep learning methods to accurately and quickly toss random objects in unstructured settings.
They have named this robot TossingBot. It can achieve up to two times faster picking speed than previous robots, with twice the effective placing range.
Challenges Involved
Throwing something isn’t an easy task for robots. There are numerous factors involved, from how objects of different shapes are picked up to physical properties of objects like mass, aerodynamics, and friction.
For instance, if you pick a heavy, long object from its edge and throw it, it would land farther away than if you had picked it from the center. However, if you pick a light object like a ping pong ball, you would require a large force (due to air resistance) to throw it at the same distance.
Manually developing a technique that explicitly controls all these parameters for each arbitrary object is almost impossible. Applying trial-and-error technique is also not a good idea as it is both expensive and time-consuming.
Combining Deep Learning And Physics
While deep learning can help robot learn from experience instead of relyin g on case-by-case mechanisms, tossing objects precisely to their target position requires a good understanding of projectile physics.
Reference: arXiv:1903.11239 | Google AI Blog
By integrating both these features, engineers enabled TossingBot to train rapidly and generalize to new scenarios. Using some fundamental laws of projectile physics, the robot develops initial controllers, for instance, it calculates how much throwing velocity is required to throw a specific object at a target position.
Then neural networks predict adjustments on top of that physics calculations, to compensate for external factors like variability and noise in the real world. Besides training, the robotic arm relies on the NVIDIA Titan GPU for reconstructing scenes (captured with RGB depth cameras) in 3D in real time and aggregate 3D data as the arm moves around.
Within 14 hours of training time, TossingBot achieved a throwing accuracy of 85%, with 87% grasping reliability in clutter. It was able to pick and throw more than 500 arbitrary objects into the boxes placed outside its maximum reach range.
What’s Next?
Although results seem quite impressive, the robot does have its shortcomings. For instance, it presumes that objects are strong enough to resist landing collisions. Also, it evaluates control variables only from visual information.
Read: New AI System Teaches Robot To Learn Directly From Humans
In the next study, researchers will train their system to catch objects in ways that cushion the landing. They also plan to explore additional sensing modalities (like tactile and force-torque) that may enable the robotic arm to better adapt its throwing velocities.
