Recognizing the Right Tool for the Job

Estimated reading time: 1 minute

A*STAR researchers working with colleagues in Japan have developed a method by which robots can automatically recognize an object as a potential tool and use it, despite never having seen it before.

For humans, the ability to recognize and use tools is almost instinctive. There are also many examples in which tool use seems hardwired into the brain of animals: some birds and primates use sticks or stones to obtain food, for example. One proposed reason for this neurologically embedded ability to use tools is that the animal’s brain perceives the external object as an extension of its own body. Inspired by this idea, Keng Peng Tee and his colleagues from the A*STAR Institute for Infocomm Research, along with Gowrishankar Ganesh from the CNRS-AIST Joint Robotics Laboratory located in Tsukuba, Japan developed an algorithm that enables robots to recognize, and immediately use tools that they have never seen before.

Previous research by Ganesh has indicated that the human brain recognizes a limb not just by its physical features, but by its functionality. Building on this insight, Ganesh and the A*STAR team proposed that a robot can recognize the potential of a tool by comparing its shape with that of its own hand and arm when used to achieve this same task. They wrote an algorithm based on this idea and tested it by setting a robot the task of moving, without grasping, a disk to a desired end point on a table. This involved the robot either pulling the disk toward itself, pushing it away, or shifting it sideways.

“The robot is programmed with the ability to complete the task by itself, represented by a hand shape template,” explains Tee. “When the task cannot be completed, because its arms aren’t long enough for example, the robot is able to recognize a viable tool by matching the tool shape with the hand shape template.”

Thus, the algorithm enabled the robot to successfully recognize when a task cannot be performed without a tool, to identify an unknown object as a potential tool, and then to use the tool to achieve the task. “Next we will look into automatic learning of the features that represent existing skills, rather than pre-defining hand shape templates,” says Tee.