Grasping and Manipulation

Work performed at the KTH Royal Institute of Technology

To allow humanoid robots to operate in real environments they must be capable of dealing with novelty, uncertainty and unforeseen situations. The ultimate goal of the project was the design of a cognitive system capable of performing grasping and manipulation tasks in open-ended environments. This work focused on the perception issues in such a system. One outcome was a novel technique for the classification of 3D data where Gaussian processes were used to infer spatial occupancy by combining stereo and tactile sensing with a humanoid hand. To aid in the practical use of robots in domestic applications, another platform was developed with natural language processing. This system allowed for the interactive segmentation and recognition of household objects from stereo data. The underlying processing relied on a Markov random field framework to perform sensor fusion between speech and vision.

Associated Publications: