Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Clutter

Abstract

When operating in unstructured environments such as warehouses, homes, and retail centers, robots are frequently required to interactively search for and retrieve specific objects from cluttered bins, shelves, or tables. Distinct from prior work on grasping in clutter, Mechanical Search describes a class of tasks where the number of actions required to locate and extract the target object scales with the size of the clutter set. In this paper we formalize this class and study an instance where distractor objects are heaped over the target object in a bin. The robot uses an RGBD perception system and control policies to iteratively select among, parameterize, and perform 3 actions – push, suction, grasp – until the target object is successfully retrieved. We present a detailed study of 5 algorithmic policies for mechanical search, with over 15,000 simulated trials and 250 physical trials for heap sizes ranging from 10 to 20 objects. Results suggest that success can be achieved in this long-horizon task with algorithmic policies in over 95% of instances and that the number of actions required scales approximately linearly with the size of the clutter set.