This paper presents a robot system for handling various objects in home or office environments. The robot, which has a manipulator, utilizes marks on objects for handling and motion planning. A mark consists of two parts. One is the outer part, which indicates its pose (In this paper “pose” denotes 3D position and orientation.), and the other is a QR code, which is a kind of 2D barcode. Several marks are attached on each object. A QR code in the mark is stored information of an object (e.g. its name, its shape, etc.). The robot accesses the information using a camera on its hand. The robot decides its handling motion for the object based on the information and poses of the object that are estimated from pose of marks. Using that information, the robot can plan a complex handling motion. Experiments are conducted to verify this method.