We have proposed “Attentive Workbench (AWB),” a deskwork support system. Using cameras, projectors, and self- moving trays, the system supports human from both physical and informational aspects. This paper deals with the problem for estimating the object that a user needs. We present one of the approaches for accurate estimation of a necessary object from the user’s pointing gesture. The system arranges the self- moving trays by itself based on the user’s past action sequences. Using force-directed method, good placement, which decrease false estimations and is not moved very much from the initial placement, can be obtained. A description of the experimental results demonstrates the usefulness of the method proposed here.