We have proposed a system called Attentive Workbench (AWB) that is designed to support desktop work. AWB is the system which estimates a worker’s condition using some sensors and assists a worker using movable trays and projectors from both physical and informational viewpoints. A method for estimating an AWB user’s intention is proposed and presented herein. An AWB user expresses his/her intention by pointing a finger at a necessary tray. For estimating which tray a user needs, we take two approaches. One is to consider user’s subjective pointing direction, and the other is to integrate the sensor data and user’s past action sequences using Dynamic Bayesian Network. A description of the experimental results will demonstrate the usefulness of the method proposed here.