Dining table is one of the environments that remain to be explored by information technology and that could be important for happy and healthy life of everybody. We have been studying the Future Dining Table system, dining environment of the future. The system features the real time recognition of dining activity, which is represented by the history of the hand movement to the dishes on the table. The history of dining actions is used to know the current status of dining. Currently the system recommends additional dishes according to the dining status. It is expected to be useful as a substitute of a food server in the age of labor shortage.