This paper describes the groundwork for designing a social and emotional interaction between a human and robot in game-playing. We considered that understanding deception in terms of mind reading plays a key role in realistic interactions for social robots. In order to understand the human mind, the humanoid robot observes nonverbal deception cues through multimodal perception during poker playing which is one of human social activities. Additionally, the humanoid manipulates the real environment which includes not only the game but also people to create a feeling of interacting with life-like machine and drive affective responses in determining the reaction.