This paper presents a novel approach to visuo-haptic perception of grasping/manipulative tasks. The proposed approach is founded on a hierarchical Bayesian model which integrates the visual information with the haptic data to reach a reasonable percept of what is happening in grasping tasks. The primary goal of the approach is to identify what type of grasping behaviour is being performed by the human subject, and as a secondary goal, to simultaneously assess the quality of the respective grasping behaviour. For a simple set of grasping behaviours defined in this paper, preliminary experimental results indicate that the proposed approach could result in a robust and efficient perception of grasp behaviours.