The exploration of large video data is a task which is now possible because of the advances made on object detection and tracking. Data mining techniques such as clustering are typically employed. Such techniques have mainly been applied for segmentation/indexation of video but knowledge extraction on the activity contained in the video has been only partially addressed. In this paper we present how video information is processed with the ultimate aim to achieve knowledge discovery of people activity in the video. First, objects of interest are detected in real time. Then, in an off-line process, the information related to detected objects is set into a model format suitable for knowledge representation and discovery. We then apply two clustering processes: 1) Agglomerative hierarchical clustering to find the main trajectory patterns of people in the video 2) Relational analysis clustering, which we employ to extract spatio-temporal relations between people and contextual objects in the scene. We present results obtained on real videos of the Torino metro (Italy).