Vision algorithms face many challenging issues when it comes to analyze human activities in video surveillance applications. For instance, occlusions makes the detection and tracking of people a hard task to perform. Hence advanced and adapted solutions are required to analyze the content of video sequences. We here present a people detection algorithm based on a hierarchical tree of Histogram of Oriented Gradients referred to as HOG. The detection is coupled with independently trained body part detectors to enhance the detection performance and to reach state of the art performances. We adopt a person tracking scheme which calculates HOG dissimilarities between detected persons throughout a sequence. The algorithms are tested in videos with challenging situations such as occlusions. False alarms are further reduced by using 2D and 3D information of moving objects segmented from a background reference frame.