This paper presents a new algorithm to track a high number of points in a video sequence in real-time. We propose a fast keypoint detector, used to create new particles, and an associated multiscale de-scriptor (feature) used to match the particles from one frame to the next. The tracking algorithm updates for each particle a series of appearance and kinematic states, that are temporally filtered. It is robust to hand held camera accelerations thanks to a coarse-to-fine dominant movement estimation. Each step is designed to reach the maximal level of data par-allelism, to target the most common parallel platforms. Using graphics processing unit, our current implementation handles 10 000 points per frame at 55 frames-per-second on 640 × 480 videos.