This work aims at recognizing activities from large video datasets, using the object trajectories as the activity descriptors. We make usage of a compact structure based on 6 features to represent trajectories. This structure allows us to apply standard techniques for unsupervised clustering. We present a method to optimize trajectory clustering by tuning the set of trajectory feature weights in order to improve the performance of activity recognition. We have learned the weights and test the approach using different sets of real video data achieving domain knowledge independence. Moreover we define new performance measures that better describes the evaluation of the clustering process comparing the learned clusters with activity ground truth.