This paper presents an empirical comparison of covariance matrix averaging methods for EEG signal classification. Indeed , averaging EEG signal covariance matrices is a key step in designing brain-computer interfaces (BCI) based on the popular common spatial pattern (CSP) algorithm. BCI paradigms are typically structured into trials and we argue that this structure should be taken into account. Moreover, the non-Euclidean structure of covariance matrices should be taken into consideration as well. We review several approaches from the literature for averaging covariance matrices in CSP and compare them empirically on three publicly available datasets. Our results show that using Riemannian geometry for averaging covariance matrices improves performances for small dimensional problems, but also the limits of this approach when the dimensionality increases.