Defining and Quantifying Users' Mental Imagery-based BCI skills: a first step - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Neural Engineering Year : 2018

Defining and Quantifying Users' Mental Imagery-based BCI skills: a first step


Objective: While promising for many applications, Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are still scarcely used outside laboratories , due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires the use of appropriate reliability metrics to quantify both the classification algorithm and the BCI user's performances. So far, Classification Accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study BCI users' skills. Here, we propose a definition and new metrics to quantify such BCI skills for Mental Imagery (MI) BCIs, independently of any classification algorithm. Approach: We first show in this paper that CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful self-modulation of EEG patterns by the user. We then propose a definition of MI-BCI skills that reflects how well the user can self-modulate EEG patterns, and thus how well he could control an MI-BCI. Finally, we propose new performance metrics, classDis, restDist and classStab that specifically measure how distinct and stable the EEG patterns produced by the user are, independently of any classifier. Main results: By re-analyzing EEG data sets with such new metrics, we indeed confirmed that CA may hide some increase in MI-BCI skills or hide the user inability to self-modulate a given EEG pattern. On the other hand, our new met-rics could reveal such skill improvements as well as identify when a mental task performed by a user was no different than rest EEG. Significance: Our results showed that when studying MI-BCI users' skills, CA should be used with care, and complemented with metrics such as the new ones proposed. Our results also stressed the need to redefine BCI user training by considering the different BCI subskills and their measures. To promote the complementary use of our new metrics, we provide the Matlab code to compute them for free and open-source.
Fichier principal
Vignette du fichier
JNE18_MetricsBCISkills_revised.pdf (4.18 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-01846434 , version 1 (21-07-2018)



Fabien Lotte, Camille Jeunet. Defining and Quantifying Users' Mental Imagery-based BCI skills: a first step. Journal of Neural Engineering, 2018, 15 (4), pp.1-37. ⟨10.1088/1741-2552/aac577⟩. ⟨hal-01846434⟩
302 View
637 Download



Gmail Facebook X LinkedIn More