Reward-­based online learning in non­stationary environments: adapting a P300­-speller with a ``Backspace’’ key - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2015

Reward-­based online learning in non­stationary environments: adapting a P300­-speller with a ``Backspace’’ key

Emmanuel Daucé
  • Fonction : Auteur
  • PersonId : 943315
Timothée Proix
  • Fonction : Auteur
  • PersonId : 969978

Résumé

We adapt a policy gradient approach to the problem of reward-based online learning of a non-invasive EEG-based ``P300''-speller. We first clarify the nature of the P300-speller classification problem and present a general regularized gradient ascent formula. We then show that when the reward is immediate and binary (namely ``bad response'' or ``good response''), each update is expected to improve the classifier accuracy, whether the actual response is correct or not. We also estimate the robustness of the method to occasional mistaken rewards, i.e. show that the learning efficacy may only linearly decrease with the rate of invalid rewards. The effectiveness of our approach is tested in a series of simulations reproducing the conditions of real experiments. We show in a first experiment that a systematic improvement of the spelling rate is obtained for all subjects in the absence of initial calibration. In a second experiment, we consider the case of the online recovery that is expected to follow unforeseen impairments. Combined with a specific failure detection algorithm, the spelling error information (typically contained in a ``backspace'' hit), is shown useful for the policy gradient to adapt the P300 classifier to the new situation, provided the feedback is reliable enough (namely having a reliability greater than 70%).
Fichier non déposé

Dates et versions

hal-01196513 , version 1 (09-09-2015)

Identifiants

  • HAL Id : hal-01196513 , version 1

Citer

Emmanuel Daucé, Timothée Proix, Liva Ralaivola. Reward-­based online learning in non­stationary environments: adapting a P300­-speller with a ``Backspace’’ key. IJCNN 2015, Jul 2015, Killarney, Ireland. ⟨hal-01196513⟩
86 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More