Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year :

Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling

Alexis Juven
  • Function : Author
Xavier Hinaut


Understanding the mechanisms enabling children to learn rapidly word-to-meaning mapping through cross-situational learning in uncertain conditions is still a matter of debate. In particular, many models simply look at the word level, and not at the full sentence comprehension level. We present a model of language acquisition, applying cross-situational learning on Recurrent Neural Networks with the Reservoir Computing paradigm. Using the co-occurrences between words and visual perceptions, the model learns to ground a complex sentence, describing a scene involving different objects, into a perceptual representation space. The model processes sentences describing scenes it perceives simultaneously via a simulated vision module: sentences are inputs and simulated vision are target outputs of the RNN. Evaluations of the model show its capacity to extract the semantics of virtually hundred of thousands possible combinations of sentences (based on a context-free grammar); remarkably the model generalises only after a few hundred of partially described scenes via cross-situational learning. Furthermore, it handles polysemous and synonymous words, and deals with complex sentences where word order is crucial for understanding. Finally, further improvements of the model are discussed in order to reach proper reinforced and self-supervised learning schemes, with the goal to enable robots to acquire and ground language by themselves (with no oracle supervision).
Fichier principal
Vignette du fichier
JuvenHinaut2020_IJCNN.pdf (657.64 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02594725 , version 1 (15-05-2020)


  • HAL Id : hal-02594725 , version 1


Alexis Juven, Xavier Hinaut. Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling. 2020 International Joint Conference on Neural Networks (IJCNN 2020), Jul 2020, Glasgow, Scotland, United Kingdom. ⟨hal-02594725⟩
180 View
309 Download


Gmail Facebook Twitter LinkedIn More