Primates can learn complex sequences that can be represented in the form of abstract categories, and even more abstract hierarchical structures such as language. In order to study how these abstractions are formed and because of the highly recurrent connectivity in prefrontal cortex (PFC) we model part of it using recurrent neural networks. Particularly, we use the Reservoir Computing paradigm to model PFC and part of the basal ganglia: a recurrent neural network with random connections kept constant models the prefrontal cortex, and a read-out layer (i.e. output layer) models the striatum. This model was trained to perform language syntactic processing; in particular, thematic role assignment: for a given sentence this corresponds to answer the question "Who did what to whom?". Inspiring from language acquisition theories (Tomasello 2003), the model processes categories (i.e. abstractions) of sentences which are called "grammatical constructions" (Goldberg 1995). After training, it is able to (1) process correctly the majority of the grammatical constructions that were not learned, demonstrating generalization capabilities, and (2) to make online predictions (of thematic roles) while processing a grammatical construction. Moreover, we observed that when the model processes less frequent constructions an important shift in output predictions occurs. It is proposed that a significant modification of predictions in a short period of time is responsible for generating Evoked-Related Potentials (ERP) such as the P600 which typically occurs when unusual sentences structures are processed (Hinaut & Dominey 2013). Subsequently, to show the ability of the model to deal with a real-world application, the model was successfully applied in the framework of human-robot interaction for both sentence comprehension and production (Hinaut et al, 2014). Recently, we showed that the very same instance of reservoir could learn both English and French sentences at the same time, suggesting that a common "output" (striatal) representations could be used even in the case of different languages (Hinaut et al, 2015). Moreover, the model is able to learn small corpora in fifteen European or Asian languages with different word order (Hinaut et al, in revision). In a nutshell, this suggests that a random neural network with no prewired structure seems enough to learn the syntax of languages different in structure and in word order.