A neuro-computational model showing the effects of ventral striatum lesion on the computation of reward prediction error in VTA
Résumé
One of the earliest attempts to understand how animals learn involved pairing an
unconditioned stimulus (US) with a cue or conditioned stimulus (CS) and observing that
animals start responding to the CS after some point in time. This is the basis of pavlovian
learning, a fundamental learning mechanism in animals, which has been addressed by
several models of neural networks. We have developed a model focusing on the mechanism
of reward prediction error within pavlovian learning and studied the effects of Ventral
Striatum (VS) lesions to illustrate a fundamental dissociation of magnitude and timing
replicating experimental studies.
The paradigm used to evaluate the model is a simple CS-US associative learning task and
considers also how the expectation cancels out the dopamine peak at the time of the
reward. The trial duration is 500 time steps with each time step corresponding to 1ms. The
stimulus is presented at the 10th time step and is kept switched on till the arrival of the
reward at the 400th time step (400ms). The reward and the stimulus have by default a
magnitude of 1. The number of trials for the entire conditioning to happen was 16 trials.
Virtual lesions of VS to VTA GABA was made by disconnecting the link between them. Two
experiments were conducted where the time and magnitude were varied. In the first
experiment, a reward magnitude of 2 is given instead of 1. The reward prediction error in
VTA still shows a firing of 1 indicating the magnitude is conserved even after the VS lesion. In
the second experiment, an early reward is delivered which doesn’t show firing compared to
the control scenario where firing exists thus replicating the studies done by Takahashi(2016).
The results show that there exists fundamental dissociation of magnitude and temporal
when calculating reward prediction error (RPE) in the VTA. We propose this is achieved
through magnitude being computed in the PPN FT neurons and time being computed in the
ventral striatum (VS) respectively. The implications of this model bring into light new
interpretations of dopamine firing extending to state prediction errors and creating a
sensory representation before learning the reward fully.
Domaines
Réseau de neurones [cs.NE]Origine | Fichiers produits par l'(les) auteur(s) |
---|