Loughborough University
Leicestershire, UK
LE11 3TU
+44 (0)1509 263171
Loughborough University

Loughborough University Institutional Repository

Please use this identifier to cite or link to this item: https://dspace.lboro.ac.uk/2134/16992

Title: Solving the distal reward problem with rare correlations
Authors: Soltoggio, Andrea
Steil, Jochen
Issue Date: 2013
Publisher: © Massachusetts Institute of Technology Press
Citation: SOLTOGGIO, A. and STEIL, J., 2013. Solving the distal reward problem with rare correlations. Neural Computation, 25 (4), pp. 940-978
Abstract: In the course of trial-and-error learning, the results of actions, manifested as rewards or punishments, occur often seconds after the actions that caused them. How can a reward be associated with an earlier action when the neural activity that caused that action is no longer present in the network? This problem is referred to as the distal reward problem. A recent computational study proposes a solution using modulated plasticity with spiking neurons and argues that precise firing patterns in the millisecond range are essential for such a solution. In contrast, the study reported in this letter shows that it is the rarity of correlating neural activity, and not the spike timing, that allows the network to solve the distal reward problem.In this study, rare correlations are detected in a standard rate-based computational model by means of a thresholdaugmented Hebbian rule. The novel modulated plasticity rule allows a randomly connected network to learn in classical and instrumental conditioning scenarios with delayed rewards. The rarity of correlations is shown to be a pivotal factor in the learning and in handling various delays of the reward. This study additionally suggests the hypothesis that short-term synaptic plasticity may implement eligibility traces and thereby serve as a selectionmechanism in promoting candidate synapses for long-term storage.
Description: This article is © Massachusetts Institute of Technology Press.
Sponsor: This work was supported by the European Community’s Seventh Framework Programme FP7/2007-2013, Challenge 2 Cognitive Systems, Interaction, Robotics (Grant No. 248311—AMARSi).
Version: Published
DOI: 10.1162/NECO_a_00419
URI: https://dspace.lboro.ac.uk/2134/16992
Publisher Link: http://dx.doi.org/10.1162/NECO_a_00419
ISSN: 0899-7667
Appears in Collections:Published Articles (Computer Science)

Files associated with this item:

File Description SizeFormat
NECO_a_00419.pdfPublished version1.59 MBAdobe PDFView/Open


SFX Query

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.