Loughborough University
Leicestershire, UK
LE11 3TU
+44 (0)1509 263171
Loughborough University

Loughborough University Institutional Repository

Please use this identifier to cite or link to this item: https://dspace.lboro.ac.uk/2134/5796

Title: On the choice of parameters of the cost function in nested modular RNN's
Authors: Mandic, Danilo P.
Chambers, Jonathon
Keywords: Cost function
Forgetting factor
Pipelined recurrent neural networks
Recurrent neural networks
Issue Date: 2000
Publisher: © IEEE
Citation: MANDIC, D.P. and CHAMBERS, J.A., 2000. On the choice of parameters of the cost function in nested modular RNN's. IEEE Transactions on Neural Networks, 11(2), pp. 315 - 322
Abstract: We address the choice of the coefficients in the cost function of a modular nested recurrent neural-network (RNN) architecture, known as the pipelined recurrent neural network (PRNN). Such a network can cope with the problem of vanishing gradient, experienced in prediction with RNN’s. Constraints on the coefficients of the cost function, in the form of a vector norm, are considered. Unlike the previous cost function for the PRNN, which included a forgetting factor motivated by the recursive least squares (RLS) strategy, the proposed forms of cost function provide “forgetting” of the outputs of adjacent modules based upon the network architecture. Such an approach takes into account the number of modules in the PRNN, through the unit norm constraint on the coefficients of the cost function of the PRNN. This is shown to be particularly suitable, since due to inherent nesting in the PRNN, every module gives its full contribution to the learning process, whereas the unit norm constrained cost function introduces a sense of forgetting in the memory management of the PRNN. The PRNN based upon a modified cost function outperforms existing PRNN schemes in the time series prediction simulations presented.
Description: This article was published in the journal IEEE Transactions on Neural Networks [© IEEE] and is also available at: http://ieeexplore.ieee.org/. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Version: Published
DOI: 10.1109/72.839003
URI: https://dspace.lboro.ac.uk/2134/5796
ISSN: 1045-9227
Appears in Collections:Published Articles (Mechanical, Electrical and Manufacturing Engineering)

Files associated with this item:

File Description SizeFormat
mandic2.pdf171.2 kBAdobe PDFView/Open


SFX Query

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.