Loughborough University
Leicestershire, UK
LE11 3TU
+44 (0)1509 263171
Loughborough University

Loughborough University Institutional Repository

Please use this identifier to cite or link to this item: https://dspace.lboro.ac.uk/2134/27232

Title: An MDP model-based reinforcement learning approach for production station ramp-up optimization: Q-learning analysis
Authors: Doltsinis, Stefanos
Ferreira, Pedro
Lohse, Niels
Keywords: Decision-making
Learning systems
Manufacturing automation
Markov processes
Issue Date: 2014
Publisher: © IEEE
Citation: DOLTSINIS, S., FERREIRA, P. and LOHSE, N., 2014. An MDP model-based reinforcement learning approach for production station ramp-up optimization: Q-learning analysis. IEEE Transactions on Systems, Man and Cybernetics. Systems, 44 (9), pp.1125-1138.
Abstract: Ramp-up is a significant bottleneck for the introduction of new or adapted manufacturing systems. The effort and time required to ramp-up a system is largely dependent on the effectiveness of the human decision making process to select the most promising sequence of actions to improve the system to the required level of performance. Although existing work has identified significant factors influencing the effectiveness of rampup, little has been done to support the decision making during the process. This paper approaches ramp-up as a sequential adjustment and tuning process that aims to get a manufacturing system to a desirable performance in the fastest possible time. Production stations and machines are the key resources in a manufacturing system. They are often functionally decoupled and can be treated in the first instance as independent rampup problems. Hence, this paper focuses on developing a Markov decision process (MDP) model to formalize ramp-up of production stations and enable their formal analysis. The aim is to capture the cause-and-effect relationships between an operator’s adaptation or adjustment of a station and the station’s response to improve the effectiveness of the process. Reinforcement learning has been identified as a promising approach to learn from rampup experience and discover more successful decision-making policies. Batch learning in particular can perform well with little data. This paper investigates the application of a Q-batch learning algorithm combined with an MDP model of the ramp-up process. The approach has been applied to a highly automated production station where several ramp-up processes are carried out. The convergence of the Q-learning algorithm has been analyzed along with the variation of its parameters. Finally, the learned policy has been applied and compared against previous ramp-up cases.
Sponsor: This work was supported by the European Commission as a part of the CP-FP 246083-2 IDEAS and CP-FP 229208-2 FRAME Project.
Version: Not specified
DOI: 10.1109/TSMC.2013.2294155
URI: https://dspace.lboro.ac.uk/2134/27232
Publisher Link: https://doi.org/10.1109/TSMC.2013.2294155
ISSN: 2168-2216
Appears in Collections:Closed Access (Mechanical, Electrical and Manufacturing Engineering)

Files associated with this item:

File Description SizeFormat
06702489.pdfPublished version15.22 MBAdobe PDFView/Open


SFX Query

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.