Decision making study: methods and applications of evidential reasoning and judgment analysis

Decision making study has been the multi-disciplinary research involving operations researchers, management scientists, statisticians, mathematical psychologists and economists as well as others. This study aims to investigate the theory and methodology of decision making research and apply them to different contexts in real cases. The study has reviewed the literature of Multiple Criteria Decision Making (MCDM), Evidential Reasoning (ER) approach, Naturalistic Decision Making (NDM) movement, Social Judgment Theory (SJT), and Adaptive Toolbox (AT) program. On the basis of these literatures, two methods, Evidence-based Trade-Off (EBTO) and Judgment Analysis with Heuristic Modelling (JA-HM), have been proposed and developed to accomplish decision making problems under different conditions. In the EBTO method, we propose a novel framework to aid people s decision making under uncertainty and imprecise goal. Under the framework, the imprecise goal is objectively modelled through an analytical structure, and is independent of the task requirement; the task requirement is specified by the trade-off strategy among criteria of the analytical structure through an importance weighting process, and is subject to the requirement change of a particular decision making task; the evidence available, that could contribute to the evaluation of general performance of the decision alternatives, are formulated with belief structures which are capable of capturing various format of uncertainties that arise from the absence of data, incomplete information and subjective judgments. The EBTO method was further applied in a case study of Soldier system decision making. The application has demonstrated that EBTO, as a tool, is able to provide a holistic analysis regarding the requirements of Soldier missions, the physical conditions of Soldiers, and the capability of their equipment and weapon systems, which is critical in domain. By drawing the cross-disciplinary literature from NDM and AT, the JA-HM extended the traditional Judgment Analysis (JA) method, through a number of novel methodological procedures, to account for the unique features of decision making tasks under extreme time pressure and dynamic shifting situations. These novel methodological procedures include, the notion of decision point to deconstruct the dynamic shifting situations in a way that decision problem could be identified and formulated; the classification of routine and non-routine problems, and associated data alignment process to enable meaningful decision data analysis across different decision makers (DMs); the notion of composite cue to account for the DMs iterative process of information perception and comprehension in dynamic task environment; the application of computational models of heuristics to account for the time constraints and process dynamics of DMs decision making process; and the application of cross-validation process to enable the methodological principle of competitive testing of decision models. The JA-HM was further applied in a case study of fire emergency decision making. The application has been the first behavioural test of the validity of the computational models of heuristics, in predicting the DMs decision making during fire emergency response. It has also been the first behavioural test of the validity of the non-compensatory heuristics in predicting the DMs decisions on ranking task. The findings identified extend the literature of AT and NDM, and have implications for the fire emergency decision making.