In recent years pattern recognition has evolved to a mature discipline and has been
successfully applied to various problems. A fundamental part of an automatic pattern
recognition system is classification, where a pattern vector is assigned to one of a finite
number of classes. This thesis reports on the development and design of pattern
classifier algorithms, with particular emphasis on statistical algorithms which employ
The first part of this research work investigates the use of linear discriminant functions
as pattern classifiers. A comparison of some well known methods, including
Perceptron, Widrow-Hoff and Ho-Kashyap, is presented.
Using generalised linear modelling a new method of training discriminant functions is
developed. In this method the linear discriminant function is transformed by a non-linear
link function which associates with each pattern vector a measure which is bounded in
the range of 0 to 1 according to the class membership of the pattern. In simulations the
GLM approach is applied both to synthetic data and to experimental data from a binary
pattern matching problem. It is seen that GLM exhibits faster and more reliable
convergence than existing linear discriminant approaches.
Extensions of this method to Piecewise linear discriminant functions and to polynomial
discriminant functions are explored. Application of self-organising methods for efficient
generation of polynomial discriminant functions is also investigated.
In the second part of the work a review of neural networks is presented, followed by an
analysis and formulation of a popular neural network training algorithm, namely Backpropagation
(BP). The capabilities and deficiencies of BP and its variations are
experimentally evaluated by computer simulations.
An alternative formulation based on Empirical Maximum Likelihood (EML) is also
proposed. This approach is shown to have a simpler error landscape in comparison to
the original BP based on mean square error. Simulations show that the EML approach
generally provides faster convergence, involves fewer calculations per iteration than
conventional BP, and results in equally good classification performance.
A Doctoral Thesis. Submitted in partial fulfilment of the requirements for the award of Doctor of Philosophy of Loughborough University.