A new approach for unconstrained optimization of a function f(x)
has been investigated. The method is based on solving the differential
equation dx/dt = ± ∇f(x) which defines orthogonal trajectories in Rⁿ-space.
A number of numerical integration techniques have been used for solving
the above differential equation, the most powerful one which gives rise to
a very efficient optimization algorithm is the generalization of the
Trapezoidal rule. The interaction between the parameters which appear
as a result of using the numerical integration has been investigated.
In the above approach factorization of the positive definite matrix
(θG + λI), allowing some control over the diagonal elements of the matrix.
A Liapunov function approach has been used in constructing a number
of different differential equations of the above form. It is well known
that if a Liapunov function which satisfies certain conditions can be
found for a given system of differential equations then the origin of
the system is stable. Pursuing this idea further we constructed a Liapunov
function and then the corresponding differential equation. Application
of this differential equation to the problem of finding a minimum of a
function f is shown to yield a vector that converges to a point where
∇f = 0.
The use of differential equations is also extended to the optimal
control problem. The technique is only applicable to unconstrained optimal
control problems. If a terminal condition and inequality constraints are
presented, the problem should be converted to unconstrained form, e.g. by
the use of penalty functions. The method tends to converge, even from a
poor approximation point to the minimum without using line searches.
A Doctoral Thesis. Submitted in partial fulfilment of the requirements for the award of Doctor of Philosophy of Loughborough University.