NOVEL OPTIMIZATION TECHNIQUES BY NEURAL NETWORKS

Abstract

Optimization plays a significant role in almost every field of applied sciences (e.g., signal processing, optimum resource management, ... etc.). In spite of the ever-growing need for implementable solutions, the traditional methods of optimization suffer from the drawbacks of either failing to achieve the global optimum or yielding high complexity algorithms which are numerically cumbersome to perform. As a result, novel techniques using neural networks (e.g. Boltzmann machines and Hopfield networks) have been instrumental to optimization theory. Unfortunately, these novel techniques fell short of the expectations for two reasons: (i) in the case of statistical optimization the associated computational complexity is rather high leading to tedious algorithms, and (ii) in the case of Hopfield network only local optimization can be carried out. Therefore, the aim of this paper is to introduce new algorithms for global minimization of : Multi-Variable Quadratic Forms (MVQF) defined over discrete sets and for the statistical resource management problem. The global minimization of MVQFs will be solved by a modified Hopfield algorithm, where the convergence speed is proven to be a polynomial function of the dimension (in contrast to the exponential complexity of exhaustive search). A straightforward application of the result is to implement low complexity algorithms for the detection problem of linearly distorted signals corrupted by Gaussian noise. The constrained optimization problem of statistical resource management will be interpreted as a set separation problem approximated by a neural network. Based on the underlying tail estimation of the aggregate load, the weights of the network can be properly trained. The results can be directly applied to the traffic design of communication networks, automated factories, ... etc.