Optimum seeking is one of the central issues in science, engineering, industry, economy, business and even in everyday life. Every problem we solve, every product we design and produce and every single thing we do are the outcome of the best possible choice. A variety of tools and techniques have been developed and applied to manmade artificial systems for optimum seeking; meanwhile, optimum seeking in nature, biological and social systems takes place in a completely different way by means of natural evolution. In all optimum seeking in artificial or natural systems, there are goals or objectives to be satisfied and there are constraints to meet within which the optimum has to be found. Eventually, the optimum seeking can be formulated as an optimization problem. That is, it is reduced to finding the best solution measured by a performance index. The performance indices are functionals (often known as objective functions in many areas of computing and engineering) that vary from problem to problem. In general, a performance index can be given by
where Q(x, c) is the functional of the vector c = (c1, c2, …, cN), which depends on the random sequence or process x = (x1, x2, …, xN) with probability density function p(x). The goal is to find the extremum of the functional Q(x, c), i.e., the minimum or maximum depending on the problem. The ...