Even though derivative methods are mathematically efficient, the
calculations of the Jacobian vector and the Hessian matrix are
computationally hard or, simply, not possible in many cases.
To cope with this drawback, metaheuristic algorithms, many of them based
on natural processes, are widely applied in real-world problems. Among
them, swarm-based algorithms use the same search process described by
equation (1). However, in contrast with derivative-based algorithms, in
metaheuristic algorithms each solution is brought to a new position
(solution) based on a set of rules that replace the derivative
information with other kind of information, while providing a
conceptually similar approach.
We first concisely present the classical PSO, since we are going to use
it for comparisons purposes.