The loss functions (also called Cost functions) quantify the difference between the expected output and the actual one obtained from the training phase. The all process consists to calculate all the parameters in order to minimize the loss function. The loss functions can differ based on the type of learning task we are dealing with such as regression or classification tasks. The most common ones are the Mean square error, quadratic loss,  hinge loss, or entropy loss.