According to Wikipedia, “a **loss function** or **cost function** is a function that maps an event or values of one or more variables onto a real number intuitively representing some ‘cost’ associated with the event.”(link) The definition works well in Machine Learning.

Classification:

0/1 loss:

However, 0/1 loss is not convex, thus hard to optimize. To work around this problem, we use surrogate loss functions. These are convex functions that simulate the 0/1 loss and will always be the upper bound of 0/1 loss:

Hinge loss:

Logistic loss:

entropy

cross-entropy:

Exponential loss:

Squared loss:

Regression:

- L1 loss / absolute loss:
- L2 loss / squared loss:
- Gini index

Advertisements