Cost function

Cost function

• In economics, the cost curve, expressing production costs in terms of the amount produced.
• In mathematical optimization, the loss function, a function to be minimized.
• In artificial neural networks, the cost function is a function that the network wants to minimize. This is often represented as the difference between the target and the network's output. (e.g. mean absolute error, mean square error). Different cost functions implement different notions of "distance" between the target and the network output (e.g. softmax cross-entropy, Kullback-Leibler divergence), but the general idea remains the same: the cost function gives a distance metric, and the network tries to make the distance negligible.