When we were a child, we ate a red food item. Oh my God, that was Sweet. You thought I will write hot. Well in your experience it may have been hot. So we associated the red colour of food items with either sweetness or hotness. That’s calibration at work in the human system.
It depends on how quickly we learn things. Exactly in the same fashion, the weights of the inputs are calibrated by assigning new weights in neural networks.
The new weights are arrived at by subtraction the rate of change of loss function with respect to the old weights from the old weights. Confusing. See the equation in the picture.
The learning rate decides the rate of calibration and is a crucial component. For multiple inputs with multiple weights, the backpropagation follows the chain rule of derivatives which results in a scary-looking formula.
Well, let’s forget the formula as of now.
Thus, back propagation ensures the calibration of weights until the loss function is minimized. But, how loss function/ error is minimized. Well, with the help of Optimizer. We will take up Gradient Descent (one of the optimizers) in our next discussion.
To know more, you can mail to smartsubu2020@gmail.com.
