Neural Networks

Backpropagation

Sends the error backwards through the network to find how much each weight is to blame.

Run an input forward, measure the loss, then apply the chain rule layer by layer backwards to get the gradient of that loss with respect to every weight. An optimiser then nudges each weight against its gradient.

It is worth implementing once by hand. After that, vanishing and exploding gradients, the point of residual connections and the reason initialisation matters all stop being folklore.

Also in Neural Networks

Perceptron Feed Forward Hopfield Network Boltzmann Machine Self Organising Maps Deep Belief Network

Apply

Begin the first module

Become AI native, it is the real deal today, and if it is not for you, you have lost nothing but learnt a new skill.

Apply