0 votes
in Deep Learning by
can you please explain difference between exploding and vanishing gradients?

1 Answer

0 votes
by

By taking incremental steps towards the minimal value, the gradient descent algorithm aims to minimize the error. The weights and biases in a neural network are updated using these processes.

However, at times, the steps grow excessively large, resulting in increased updates to weights and bias terms — to the point where the weights overflow (or become NaN, that is, Not a Number). An exploding gradient is the result of this, and it is an unstable method.

On the other hand, if the steps are excessively small, it results in minor – even negligible – changes in the weights and bias terms. As a result, we may end up training a deep learning model with nearly identical weights and biases every time, never reaching the least error function. The vanishing gradient is what it's called.

Related questions

0 votes
asked Dec 12, 2022 in Deep Learning by Robin
0 votes
asked Apr 25, 2023 in Looker by SakshiSharma
...