in Data Handling by
Q:
What do you mean by exploding and vanishing gradients in Deep learning?

1 Answer

0 votes
by

The key here is to make the explanation as simple as possible. As we know, the gradient descent algorithm tries to minimize the error by taking small steps towards the minimum value. These steps are used to update the weights and biases in a neural network.

However, at times, the steps become too large and this results in larger updates to weights and bias terms – so much so as to cause an overflow (or a NaN) value in the weights. This leads to an unstable algorithm and is called an exploding gradient.

On the other hand, the steps are too small and this leads to minimal changes in the weights and bias terms – even negligible changes at times. We thus might end up training a deep learning model with almost the same weights and biases each time and never reach the minimum error function. This is called the vanishing gradient.

A point to note is that both these issues are specifically evident in Recurrent Neural Networks – so be prepared for follow-up questions on RNN!

 

Click here to read more about Loan/Mortgage
Click here to read more about Insurance

Related questions

0 votes
asked Nov 2, 2020 in Data Handling by AdilsonLima
0 votes
asked Nov 2, 2020 in Data Handling by AdilsonLima
0 votes
asked Aug 28, 2020 in Service Now by RShastri
0 votes
asked Sep 17, 2020 in Agile by SakshiSharma
0 votes
asked May 27, 2019 in Data Handling by tempuser
...