0 votes
in Deep Learning by
Explain Stochastic Gradient Descent. How is it different from Batch Gradient Descent?

1 Answer

0 votes
by

Stochastic Gradient Descent: Stochastic Gradient Descent seeks to tackle the major difficulty with Batch Gradient Descent, which is the use of the entire training set to calculate gradients at each step. It is stochastic in nature, which means it chooses up a "random" instance of training data at each step and then computes the gradient, which is significantly faster than Batch Gradient Descent because there are much fewer data to modify at once. Stochastic Gradient Descent is best suited for unconstrained optimization problems. The stochastic nature of SGD has a drawback in that once it gets close to the minimum value, it doesn't settle down and instead bounces around, giving us a good but not optimal value for model parameters. This can be solved by lowering the learning rate at each step, which will reduce the bouncing and allow SGD to settle down at the global minimum after some time.

Related questions

0 votes
asked Dec 13, 2022 in Deep Learning by Robindeniel
0 votes
asked Feb 8, 2023 in Data Science by john ganales
...