Question

What are the similarities and differences between bagging and boosting in Machine Learning?

Answer 1

Similarities of Bagging and Boosting

• Both are the ensemble methods to get N learns from 1 learner.
• Both generate several training data sets with random sampling.
• Both generate the final result by taking the average of N learners.
• Both reduce variance and provide higher scalability.

Differences between Bagging and Boosting

• Although they are built independently, but for Bagging, Boosting tries to add new models which perform well where previous models fail.
• Only Boosting determines the weight for the data to tip the scales in favor of the most challenging cases.
• Only Boosting tries to reduce bias. Instead, Bagging may solve the problem of over-fitting while boosting can increase it.