Categories

Dec 27, 2019 in Data Science

Q: What are Entropy and Information gain in Decision tree algorithm?

1 Answer

Dec 27, 2019

The core algorithm for building a decision tree is called ID3. ID3 uses Entropy and Information Gain to construct a decision tree.

Entropy

A decision tree is built top-down from a root node and involve partitioning of data into homogenious subsets. ID3 uses enteropy to check the homogeneity of a sample. If the sample is completely homogenious then entropy is zero and if the sample is an equally divided it has entropy of one.

 

Information Gain

The Information Gain is based on the decrease in entropy after a dataset is split on an attribute. Constructing a decision tree is all about finding attributes that return the highest information gain.

Click here to read more about Loan/Mortgage
Click here to read more about Insurance

Related questions

Madanswer
Nov 29, 2019 in Machine Learning
Jan 18 in Machine Learning
Dec 27, 2019 in Data Science
...