in Data Science by
What are Entropy and Information gain in Decision tree algorithm?

1 Answer

0 votes

The core algorithm for building a decision tree is called ID3. ID3 uses Entropy and Information Gain to construct a decision tree.


A decision tree is built top-down from a root node and involve partitioning of data into homogenious subsets. ID3 uses enteropy to check the homogeneity of a sample. If the sample is completely homogenious then entropy is zero and if the sample is an equally divided it has entropy of one.


Information Gain

The Information Gain is based on the decrease in entropy after a dataset is split on an attribute. Constructing a decision tree is all about finding attributes that return the highest information gain.

Related questions

0 votes
asked Oct 21, 2021 in Artificial Intelligence by DavidAnderson
0 votes
asked Feb 3, 2020 in Cassandra by rajeshsharma
+2 votes
asked Jan 21, 2020 in Data Science by sharadyadav1986