+1 vote
in Data Science by
What are Entropy and Information gain in Decision tree algorithm?

1 Answer

0 votes
by

The core algorithm for building a decision tree is called ID3. ID3 uses Entropy and Information Gain to construct a decision tree.

Entropy

A decision tree is built top-down from a root node and involve partitioning of data into homogenious subsets. ID3 uses enteropy to check the homogeneity of a sample. If the sample is completely homogenious then entropy is zero and if the sample is an equally divided it has entropy of one.

 

Information Gain

The Information Gain is based on the decrease in entropy after a dataset is split on an attribute. Constructing a decision tree is all about finding attributes that return the highest information gain.

Related questions

+1 vote
asked Oct 21, 2021 in Artificial Intelligence by DavidAnderson
0 votes
asked Dec 27, 2019 in Data Science by sharadyadav1986
...