Unsupervised Method (No Dependent Variable)
Principal Component Analysis (PCA)
Hierarchical Variable Clustering (Proc Varclus in SAS)
Variance Inflation Factor (VIF)
Remove zero and near-zero variance predictors
Mean absolute correlation. Removes the variable with the largest mean absolute correlation. See the detailed explanation of mean absolute correlation
Supervised Method (In respect to Dependent Variable)
For Binary / Categorical Dependent Variable
Information Value
Wald Chi-Square
Random Forest Variable Importance
Gradient Boosting Variable Importance
Forward/Backward/Stepwise - Variable Significance (p-value)
AIC / BIC score
For Continuous Dependent Variable
Adjusted R-Square
Mallows' Cp Statistic
Random Forest Variable Importance
AIC / BIC score
Forward / Backward / Stepwise - Variable Significance
