Decision Tree

Decision tree is one of the predictive modelling approaches used in statistics, data mining and machine learning.

It uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item’s target value (represented in the leaves).

• Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.

• Decision trees where the target variable can take continuous values (typically real numbers) are called regression trees.

Decision trees are among the most popular machine learning algorithms given their intelligibility and simplicity.

Note

Quinlan, J. R. 1986. Induction of Decision Trees. Mach. Learn. 1, 1 (Mar. 1986), 81-106

Growing trees

• Introduction decision_tree_introduction.pdf.

• Classification classification.pdf

• Regression regression.pdf.

• Overfitting for decision tree overfitting_dt.pdf

• Summary summary_dt.pdf

Random Forest

• How it works How_it_works.pdf.

• How to evaluate How_to_evaluate.pdf.

• How to optimize How_to_optimize.pdf.

Adaboost

• How it works adaboost.pdf.

Gradient Boosting

• Classification gb_classification.pdf.

• Regression gb_regression.pdf.

XGBoost

• Classification xgboost_classification.pdf.

• Regression xgboost_regression.pdf.