Ensemble strategies mix predictions from a number of fashions (usually choice bushes) to enhance accuracy and scale back overfitting. The 2 main gamers on this area are:
- Random Forest
- Gradient Boosting
Let’s break them down.
Consider Random Forest as a forest filled with unbiased choice bushes. Every tree will get a random subset of the info (each rows and columns), makes a prediction, and the ultimate prediction is the majority vote (for classification) or common (for regression).
Key Traits:
- Bagging approach: Every tree learns from a random subset (bootstrap pattern).
- Reduces variance, making the mannequin much less more likely to overfit.
- Performs effectively on many datasets with minimal tuning.
- Simply parallelizable = quick coaching.
🛠 Use case: Fast, dependable mannequin with good baseline efficiency.
Gradient Boosting builds bushes sequentially, the place every new tree corrects the errors of the earlier ones.
It focuses extra on examples that had been beforehand mispredicted and tries to reduce the general loss operate.
Key Traits:
- Boosting approach: Fashions are constructed one after the opposite.
- Reduces bias, resulting in greater accuracy (however greater danger of overfitting).
- Extra delicate to hyperparameters like studying charge and variety of estimators.
- Slower, however usually extra highly effective.
🛠 Use case: If you wish to squeeze the best accuracy out of your information, and you’ve got time to tune.
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score# Load information
information = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(information.information, information.goal, random_state=42)
# Random Forest
rf = RandomForestClassifier(n_estimators=100)
rf.match(X_train, y_train)
rf_preds = rf.predict(X_test)
# Gradient Boosting
gb = GradientBoostingClassifier(n_estimators=100, learning_rate=0.1)
gb.match(X_train, y_train)
gb_preds = gb.predict(X_test)
print("Random Forest Accuracy:", accuracy_score(y_test, rf_preds))
print("Gradient Boosting Accuracy:", accuracy_score(y_test, gb_preds))
You’ll usually discover Gradient Boosting sneaking forward in accuracy — however not at all times! It will depend on your dataset.
