1. Filter Strategies
These use statistical strategies to attain every function individually.
- Examples: Correlation coefficient, Chi-squared take a look at, ANOVA
- Greatest for: Fast pre-processing earlier than mannequin coaching
from sklearn.feature_selection import SelectKBest, f_classif
selector = SelectKBest(score_func=f_classif, ok=5)
X_new = selector.fit_transform(X, y)
2. Wrapper Strategies
These use a predictive mannequin to attain function subsets primarily based on efficiency.
- Examples: Recursive Function Elimination (RFE)
- Greatest for: Smaller datasets the place accuracy is a precedence
from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegressionmannequin = LogisticRegression()
rfe = RFE(mannequin, n_features_to_select=5)
X_new = rfe.fit_transform(X, y)
3. Embedded Strategies
Function choice occurs naturally as a part of mannequin coaching.
- Examples: Lasso Regression (L1 regularization), Tree-based fashions (like Random Forests)
- Greatest for: Fashions that help built-in choice
from sklearn.linear_model import LassoCV
mannequin = LassoCV().match(X, y)
significance = mannequin.coef_
