Hyper-parameters are parameters that are not directly learnt within estimators. In scikit-learn they are passed as arguments to the constructor of the estimator classes. Typical examples include C
, kernel
and gamma
for Support Vector Classifier, alpha
for Lasso, etc.
It is possible and recommended to search the hyper-parameter space for the best cross validation score.
Any parameter provided when constructing an estimator may be optimized in this manner. Specifically, to find the names and current values for all parameters for a given estimator, use:
estimator.get_params()
A search consists of:
-
an estimator (regressor or classifier such as
sklearn.svm.SVC()
); -
a parameter space;
-
a method for searching or sampling candidates;
-
a cross-validation scheme; and
Some models allow for specialized, efficient parameter search strategies. Two generic approaches to sampling search candidates are provided in scikit-learn: for given values, GridSearchCV
exhaustively considers all parameter combinations, while RandomizedSearchCV
can sample a given number of candidates from a parameter space with a specified distribution.
To learn more about hyperparameter tuning in scikit-learn, please visit this link.
A code example of Parameter estimation using grid search with cross-validation is provided at this link.
Comments
0 comments
Article is closed for comments.