注意
前往末尾 下载完整的示例代码。
手动指定超参数
理所当然，您可能想先尝试一些特定的超参数组合，例如初始学习率和叶子节点的数量。此外，在让 Optuna 找到更好的超参数组合之前，您可能已经尝试过这些组合了。
Optuna 提供了两个 API 来支持这种情况：
传递这些超参数组合并让 Optuna 进行评估 -
enqueue_trial()将这些组合的结果添加为已完成的
Trial-add_trial()
第一种场景：让 Optuna 评估您的超参数
在此场景中，我们假设您有一些开箱即用的超参数组合，但尚未对其进行评估，并决定使用 Optuna 来寻找更好的超参数组合。
Optuna 提供了 optuna.study.Study.enqueue_trial()，它允许您将这些超参数组合传递给 Optuna，然后 Optuna 将对其进行评估。
本节将通过 LightGBM 向您展示如何使用此 API。
import lightgbm as lgb
import numpy as np
import sklearn.datasets
import sklearn.metrics
from sklearn.model_selection import train_test_split
import optuna
定义目标函数。
def objective(trial):
data, target = sklearn.datasets.load_breast_cancer(return_X_y=True)
train_x, valid_x, train_y, valid_y = train_test_split(data, target, test_size=0.25)
dtrain = lgb.Dataset(train_x, label=train_y)
dvalid = lgb.Dataset(valid_x, label=valid_y)
param = {
"objective": "binary",
"metric": "auc",
"verbosity": -1,
"boosting_type": "gbdt",
"bagging_fraction": min(trial.suggest_float("bagging_fraction", 0.4, 1.0 + 1e-12), 1),
"bagging_freq": trial.suggest_int("bagging_freq", 0, 7),
"min_child_samples": trial.suggest_int("min_child_samples", 5, 100),
}
gbm = lgb.train(param, dtrain, valid_sets=[dvalid])
preds = gbm.predict(valid_x)
pred_labels = np.rint(preds)
accuracy = sklearn.metrics.accuracy_score(valid_y, pred_labels)
return accuracy
然后，构建用于超参数优化的 Study。
study = optuna.create_study(direction="maximize", pruner=optuna.pruners.MedianPruner())
在这里，我们让 Optuna 评估一些具有较大 "bagging_fraq" 值和默认值的组合。
study.enqueue_trial(
{
"bagging_fraction": 1.0,
"bagging_freq": 0,
"min_child_samples": 20,
}
)
study.enqueue_trial(
{
"bagging_fraction": 0.75,
"bagging_freq": 5,
"min_child_samples": 20,
}
)
import logging
import sys
# Add stream handler of stdout to show the messages to see Optuna works expectedly.
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))
study.optimize(objective, n_trials=100, timeout=600)
Trial 0 finished with value: 0.9230769230769231 and parameters: {'bagging_fraction': 1.0, 'bagging_freq': 0, 'min_child_samples': 20}. Best is trial 0 with value: 0.9230769230769231.
Trial 1 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.75, 'bagging_freq': 5, 'min_child_samples': 20}. Best is trial 1 with value: 0.958041958041958.
Trial 2 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4936956428097461, 'bagging_freq': 2, 'min_child_samples': 44}. Best is trial 2 with value: 0.9790209790209791.
Trial 3 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6096086401568614, 'bagging_freq': 1, 'min_child_samples': 12}. Best is trial 2 with value: 0.9790209790209791.
Trial 4 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4278833097442535, 'bagging_freq': 6, 'min_child_samples': 68}. Best is trial 2 with value: 0.9790209790209791.
Trial 5 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6393483189813849, 'bagging_freq': 1, 'min_child_samples': 76}. Best is trial 2 with value: 0.9790209790209791.
Trial 6 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8564403863197028, 'bagging_freq': 5, 'min_child_samples': 73}. Best is trial 2 with value: 0.9790209790209791.
Trial 7 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.919851864574096, 'bagging_freq': 7, 'min_child_samples': 85}. Best is trial 2 with value: 0.9790209790209791.
Trial 8 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7206383092904824, 'bagging_freq': 0, 'min_child_samples': 89}. Best is trial 2 with value: 0.9790209790209791.
Trial 9 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.4927959619824987, 'bagging_freq': 1, 'min_child_samples': 39}. Best is trial 2 with value: 0.9790209790209791.
Trial 10 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.520842854629073, 'bagging_freq': 3, 'min_child_samples': 48}. Best is trial 2 with value: 0.9790209790209791.
Trial 11 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4629721126946265, 'bagging_freq': 3, 'min_child_samples': 60}. Best is trial 2 with value: 0.9790209790209791.
Trial 12 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4114315539780604, 'bagging_freq': 7, 'min_child_samples': 37}. Best is trial 12 with value: 0.986013986013986.
Trial 13 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5608611860101326, 'bagging_freq': 4, 'min_child_samples': 34}. Best is trial 12 with value: 0.986013986013986.
Trial 14 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4209390142471938, 'bagging_freq': 2, 'min_child_samples': 37}. Best is trial 12 with value: 0.986013986013986.
Trial 15 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5731559318511987, 'bagging_freq': 7, 'min_child_samples': 51}. Best is trial 12 with value: 0.986013986013986.
Trial 16 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.6531947704610194, 'bagging_freq': 4, 'min_child_samples': 27}. Best is trial 12 with value: 0.986013986013986.
Trial 17 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.410208903092778, 'bagging_freq': 2, 'min_child_samples': 6}. Best is trial 12 with value: 0.986013986013986.
Trial 18 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7995565903496722, 'bagging_freq': 6, 'min_child_samples': 46}. Best is trial 12 with value: 0.986013986013986.
Trial 19 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5194882802433703, 'bagging_freq': 2, 'min_child_samples': 60}. Best is trial 12 with value: 0.986013986013986.
Trial 20 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.47808368717381217, 'bagging_freq': 5, 'min_child_samples': 28}. Best is trial 12 with value: 0.986013986013986.
Trial 21 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.4060933294834282, 'bagging_freq': 6, 'min_child_samples': 100}. Best is trial 12 with value: 0.986013986013986.
Trial 22 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4408884104829003, 'bagging_freq': 6, 'min_child_samples': 62}. Best is trial 12 with value: 0.986013986013986.
Trial 23 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5491334294197904, 'bagging_freq': 7, 'min_child_samples': 68}. Best is trial 12 with value: 0.986013986013986.
Trial 24 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4628796502623389, 'bagging_freq': 6, 'min_child_samples': 44}. Best is trial 12 with value: 0.986013986013986.
Trial 25 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.4937713961525373, 'bagging_freq': 7, 'min_child_samples': 42}. Best is trial 12 with value: 0.986013986013986.
Trial 26 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5942831244759195, 'bagging_freq': 4, 'min_child_samples': 57}. Best is trial 12 with value: 0.986013986013986.
Trial 27 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4624174743111566, 'bagging_freq': 3, 'min_child_samples': 27}. Best is trial 12 with value: 0.986013986013986.
Trial 28 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.5365097907040215, 'bagging_freq': 5, 'min_child_samples': 52}. Best is trial 12 with value: 0.986013986013986.
Trial 29 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6647476873937921, 'bagging_freq': 0, 'min_child_samples': 18}. Best is trial 12 with value: 0.986013986013986.
Trial 30 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.9678742713188756, 'bagging_freq': 6, 'min_child_samples': 31}. Best is trial 12 with value: 0.986013986013986.
Trial 31 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.459469143256188, 'bagging_freq': 6, 'min_child_samples': 42}. Best is trial 12 with value: 0.986013986013986.
Trial 32 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.40207821813900935, 'bagging_freq': 7, 'min_child_samples': 69}. Best is trial 12 with value: 0.986013986013986.
Trial 33 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.43878069808879644, 'bagging_freq': 6, 'min_child_samples': 45}. Best is trial 12 with value: 0.986013986013986.
Trial 34 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5014594151208983, 'bagging_freq': 5, 'min_child_samples': 55}. Best is trial 12 with value: 0.986013986013986.
Trial 35 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.6115355623563453, 'bagging_freq': 7, 'min_child_samples': 82}. Best is trial 12 with value: 0.986013986013986.
Trial 36 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.43351312923622937, 'bagging_freq': 6, 'min_child_samples': 22}. Best is trial 12 with value: 0.986013986013986.
Trial 37 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7642631577675119, 'bagging_freq': 5, 'min_child_samples': 65}. Best is trial 12 with value: 0.986013986013986.
Trial 38 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6871788119004733, 'bagging_freq': 2, 'min_child_samples': 75}. Best is trial 12 with value: 0.986013986013986.
Trial 39 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.508260251855874, 'bagging_freq': 1, 'min_child_samples': 39}. Best is trial 12 with value: 0.986013986013986.
Trial 40 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.46999475663106877, 'bagging_freq': 4, 'min_child_samples': 50}. Best is trial 12 with value: 0.986013986013986.
Trial 41 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.43173795901034995, 'bagging_freq': 2, 'min_child_samples': 35}. Best is trial 12 with value: 0.986013986013986.
Trial 42 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.40029853272655497, 'bagging_freq': 3, 'min_child_samples': 37}. Best is trial 42 with value: 0.993006993006993.
Trial 43 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.44830999595847487, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 42 with value: 0.993006993006993.
Trial 44 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.40158087099344275, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 42 with value: 0.993006993006993.
Trial 45 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.401844651351964, 'bagging_freq': 3, 'min_child_samples': 31}. Best is trial 42 with value: 0.993006993006993.
Trial 46 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.45287278640552936, 'bagging_freq': 3, 'min_child_samples': 22}. Best is trial 42 with value: 0.993006993006993.
Trial 47 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4837200180010109, 'bagging_freq': 3, 'min_child_samples': 40}. Best is trial 42 with value: 0.993006993006993.
Trial 48 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.4342606459719654, 'bagging_freq': 4, 'min_child_samples': 16}. Best is trial 42 with value: 0.993006993006993.
Trial 49 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5772739709768128, 'bagging_freq': 3, 'min_child_samples': 47}. Best is trial 42 with value: 0.993006993006993.
Trial 50 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8936034581903496, 'bagging_freq': 4, 'min_child_samples': 34}. Best is trial 42 with value: 0.993006993006993.
Trial 51 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4001940532656309, 'bagging_freq': 3, 'min_child_samples': 32}. Best is trial 42 with value: 0.993006993006993.
Trial 52 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.4216530249407962, 'bagging_freq': 3, 'min_child_samples': 37}. Best is trial 42 with value: 0.993006993006993.
Trial 53 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5291871412934925, 'bagging_freq': 3, 'min_child_samples': 30}. Best is trial 42 with value: 0.993006993006993.
Trial 54 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4584223670321039, 'bagging_freq': 2, 'min_child_samples': 24}. Best is trial 42 with value: 0.993006993006993.
Trial 55 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4012749118781496, 'bagging_freq': 4, 'min_child_samples': 43}. Best is trial 42 with value: 0.993006993006993.
Trial 56 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4808184849871344, 'bagging_freq': 2, 'min_child_samples': 48}. Best is trial 42 with value: 0.993006993006993.
Trial 57 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4238464757471973, 'bagging_freq': 3, 'min_child_samples': 55}. Best is trial 42 with value: 0.993006993006993.
Trial 58 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.44414599028022217, 'bagging_freq': 1, 'min_child_samples': 39}. Best is trial 42 with value: 0.993006993006993.
Trial 59 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4181618304630704, 'bagging_freq': 4, 'min_child_samples': 10}. Best is trial 42 with value: 0.993006993006993.
Trial 60 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5063967714496723, 'bagging_freq': 3, 'min_child_samples': 35}. Best is trial 42 with value: 0.993006993006993.
Trial 61 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.4845299204569288, 'bagging_freq': 3, 'min_child_samples': 41}. Best is trial 42 with value: 0.993006993006993.
Trial 62 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4715840429534251, 'bagging_freq': 3, 'min_child_samples': 45}. Best is trial 42 with value: 0.993006993006993.
Trial 63 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.44792767991143734, 'bagging_freq': 2, 'min_child_samples': 38}. Best is trial 42 with value: 0.993006993006993.
Trial 64 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4192701585249307, 'bagging_freq': 3, 'min_child_samples': 28}. Best is trial 42 with value: 0.993006993006993.
Trial 65 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5428588439062996, 'bagging_freq': 4, 'min_child_samples': 49}. Best is trial 42 with value: 0.993006993006993.
Trial 66 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.45257284654564245, 'bagging_freq': 2, 'min_child_samples': 41}. Best is trial 42 with value: 0.993006993006993.
Trial 67 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.49261849432836113, 'bagging_freq': 3, 'min_child_samples': 32}. Best is trial 42 with value: 0.993006993006993.
Trial 68 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.8056662278294942, 'bagging_freq': 7, 'min_child_samples': 53}. Best is trial 42 with value: 0.993006993006993.
Trial 69 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.41560200333706937, 'bagging_freq': 4, 'min_child_samples': 25}. Best is trial 42 with value: 0.993006993006993.
Trial 70 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.47078858049743927, 'bagging_freq': 5, 'min_child_samples': 44}. Best is trial 42 with value: 0.993006993006993.
Trial 71 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.456097745716627, 'bagging_freq': 2, 'min_child_samples': 23}. Best is trial 42 with value: 0.993006993006993.
Trial 72 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5220750908086181, 'bagging_freq': 2, 'min_child_samples': 30}. Best is trial 42 with value: 0.993006993006993.
Trial 73 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.40115406878048854, 'bagging_freq': 1, 'min_child_samples': 16}. Best is trial 42 with value: 0.993006993006993.
Trial 74 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.43691691631054713, 'bagging_freq': 3, 'min_child_samples': 35}. Best is trial 42 with value: 0.993006993006993.
Trial 75 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.48622231755155987, 'bagging_freq': 3, 'min_child_samples': 40}. Best is trial 42 with value: 0.993006993006993.
Trial 76 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.46682739426762015, 'bagging_freq': 2, 'min_child_samples': 26}. Best is trial 42 with value: 0.993006993006993.
Trial 77 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4277813098685113, 'bagging_freq': 7, 'min_child_samples': 46}. Best is trial 42 with value: 0.993006993006993.
Trial 78 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4518027054180081, 'bagging_freq': 2, 'min_child_samples': 36}. Best is trial 42 with value: 0.993006993006993.
Trial 79 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5127408685379594, 'bagging_freq': 3, 'min_child_samples': 19}. Best is trial 42 with value: 0.993006993006993.
Trial 80 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.56029734214216, 'bagging_freq': 4, 'min_child_samples': 29}. Best is trial 42 with value: 0.993006993006993.
Trial 81 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.418303164753634, 'bagging_freq': 3, 'min_child_samples': 33}. Best is trial 42 with value: 0.993006993006993.
Trial 82 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.41796418621164916, 'bagging_freq': 3, 'min_child_samples': 28}. Best is trial 42 with value: 0.993006993006993.
Trial 83 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.4338438683411831, 'bagging_freq': 3, 'min_child_samples': 24}. Best is trial 42 with value: 0.993006993006993.
Trial 84 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.44079917351184045, 'bagging_freq': 3, 'min_child_samples': 24}. Best is trial 42 with value: 0.993006993006993.
Trial 85 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.464224644963808, 'bagging_freq': 2, 'min_child_samples': 21}. Best is trial 42 with value: 0.993006993006993.
Trial 86 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.4110866130687822, 'bagging_freq': 4, 'min_child_samples': 43}. Best is trial 42 with value: 0.993006993006993.
Trial 87 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.4975376670457469, 'bagging_freq': 3, 'min_child_samples': 12}. Best is trial 42 with value: 0.993006993006993.
Trial 88 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.43538035630532607, 'bagging_freq': 6, 'min_child_samples': 37}. Best is trial 42 with value: 0.993006993006993.
Trial 89 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.73417196754949, 'bagging_freq': 1, 'min_child_samples': 32}. Best is trial 42 with value: 0.993006993006993.
Trial 90 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4798030122389165, 'bagging_freq': 3, 'min_child_samples': 16}. Best is trial 42 with value: 0.993006993006993.
Trial 91 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.4643586819094848, 'bagging_freq': 2, 'min_child_samples': 22}. Best is trial 42 with value: 0.993006993006993.
Trial 92 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4540525100116518, 'bagging_freq': 2, 'min_child_samples': 19}. Best is trial 42 with value: 0.993006993006993.
Trial 93 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.40079217301390413, 'bagging_freq': 2, 'min_child_samples': 39}. Best is trial 42 with value: 0.993006993006993.
Trial 94 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.43263196120328473, 'bagging_freq': 3, 'min_child_samples': 13}. Best is trial 42 with value: 0.993006993006993.
Trial 95 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6139074856958611, 'bagging_freq': 2, 'min_child_samples': 21}. Best is trial 42 with value: 0.993006993006993.
Trial 96 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4450292577675094, 'bagging_freq': 5, 'min_child_samples': 26}. Best is trial 42 with value: 0.993006993006993.
Trial 97 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.461922038271511, 'bagging_freq': 1, 'min_child_samples': 42}. Best is trial 42 with value: 0.993006993006993.
Trial 98 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.4115962361373064, 'bagging_freq': 3, 'min_child_samples': 94}. Best is trial 42 with value: 0.993006993006993.
Trial 99 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4824227366890866, 'bagging_freq': 4, 'min_child_samples': 51}. Best is trial 42 with value: 0.993006993006993.
第二种场景：让 Optuna 利用已评估的超参数
在此场景中，我们假设您有一些开箱即用的超参数组合，并且您已经对其进行了评估，但结果并不理想，因此您正在考虑使用 Optuna。
Optuna 提供了 optuna.study.Study.add_trial()，它允许您将这些结果注册到 Optuna，然后 Optuna 将在采样超参数时将它们考虑在内。
在本节中，objective 与第一种场景相同。
study = optuna.create_study(direction="maximize", pruner=optuna.pruners.MedianPruner())
study.add_trial(
optuna.trial.create_trial(
params={
"bagging_fraction": 1.0,
"bagging_freq": 0,
"min_child_samples": 20,
},
distributions={
"bagging_fraction": optuna.distributions.FloatDistribution(0.4, 1.0 + 1e-12),
"bagging_freq": optuna.distributions.IntDistribution(0, 7),
"min_child_samples": optuna.distributions.IntDistribution(5, 100),
},
value=0.94,
)
)
study.add_trial(
optuna.trial.create_trial(
params={
"bagging_fraction": 0.75,
"bagging_freq": 5,
"min_child_samples": 20,
},
distributions={
"bagging_fraction": optuna.distributions.FloatDistribution(0.4, 1.0 + 1e-12),
"bagging_freq": optuna.distributions.IntDistribution(0, 7),
"min_child_samples": optuna.distributions.IntDistribution(5, 100),
},
value=0.95,
)
)
study.optimize(objective, n_trials=100, timeout=600)
A new study created in memory with name: no-name-16de5d9b-8b7a-4ecc-a4b0-80337327e76c
Trial 2 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.6934887688419089, 'bagging_freq': 2, 'min_child_samples': 69}. Best is trial 2 with value: 0.951048951048951.
Trial 3 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.8762178737256443, 'bagging_freq': 5, 'min_child_samples': 40}. Best is trial 2 with value: 0.951048951048951.
Trial 4 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.4042165302994373, 'bagging_freq': 2, 'min_child_samples': 33}. Best is trial 2 with value: 0.951048951048951.
Trial 5 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6588929281128468, 'bagging_freq': 1, 'min_child_samples': 87}. Best is trial 5 with value: 0.9790209790209791.
Trial 6 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.792302297375862, 'bagging_freq': 2, 'min_child_samples': 69}. Best is trial 5 with value: 0.9790209790209791.
Trial 7 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4986313094172332, 'bagging_freq': 2, 'min_child_samples': 87}. Best is trial 5 with value: 0.9790209790209791.
Trial 8 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.6985825644614032, 'bagging_freq': 0, 'min_child_samples': 89}. Best is trial 5 with value: 0.9790209790209791.
Trial 9 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.5619889268281065, 'bagging_freq': 5, 'min_child_samples': 95}. Best is trial 5 with value: 0.9790209790209791.
Trial 10 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5947890537709518, 'bagging_freq': 3, 'min_child_samples': 63}. Best is trial 5 with value: 0.9790209790209791.
Trial 11 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4426601198423865, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 5 with value: 0.9790209790209791.
Trial 12 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5450251077193055, 'bagging_freq': 3, 'min_child_samples': 80}. Best is trial 5 with value: 0.9790209790209791.
Trial 13 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.4884044562388388, 'bagging_freq': 7, 'min_child_samples': 98}. Best is trial 5 with value: 0.9790209790209791.
Trial 14 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.6145265980774639, 'bagging_freq': 1, 'min_child_samples': 54}. Best is trial 5 with value: 0.9790209790209791.
Trial 15 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6317626422186221, 'bagging_freq': 4, 'min_child_samples': 79}. Best is trial 5 with value: 0.9790209790209791.
Trial 16 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.49468664093578124, 'bagging_freq': 1, 'min_child_samples': 100}. Best is trial 5 with value: 0.9790209790209791.
Trial 17 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8272753301519974, 'bagging_freq': 0, 'min_child_samples': 5}. Best is trial 5 with value: 0.9790209790209791.
Trial 18 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6435171813011119, 'bagging_freq': 4, 'min_child_samples': 55}. Best is trial 5 with value: 0.9790209790209791.
Trial 19 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5124685139014498, 'bagging_freq': 7, 'min_child_samples': 74}. Best is trial 5 with value: 0.9790209790209791.
Trial 20 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8909140583045769, 'bagging_freq': 2, 'min_child_samples': 86}. Best is trial 5 with value: 0.9790209790209791.
Trial 21 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4054421823764317, 'bagging_freq': 1, 'min_child_samples': 86}. Best is trial 5 with value: 0.9790209790209791.
Trial 22 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.45414151365030586, 'bagging_freq': 1, 'min_child_samples': 87}. Best is trial 5 with value: 0.9790209790209791.
Trial 23 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.4568071619389841, 'bagging_freq': 1, 'min_child_samples': 64}. Best is trial 5 with value: 0.9790209790209791.
Trial 24 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5429492356296662, 'bagging_freq': 3, 'min_child_samples': 76}. Best is trial 5 with value: 0.9790209790209791.
Trial 25 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.44713157950029664, 'bagging_freq': 2, 'min_child_samples': 92}. Best is trial 25 with value: 0.993006993006993.
Trial 26 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5833317572193473, 'bagging_freq': 2, 'min_child_samples': 92}. Best is trial 25 with value: 0.993006993006993.
Trial 27 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6654483152158641, 'bagging_freq': 3, 'min_child_samples': 45}. Best is trial 25 with value: 0.993006993006993.
Trial 28 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7444695366562302, 'bagging_freq': 0, 'min_child_samples': 100}. Best is trial 25 with value: 0.993006993006993.
Trial 29 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.9752356303055716, 'bagging_freq': 4, 'min_child_samples': 80}. Best is trial 25 with value: 0.993006993006993.
Trial 30 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5310303638208074, 'bagging_freq': 0, 'min_child_samples': 61}. Best is trial 25 with value: 0.993006993006993.
Trial 31 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.5186637635747653, 'bagging_freq': 0, 'min_child_samples': 61}. Best is trial 25 with value: 0.993006993006993.
Trial 32 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4633895073961637, 'bagging_freq': 0, 'min_child_samples': 73}. Best is trial 25 with value: 0.993006993006993.
Trial 33 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5626853389722344, 'bagging_freq': 2, 'min_child_samples': 24}. Best is trial 25 with value: 0.993006993006993.
Trial 34 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.42760177427388774, 'bagging_freq': 1, 'min_child_samples': 46}. Best is trial 25 with value: 0.993006993006993.
Trial 35 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.49323697410991674, 'bagging_freq': 2, 'min_child_samples': 92}. Best is trial 25 with value: 0.993006993006993.
Trial 36 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7584306325197702, 'bagging_freq': 0, 'min_child_samples': 82}. Best is trial 25 with value: 0.993006993006993.
Trial 37 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6597371445127723, 'bagging_freq': 1, 'min_child_samples': 71}. Best is trial 25 with value: 0.993006993006993.
Trial 38 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5375685031860322, 'bagging_freq': 2, 'min_child_samples': 34}. Best is trial 25 with value: 0.993006993006993.
Trial 39 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7153456014833542, 'bagging_freq': 0, 'min_child_samples': 42}. Best is trial 25 with value: 0.993006993006993.
Trial 40 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5995943807882254, 'bagging_freq': 6, 'min_child_samples': 32}. Best is trial 25 with value: 0.993006993006993.
Trial 41 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5231900466661429, 'bagging_freq': 2, 'min_child_samples': 35}. Best is trial 25 with value: 0.993006993006993.
Trial 42 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.475413511290781, 'bagging_freq': 3, 'min_child_samples': 16}. Best is trial 25 with value: 0.993006993006993.
Trial 43 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.41472501903607906, 'bagging_freq': 2, 'min_child_samples': 95}. Best is trial 25 with value: 0.993006993006993.
Trial 44 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5784339945881604, 'bagging_freq': 2, 'min_child_samples': 67}. Best is trial 25 with value: 0.993006993006993.
Trial 45 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5702975373019404, 'bagging_freq': 2, 'min_child_samples': 50}. Best is trial 25 with value: 0.993006993006993.
Trial 46 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.5680025921725275, 'bagging_freq': 3, 'min_child_samples': 50}. Best is trial 25 with value: 0.993006993006993.
Trial 47 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.529532061050215, 'bagging_freq': 4, 'min_child_samples': 65}. Best is trial 25 with value: 0.993006993006993.
Trial 48 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.543516805731031, 'bagging_freq': 5, 'min_child_samples': 60}. Best is trial 25 with value: 0.993006993006993.
Trial 49 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.43425515941896153, 'bagging_freq': 4, 'min_child_samples': 67}. Best is trial 25 with value: 0.993006993006993.
Trial 50 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.612783424241059, 'bagging_freq': 6, 'min_child_samples': 57}. Best is trial 25 with value: 0.993006993006993.
Trial 51 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5320131906898412, 'bagging_freq': 3, 'min_child_samples': 67}. Best is trial 25 with value: 0.993006993006993.
Trial 52 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5841139352782472, 'bagging_freq': 4, 'min_child_samples': 38}. Best is trial 25 with value: 0.993006993006993.
Trial 53 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5050704564962915, 'bagging_freq': 2, 'min_child_samples': 50}. Best is trial 25 with value: 0.993006993006993.
Trial 54 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6283780073210466, 'bagging_freq': 5, 'min_child_samples': 28}. Best is trial 25 with value: 0.993006993006993.
Trial 55 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.47579738125446025, 'bagging_freq': 3, 'min_child_samples': 17}. Best is trial 25 with value: 0.993006993006993.
Trial 56 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5601646892279403, 'bagging_freq': 2, 'min_child_samples': 58}. Best is trial 25 with value: 0.993006993006993.
Trial 57 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.6884533962815127, 'bagging_freq': 1, 'min_child_samples': 65}. Best is trial 25 with value: 0.993006993006993.
Trial 58 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5727188808509586, 'bagging_freq': 4, 'min_child_samples': 54}. Best is trial 25 with value: 0.993006993006993.
Trial 59 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.6013979342574135, 'bagging_freq': 3, 'min_child_samples': 48}. Best is trial 25 with value: 0.993006993006993.
Trial 60 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6101095223296011, 'bagging_freq': 3, 'min_child_samples': 42}. Best is trial 25 with value: 0.993006993006993.
Trial 61 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5463490451110874, 'bagging_freq': 3, 'min_child_samples': 45}. Best is trial 25 with value: 0.993006993006993.
Trial 62 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5920989178876361, 'bagging_freq': 2, 'min_child_samples': 51}. Best is trial 25 with value: 0.993006993006993.
Trial 63 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5078546074508539, 'bagging_freq': 1, 'min_child_samples': 47}. Best is trial 25 with value: 0.993006993006993.
Trial 64 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.6414328949883488, 'bagging_freq': 4, 'min_child_samples': 60}. Best is trial 25 with value: 0.993006993006993.
Trial 65 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5281422158309976, 'bagging_freq': 2, 'min_child_samples': 70}. Best is trial 25 with value: 0.993006993006993.
Trial 66 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.47786651130597463, 'bagging_freq': 2, 'min_child_samples': 76}. Best is trial 25 with value: 0.993006993006993.
Trial 67 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.6796886130565734, 'bagging_freq': 3, 'min_child_samples': 38}. Best is trial 25 with value: 0.993006993006993.
Trial 68 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5461803907227384, 'bagging_freq': 1, 'min_child_samples': 63}. Best is trial 25 with value: 0.993006993006993.
Trial 69 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.44809540933403275, 'bagging_freq': 2, 'min_child_samples': 55}. Best is trial 25 with value: 0.993006993006993.
Trial 70 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.6220817403067695, 'bagging_freq': 3, 'min_child_samples': 30}. Best is trial 25 with value: 0.993006993006993.
Trial 71 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5778928688491629, 'bagging_freq': 4, 'min_child_samples': 39}. Best is trial 25 with value: 0.993006993006993.
Trial 72 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6026972865626621, 'bagging_freq': 4, 'min_child_samples': 48}. Best is trial 25 with value: 0.993006993006993.
Trial 73 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.5816763755021318, 'bagging_freq': 5, 'min_child_samples': 26}. Best is trial 25 with value: 0.993006993006993.
Trial 74 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.5591226122237394, 'bagging_freq': 4, 'min_child_samples': 37}. Best is trial 25 with value: 0.993006993006993.
Trial 75 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6475790088576054, 'bagging_freq': 4, 'min_child_samples': 41}. Best is trial 25 with value: 0.993006993006993.
Trial 76 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.4890389824533843, 'bagging_freq': 3, 'min_child_samples': 35}. Best is trial 25 with value: 0.993006993006993.
Trial 77 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.5158779119484311, 'bagging_freq': 5, 'min_child_samples': 43}. Best is trial 25 with value: 0.993006993006993.
Trial 78 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5390854783753989, 'bagging_freq': 2, 'min_child_samples': 58}. Best is trial 25 with value: 0.993006993006993.
Trial 79 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5861666131546844, 'bagging_freq': 1, 'min_child_samples': 23}. Best is trial 25 with value: 0.993006993006993.
Trial 80 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.49787321935140555, 'bagging_freq': 3, 'min_child_samples': 33}. Best is trial 25 with value: 0.993006993006993.
Trial 81 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4977564346293602, 'bagging_freq': 3, 'min_child_samples': 83}. Best is trial 25 with value: 0.993006993006993.
Trial 82 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.46135594342113984, 'bagging_freq': 3, 'min_child_samples': 33}. Best is trial 25 with value: 0.993006993006993.
Trial 83 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.521302722237065, 'bagging_freq': 4, 'min_child_samples': 30}. Best is trial 25 with value: 0.993006993006993.
Trial 84 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5549254590178325, 'bagging_freq': 2, 'min_child_samples': 36}. Best is trial 25 with value: 0.993006993006993.
Trial 85 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4316647842566365, 'bagging_freq': 3, 'min_child_samples': 52}. Best is trial 25 with value: 0.993006993006993.
Trial 86 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.47945812821378514, 'bagging_freq': 4, 'min_child_samples': 48}. Best is trial 25 with value: 0.993006993006993.
Trial 87 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5298254616040433, 'bagging_freq': 2, 'min_child_samples': 67}. Best is trial 25 with value: 0.993006993006993.
Trial 88 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5948567785172547, 'bagging_freq': 1, 'min_child_samples': 62}. Best is trial 25 with value: 0.993006993006993.
Trial 89 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.4013173712389284, 'bagging_freq': 2, 'min_child_samples': 44}. Best is trial 25 with value: 0.993006993006993.
Trial 90 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.40946393817479987, 'bagging_freq': 2, 'min_child_samples': 44}. Best is trial 25 with value: 0.993006993006993.
Trial 91 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.4158079642530892, 'bagging_freq': 2, 'min_child_samples': 40}. Best is trial 25 with value: 0.993006993006993.
Trial 92 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5057504261927879, 'bagging_freq': 2, 'min_child_samples': 53}. Best is trial 25 with value: 0.993006993006993.
Trial 93 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4470916359862919, 'bagging_freq': 3, 'min_child_samples': 56}. Best is trial 25 with value: 0.993006993006993.
Trial 94 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.40185057688998704, 'bagging_freq': 3, 'min_child_samples': 32}. Best is trial 25 with value: 0.993006993006993.
Trial 95 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.4685680142021178, 'bagging_freq': 2, 'min_child_samples': 34}. Best is trial 25 with value: 0.993006993006993.
Trial 96 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.42801552109819496, 'bagging_freq': 1, 'min_child_samples': 49}. Best is trial 25 with value: 0.993006993006993.
Trial 97 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.5743960092530968, 'bagging_freq': 0, 'min_child_samples': 46}. Best is trial 25 with value: 0.993006993006993.
Trial 98 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.8389728628789229, 'bagging_freq': 6, 'min_child_samples': 38}. Best is trial 25 with value: 0.993006993006993.
Trial 99 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5547292270233777, 'bagging_freq': 4, 'min_child_samples': 91}. Best is trial 25 with value: 0.993006993006993.
Trial 100 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5310543879852566, 'bagging_freq': 3, 'min_child_samples': 75}. Best is trial 25 with value: 0.993006993006993.
Trial 101 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.724785377976271, 'bagging_freq': 0, 'min_child_samples': 65}. Best is trial 25 with value: 0.993006993006993.
脚本总运行时间： (0 分钟 5.527 秒)