手动指定超参数

首先尝试一些特定的超参数集合是很自然的，例如初始学习率值和叶子数。此外，在让 Optuna 寻找更好的超参数集合之前，您可能已经尝试过这些集合。

Optuna 提供了两种 API 来支持这些情况

1. 传递这些超参数集合并让 Optuna 进行评估 - enqueue_trial()

2. 将这些集合的结果作为已完成的 Trial 添加 - add_trial()

场景一：让 Optuna 评估您的超参数

在此场景中，假设您有一些现成的超参数集合，但尚未评估它们，并决定使用 Optuna 来寻找更好的超参数集合。

Optuna 提供了 optuna.study.Study.enqueue_trial()，它允许您将这些超参数集合传递给 Optuna，然后 Optuna 将评估它们。

本节将引导您了解如何结合 LightGBM 使用此 API。

import lightgbm as lgb
import numpy as np
import sklearn.datasets
import sklearn.metrics
from sklearn.model_selection import train_test_split

import optuna

定义目标函数。

def objective(trial):
    data, target = sklearn.datasets.load_breast_cancer(return_X_y=True)
    train_x, valid_x, train_y, valid_y = train_test_split(data, target, test_size=0.25)
    dtrain = lgb.Dataset(train_x, label=train_y)
    dvalid = lgb.Dataset(valid_x, label=valid_y)

    param = {
        "objective": "binary",
        "metric": "auc",
        "verbosity": -1,
        "boosting_type": "gbdt",
        "bagging_fraction": min(trial.suggest_float("bagging_fraction", 0.4, 1.0 + 1e-12), 1),
        "bagging_freq": trial.suggest_int("bagging_freq", 0, 7),
        "min_child_samples": trial.suggest_int("min_child_samples", 5, 100),
    }

    gbm = lgb.train(param, dtrain, valid_sets=[dvalid])

    preds = gbm.predict(valid_x)
    pred_labels = np.rint(preds)
    accuracy = sklearn.metrics.accuracy_score(valid_y, pred_labels)
    return accuracy

然后，构建用于超参数优化的 Study。

study = optuna.create_study(direction="maximize", pruner=optuna.pruners.MedianPruner())

在这里，我们让 Optuna 评估一些具有较大 "bagging_fraq" 值和默认值的集合。

study.enqueue_trial(
    {
        "bagging_fraction": 1.0,
        "bagging_freq": 0,
        "min_child_samples": 20,
    }
)

study.enqueue_trial(
    {
        "bagging_fraction": 0.75,
        "bagging_freq": 5,
        "min_child_samples": 20,
    }
)

import logging
import sys

# Add stream handler of stdout to show the messages to see Optuna works expectedly.
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))
study.optimize(objective, n_trials=100, timeout=600)

Trial 0 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 1.0, 'bagging_freq': 0, 'min_child_samples': 20}. Best is trial 0 with value: 0.9440559440559441.
Trial 1 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.75, 'bagging_freq': 5, 'min_child_samples': 20}. Best is trial 1 with value: 0.972027972027972.
Trial 2 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.5159987780098035, 'bagging_freq': 7, 'min_child_samples': 53}. Best is trial 1 with value: 0.972027972027972.
Trial 3 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.5317298725341097, 'bagging_freq': 6, 'min_child_samples': 12}. Best is trial 3 with value: 0.993006993006993.
Trial 4 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.9673420400210446, 'bagging_freq': 6, 'min_child_samples': 93}. Best is trial 3 with value: 0.993006993006993.
Trial 5 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8547886650269991, 'bagging_freq': 0, 'min_child_samples': 5}. Best is trial 3 with value: 0.993006993006993.
Trial 6 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9858737768143784, 'bagging_freq': 4, 'min_child_samples': 98}. Best is trial 3 with value: 0.993006993006993.
Trial 7 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7426560667514106, 'bagging_freq': 2, 'min_child_samples': 94}. Best is trial 3 with value: 0.993006993006993.
Trial 8 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.48896203037298297, 'bagging_freq': 6, 'min_child_samples': 26}. Best is trial 3 with value: 0.993006993006993.
Trial 9 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6305486747666174, 'bagging_freq': 4, 'min_child_samples': 67}. Best is trial 3 with value: 0.993006993006993.
Trial 10 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.42513826531789173, 'bagging_freq': 2, 'min_child_samples': 40}. Best is trial 3 with value: 0.993006993006993.
Trial 11 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6080840131467563, 'bagging_freq': 4, 'min_child_samples': 74}. Best is trial 3 with value: 0.993006993006993.
Trial 12 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.6209678515141976, 'bagging_freq': 3, 'min_child_samples': 73}. Best is trial 3 with value: 0.993006993006993.
Trial 13 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.868064516830019, 'bagging_freq': 7, 'min_child_samples': 49}. Best is trial 3 with value: 0.993006993006993.
Trial 14 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.838200426759286, 'bagging_freq': 5, 'min_child_samples': 83}. Best is trial 3 with value: 0.993006993006993.
Trial 15 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5299362519951025, 'bagging_freq': 5, 'min_child_samples': 6}. Best is trial 3 with value: 0.993006993006993.
Trial 16 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.40168611202665294, 'bagging_freq': 3, 'min_child_samples': 57}. Best is trial 3 with value: 0.993006993006993.
Trial 17 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.6536747254407762, 'bagging_freq': 6, 'min_child_samples': 36}. Best is trial 3 with value: 0.993006993006993.
Trial 18 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.9355418845990526, 'bagging_freq': 2, 'min_child_samples': 63}. Best is trial 3 with value: 0.993006993006993.
Trial 19 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7905053255455724, 'bagging_freq': 4, 'min_child_samples': 82}. Best is trial 3 with value: 0.993006993006993.
Trial 20 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.5688923796455473, 'bagging_freq': 1, 'min_child_samples': 41}. Best is trial 3 with value: 0.993006993006993.
Trial 21 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8911951627242634, 'bagging_freq': 5, 'min_child_samples': 85}. Best is trial 3 with value: 0.993006993006993.
Trial 22 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8034512671239742, 'bagging_freq': 5, 'min_child_samples': 99}. Best is trial 3 with value: 0.993006993006993.
Trial 23 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.6871741388843468, 'bagging_freq': 6, 'min_child_samples': 85}. Best is trial 3 with value: 0.993006993006993.
Trial 24 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.9062028968418154, 'bagging_freq': 7, 'min_child_samples': 100}. Best is trial 3 with value: 0.993006993006993.
Trial 25 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9207700965959534, 'bagging_freq': 7, 'min_child_samples': 100}. Best is trial 3 with value: 0.993006993006993.
Trial 26 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.9817536283419261, 'bagging_freq': 7, 'min_child_samples': 92}. Best is trial 3 with value: 0.993006993006993.
Trial 27 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.9274618350499295, 'bagging_freq': 6, 'min_child_samples': 75}. Best is trial 3 with value: 0.993006993006993.
Trial 28 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.46078904649140484, 'bagging_freq': 7, 'min_child_samples': 32}. Best is trial 3 with value: 0.993006993006993.
Trial 29 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.9898104339813817, 'bagging_freq': 3, 'min_child_samples': 16}. Best is trial 3 with value: 0.993006993006993.
Trial 30 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5712873566032535, 'bagging_freq': 6, 'min_child_samples': 12}. Best is trial 3 with value: 0.993006993006993.
Trial 31 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8437789968174626, 'bagging_freq': 5, 'min_child_samples': 90}. Best is trial 3 with value: 0.993006993006993.
Trial 32 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.8145069309128363, 'bagging_freq': 4, 'min_child_samples': 79}. Best is trial 3 with value: 0.993006993006993.
Trial 33 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7320079152459606, 'bagging_freq': 5, 'min_child_samples': 97}. Best is trial 3 with value: 0.993006993006993.
Trial 34 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.896565110489754, 'bagging_freq': 6, 'min_child_samples': 90}. Best is trial 3 with value: 0.993006993006993.
Trial 35 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.962322810264018, 'bagging_freq': 7, 'min_child_samples': 87}. Best is trial 3 with value: 0.993006993006993.
Trial 36 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.7730095977945458, 'bagging_freq': 4, 'min_child_samples': 48}. Best is trial 3 with value: 0.993006993006993.
Trial 37 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.8309434234274126, 'bagging_freq': 6, 'min_child_samples': 24}. Best is trial 3 with value: 0.993006993006993.
Trial 38 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.9552210892530463, 'bagging_freq': 5, 'min_child_samples': 95}. Best is trial 3 with value: 0.993006993006993.
Trial 39 finished with value: 0.9230769230769231 and parameters: {'bagging_fraction': 0.9947205266439894, 'bagging_freq': 5, 'min_child_samples': 68}. Best is trial 3 with value: 0.993006993006993.
Trial 40 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.6972663202160636, 'bagging_freq': 7, 'min_child_samples': 79}. Best is trial 3 with value: 0.993006993006993.
Trial 41 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.9022652407171698, 'bagging_freq': 7, 'min_child_samples': 100}. Best is trial 3 with value: 0.993006993006993.
Trial 42 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.9202890814444938, 'bagging_freq': 7, 'min_child_samples': 100}. Best is trial 3 with value: 0.993006993006993.
Trial 43 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.8738580140155676, 'bagging_freq': 6, 'min_child_samples': 95}. Best is trial 3 with value: 0.993006993006993.
Trial 44 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9495098539260436, 'bagging_freq': 7, 'min_child_samples': 88}. Best is trial 3 with value: 0.993006993006993.
Trial 45 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8587129699666216, 'bagging_freq': 4, 'min_child_samples': 93}. Best is trial 3 with value: 0.993006993006993.
Trial 46 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.9152034939723995, 'bagging_freq': 6, 'min_child_samples': 80}. Best is trial 3 with value: 0.993006993006993.
Trial 47 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.7664516273696164, 'bagging_freq': 0, 'min_child_samples': 96}. Best is trial 3 with value: 0.993006993006993.
Trial 48 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8801990702316199, 'bagging_freq': 6, 'min_child_samples': 57}. Best is trial 3 with value: 0.993006993006993.
Trial 49 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.7310876755113012, 'bagging_freq': 5, 'min_child_samples': 70}. Best is trial 3 with value: 0.993006993006993.
Trial 50 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.9407310961731874, 'bagging_freq': 7, 'min_child_samples': 63}. Best is trial 3 with value: 0.993006993006993.
Trial 51 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8291741186553201, 'bagging_freq': 6, 'min_child_samples': 93}. Best is trial 3 with value: 0.993006993006993.
Trial 52 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.870148635330225, 'bagging_freq': 6, 'min_child_samples': 96}. Best is trial 3 with value: 0.993006993006993.
Trial 53 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.909726454275088, 'bagging_freq': 6, 'min_child_samples': 84}. Best is trial 3 with value: 0.993006993006993.
Trial 54 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.9636683421218495, 'bagging_freq': 5, 'min_child_samples': 100}. Best is trial 3 with value: 0.993006993006993.
Trial 55 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.8509083661555373, 'bagging_freq': 7, 'min_child_samples': 89}. Best is trial 3 with value: 0.993006993006993.
Trial 56 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8830577328768927, 'bagging_freq': 4, 'min_child_samples': 92}. Best is trial 3 with value: 0.993006993006993.
Trial 57 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.46627979394788177, 'bagging_freq': 7, 'min_child_samples': 97}. Best is trial 3 with value: 0.993006993006993.
Trial 58 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.5364513739995326, 'bagging_freq': 3, 'min_child_samples': 30}. Best is trial 3 with value: 0.993006993006993.
Trial 59 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.5256898089510952, 'bagging_freq': 3, 'min_child_samples': 9}. Best is trial 3 with value: 0.993006993006993.
Trial 60 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.5599024011044265, 'bagging_freq': 2, 'min_child_samples': 29}. Best is trial 3 with value: 0.993006993006993.
Trial 61 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4971164865417207, 'bagging_freq': 4, 'min_child_samples': 15}. Best is trial 3 with value: 0.993006993006993.
Trial 62 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.5941787388671858, 'bagging_freq': 3, 'min_child_samples': 20}. Best is trial 3 with value: 0.993006993006993.
Trial 63 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.9341108685311409, 'bagging_freq': 3, 'min_child_samples': 43}. Best is trial 3 with value: 0.993006993006993.
Trial 64 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.637380496018415, 'bagging_freq': 6, 'min_child_samples': 21}. Best is trial 3 with value: 0.993006993006993.
Trial 65 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.9781054928448205, 'bagging_freq': 5, 'min_child_samples': 35}. Best is trial 3 with value: 0.993006993006993.
Trial 66 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5452420721066868, 'bagging_freq': 2, 'min_child_samples': 76}. Best is trial 3 with value: 0.993006993006993.
Trial 67 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.6656739370662127, 'bagging_freq': 4, 'min_child_samples': 83}. Best is trial 3 with value: 0.993006993006993.
Trial 68 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5007020332573816, 'bagging_freq': 6, 'min_child_samples': 7}. Best is trial 3 with value: 0.993006993006993.
Trial 69 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.44213820261942527, 'bagging_freq': 5, 'min_child_samples': 86}. Best is trial 3 with value: 0.993006993006993.
Trial 70 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.5945326411419293, 'bagging_freq': 3, 'min_child_samples': 91}. Best is trial 3 with value: 0.993006993006993.
Trial 71 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.9433467986872315, 'bagging_freq': 7, 'min_child_samples': 88}. Best is trial 3 with value: 0.993006993006993.
Trial 72 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.9988580385384007, 'bagging_freq': 7, 'min_child_samples': 97}. Best is trial 3 with value: 0.993006993006993.
Trial 73 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.9771237365469293, 'bagging_freq': 7, 'min_child_samples': 95}. Best is trial 3 with value: 0.993006993006993.
Trial 74 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.9273735059949472, 'bagging_freq': 7, 'min_child_samples': 98}. Best is trial 3 with value: 0.993006993006993.
Trial 75 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9460968548834122, 'bagging_freq': 6, 'min_child_samples': 88}. Best is trial 3 with value: 0.993006993006993.
Trial 76 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.898085216735468, 'bagging_freq': 7, 'min_child_samples': 27}. Best is trial 3 with value: 0.993006993006993.
Trial 77 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8051460442089043, 'bagging_freq': 6, 'min_child_samples': 93}. Best is trial 3 with value: 0.993006993006993.
Trial 78 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.832431151037503, 'bagging_freq': 4, 'min_child_samples': 100}. Best is trial 3 with value: 0.993006993006993.
Trial 79 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9607127784369162, 'bagging_freq': 7, 'min_child_samples': 46}. Best is trial 3 with value: 0.993006993006993.
Trial 80 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.40601503282009377, 'bagging_freq': 7, 'min_child_samples': 37}. Best is trial 3 with value: 0.993006993006993.
Trial 81 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.40523743810558993, 'bagging_freq': 7, 'min_child_samples': 38}. Best is trial 3 with value: 0.993006993006993.
Trial 82 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.47257086255799996, 'bagging_freq': 7, 'min_child_samples': 33}. Best is trial 3 with value: 0.993006993006993.
Trial 83 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8692454473096441, 'bagging_freq': 7, 'min_child_samples': 94}. Best is trial 3 with value: 0.993006993006993.
Trial 84 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.44873916043259204, 'bagging_freq': 6, 'min_child_samples': 55}. Best is trial 3 with value: 0.993006993006993.
Trial 85 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.8871391946943584, 'bagging_freq': 6, 'min_child_samples': 42}. Best is trial 3 with value: 0.993006993006993.
Trial 86 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9162890753843312, 'bagging_freq': 7, 'min_child_samples': 15}. Best is trial 3 with value: 0.993006993006993.
Trial 87 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.510411831702077, 'bagging_freq': 7, 'min_child_samples': 52}. Best is trial 3 with value: 0.993006993006993.
Trial 88 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.43337151612271435, 'bagging_freq': 6, 'min_child_samples': 61}. Best is trial 3 with value: 0.993006993006993.
Trial 89 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.41847432109641336, 'bagging_freq': 5, 'min_child_samples': 81}. Best is trial 3 with value: 0.993006993006993.
Trial 90 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4783926920571125, 'bagging_freq': 7, 'min_child_samples': 90}. Best is trial 3 with value: 0.993006993006993.
Trial 91 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.9509643995520821, 'bagging_freq': 7, 'min_child_samples': 87}. Best is trial 3 with value: 0.993006993006993.
Trial 92 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.9749074430920344, 'bagging_freq': 7, 'min_child_samples': 77}. Best is trial 3 with value: 0.993006993006993.
Trial 93 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.9401183626147279, 'bagging_freq': 7, 'min_child_samples': 95}. Best is trial 3 with value: 0.993006993006993.
Trial 94 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.7219417377300623, 'bagging_freq': 6, 'min_child_samples': 89}. Best is trial 3 with value: 0.993006993006993.
Trial 95 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.9047613480192854, 'bagging_freq': 7, 'min_child_samples': 72}. Best is trial 3 with value: 0.993006993006993.
Trial 96 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.9861614254355894, 'bagging_freq': 2, 'min_child_samples': 98}. Best is trial 3 with value: 0.993006993006993.
Trial 97 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.8580252740024845, 'bagging_freq': 3, 'min_child_samples': 85}. Best is trial 3 with value: 0.993006993006993.
Trial 98 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9209755471693623, 'bagging_freq': 5, 'min_child_samples': 22}. Best is trial 3 with value: 0.993006993006993.
Trial 99 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8771133848216582, 'bagging_freq': 7, 'min_child_samples': 91}. Best is trial 3 with value: 0.993006993006993.

场景二：让 Optuna 利用已评估的超参数

在此场景中，假设您有一些现成的超参数集合并且您已经评估过它们，但结果不理想，因此您正在考虑使用 Optuna。

Optuna 提供了 optuna.study.Study.add_trial()，它允许您将这些结果注册到 Optuna，然后 Optuna 将在采样超参数时考虑这些结果。

在本节中，objective 与第一个场景相同。

study = optuna.create_study(direction="maximize", pruner=optuna.pruners.MedianPruner())
study.add_trial(
    optuna.trial.create_trial(
        params={
            "bagging_fraction": 1.0,
            "bagging_freq": 0,
            "min_child_samples": 20,
        },
        distributions={
            "bagging_fraction": optuna.distributions.FloatDistribution(0.4, 1.0 + 1e-12),
            "bagging_freq": optuna.distributions.IntDistribution(0, 7),
            "min_child_samples": optuna.distributions.IntDistribution(5, 100),
        },
        value=0.94,
    )
)
study.add_trial(
    optuna.trial.create_trial(
        params={
            "bagging_fraction": 0.75,
            "bagging_freq": 5,
            "min_child_samples": 20,
        },
        distributions={
            "bagging_fraction": optuna.distributions.FloatDistribution(0.4, 1.0 + 1e-12),
            "bagging_freq": optuna.distributions.IntDistribution(0, 7),
            "min_child_samples": optuna.distributions.IntDistribution(5, 100),
        },
        value=0.95,
    )
)
study.optimize(objective, n_trials=100, timeout=600)

A new study created in memory with name: no-name-c324e1c6-ff28-4de1-aee5-25fdde9daaa1
Trial 2 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.9966229657175428, 'bagging_freq': 5, 'min_child_samples': 75}. Best is trial 2 with value: 0.965034965034965.
Trial 3 finished with value: 0.9370629370629371 and parameters: {'bagging_fraction': 0.5071022101241273, 'bagging_freq': 1, 'min_child_samples': 56}. Best is trial 2 with value: 0.965034965034965.
Trial 4 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.41271570887678727, 'bagging_freq': 2, 'min_child_samples': 41}. Best is trial 2 with value: 0.965034965034965.
Trial 5 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.7196596054973249, 'bagging_freq': 7, 'min_child_samples': 100}. Best is trial 2 with value: 0.965034965034965.
Trial 6 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6785596737887101, 'bagging_freq': 5, 'min_child_samples': 54}. Best is trial 6 with value: 0.9790209790209791.
Trial 7 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.7138164411200874, 'bagging_freq': 7, 'min_child_samples': 52}. Best is trial 7 with value: 0.986013986013986.
Trial 8 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9292423847563633, 'bagging_freq': 5, 'min_child_samples': 33}. Best is trial 7 with value: 0.986013986013986.
Trial 9 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.5602302184780275, 'bagging_freq': 7, 'min_child_samples': 89}. Best is trial 7 with value: 0.986013986013986.
Trial 10 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.829161230036173, 'bagging_freq': 3, 'min_child_samples': 5}. Best is trial 7 with value: 0.986013986013986.
Trial 11 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6320895747001515, 'bagging_freq': 6, 'min_child_samples': 59}. Best is trial 7 with value: 0.986013986013986.
Trial 12 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.8167450031381258, 'bagging_freq': 4, 'min_child_samples': 71}. Best is trial 7 with value: 0.986013986013986.
Trial 13 finished with value: 0.9300699300699301 and parameters: {'bagging_fraction': 0.6503428421963711, 'bagging_freq': 7, 'min_child_samples': 40}. Best is trial 7 with value: 0.986013986013986.
Trial 14 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7883390411178776, 'bagging_freq': 6, 'min_child_samples': 67}. Best is trial 7 with value: 0.986013986013986.
Trial 15 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.6470736453216693, 'bagging_freq': 4, 'min_child_samples': 48}. Best is trial 7 with value: 0.986013986013986.
Trial 16 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5440420173718656, 'bagging_freq': 6, 'min_child_samples': 80}. Best is trial 7 with value: 0.986013986013986.
Trial 17 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.8486762995830863, 'bagging_freq': 3, 'min_child_samples': 61}. Best is trial 7 with value: 0.986013986013986.
Trial 18 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6930166966962676, 'bagging_freq': 6, 'min_child_samples': 28}. Best is trial 7 with value: 0.986013986013986.
Trial 19 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.5847891495970251, 'bagging_freq': 5, 'min_child_samples': 48}. Best is trial 7 with value: 0.986013986013986.
Trial 20 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.4502571778594102, 'bagging_freq': 7, 'min_child_samples': 87}. Best is trial 7 with value: 0.986013986013986.
Trial 21 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4003131239009808, 'bagging_freq': 7, 'min_child_samples': 86}. Best is trial 7 with value: 0.986013986013986.
Trial 22 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.45872562412169265, 'bagging_freq': 6, 'min_child_samples': 98}. Best is trial 7 with value: 0.986013986013986.
Trial 23 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.5080958973790289, 'bagging_freq': 7, 'min_child_samples': 65}. Best is trial 7 with value: 0.986013986013986.
Trial 24 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6055454502899676, 'bagging_freq': 4, 'min_child_samples': 53}. Best is trial 7 with value: 0.986013986013986.
Trial 25 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.7136697800645577, 'bagging_freq': 6, 'min_child_samples': 79}. Best is trial 7 with value: 0.986013986013986.
Trial 26 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9012457958699476, 'bagging_freq': 7, 'min_child_samples': 89}. Best is trial 7 with value: 0.986013986013986.
Trial 27 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7606729982440685, 'bagging_freq': 5, 'min_child_samples': 41}. Best is trial 7 with value: 0.986013986013986.
Trial 28 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.4557853458212074, 'bagging_freq': 2, 'min_child_samples': 8}. Best is trial 7 with value: 0.986013986013986.
Trial 29 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.6635900124233576, 'bagging_freq': 0, 'min_child_samples': 20}. Best is trial 7 with value: 0.986013986013986.
Trial 30 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8729226643090574, 'bagging_freq': 6, 'min_child_samples': 47}. Best is trial 7 with value: 0.986013986013986.
Trial 31 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7009080490752115, 'bagging_freq': 6, 'min_child_samples': 27}. Best is trial 7 with value: 0.986013986013986.
Trial 32 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7473020423593607, 'bagging_freq': 7, 'min_child_samples': 16}. Best is trial 7 with value: 0.986013986013986.
Trial 33 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.6827378693792276, 'bagging_freq': 5, 'min_child_samples': 28}. Best is trial 7 with value: 0.986013986013986.
Trial 34 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.7798570061941559, 'bagging_freq': 5, 'min_child_samples': 35}. Best is trial 34 with value: 0.993006993006993.
Trial 35 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.7827709094280781, 'bagging_freq': 5, 'min_child_samples': 36}. Best is trial 34 with value: 0.993006993006993.
Trial 36 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.7512214333250757, 'bagging_freq': 4, 'min_child_samples': 29}. Best is trial 34 with value: 0.993006993006993.
Trial 37 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.919268409508447, 'bagging_freq': 4, 'min_child_samples': 21}. Best is trial 34 with value: 0.993006993006993.
Trial 38 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.9743550101630213, 'bagging_freq': 3, 'min_child_samples': 34}. Best is trial 34 with value: 0.993006993006993.
Trial 39 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6165171174624624, 'bagging_freq': 5, 'min_child_samples': 45}. Best is trial 34 with value: 0.993006993006993.
Trial 40 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.7255558309589566, 'bagging_freq': 2, 'min_child_samples': 16}. Best is trial 34 with value: 0.993006993006993.
Trial 41 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7861138441178044, 'bagging_freq': 4, 'min_child_samples': 26}. Best is trial 34 with value: 0.993006993006993.
Trial 42 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.7504321816658894, 'bagging_freq': 3, 'min_child_samples': 30}. Best is trial 34 with value: 0.993006993006993.
Trial 43 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.8180146073451998, 'bagging_freq': 5, 'min_child_samples': 23}. Best is trial 34 with value: 0.993006993006993.
Trial 44 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.7347101839473056, 'bagging_freq': 5, 'min_child_samples': 38}. Best is trial 34 with value: 0.993006993006993.
Trial 45 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6784488953229452, 'bagging_freq': 4, 'min_child_samples': 14}. Best is trial 34 with value: 0.993006993006993.
Trial 46 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.7750914691988077, 'bagging_freq': 7, 'min_child_samples': 34}. Best is trial 34 with value: 0.993006993006993.
Trial 47 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8082024575838743, 'bagging_freq': 4, 'min_child_samples': 43}. Best is trial 34 with value: 0.993006993006993.
Trial 48 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.8573620983559304, 'bagging_freq': 1, 'min_child_samples': 58}. Best is trial 34 with value: 0.993006993006993.
Trial 49 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.55201452359144, 'bagging_freq': 6, 'min_child_samples': 51}. Best is trial 34 with value: 0.993006993006993.
Trial 50 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.46266043953398994, 'bagging_freq': 7, 'min_child_samples': 30}. Best is trial 34 with value: 0.993006993006993.
Trial 51 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.8591979097698139, 'bagging_freq': 1, 'min_child_samples': 58}. Best is trial 34 with value: 0.993006993006993.
Trial 52 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.8844137605528387, 'bagging_freq': 0, 'min_child_samples': 73}. Best is trial 34 with value: 0.993006993006993.
Trial 53 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8424365620565997, 'bagging_freq': 1, 'min_child_samples': 62}. Best is trial 34 with value: 0.993006993006993.
Trial 54 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9331645266599627, 'bagging_freq': 2, 'min_child_samples': 24}. Best is trial 34 with value: 0.993006993006993.
Trial 55 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.682791462107042, 'bagging_freq': 3, 'min_child_samples': 96}. Best is trial 34 with value: 0.993006993006993.
Trial 56 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.8015420280804662, 'bagging_freq': 5, 'min_child_samples': 67}. Best is trial 34 with value: 0.993006993006993.
Trial 57 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8280290779931925, 'bagging_freq': 5, 'min_child_samples': 81}. Best is trial 34 with value: 0.993006993006993.
Trial 58 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.4317274744167428, 'bagging_freq': 4, 'min_child_samples': 32}. Best is trial 34 with value: 0.993006993006993.
Trial 59 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.7659874192085259, 'bagging_freq': 6, 'min_child_samples': 38}. Best is trial 34 with value: 0.993006993006993.
Trial 60 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5789815856329401, 'bagging_freq': 3, 'min_child_samples': 54}. Best is trial 34 with value: 0.993006993006993.
Trial 61 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.8548191440773367, 'bagging_freq': 1, 'min_child_samples': 58}. Best is trial 34 with value: 0.993006993006993.
Trial 62 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.5106117361285308, 'bagging_freq': 0, 'min_child_samples': 57}. Best is trial 34 with value: 0.993006993006993.
Trial 63 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.9756482942290551, 'bagging_freq': 1, 'min_child_samples': 50}. Best is trial 34 with value: 0.993006993006993.
Trial 64 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.71526321342115, 'bagging_freq': 1, 'min_child_samples': 70}. Best is trial 34 with value: 0.993006993006993.
Trial 65 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.893891881829698, 'bagging_freq': 2, 'min_child_samples': 77}. Best is trial 34 with value: 0.993006993006993.
Trial 66 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.742480766148404, 'bagging_freq': 7, 'min_child_samples': 64}. Best is trial 34 with value: 0.993006993006993.
Trial 67 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8629548688138796, 'bagging_freq': 1, 'min_child_samples': 92}. Best is trial 34 with value: 0.993006993006993.
Trial 68 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.6568583627892336, 'bagging_freq': 2, 'min_child_samples': 43}. Best is trial 34 with value: 0.993006993006993.
Trial 69 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8450677298771233, 'bagging_freq': 6, 'min_child_samples': 69}. Best is trial 34 with value: 0.993006993006993.
Trial 70 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.633349862318829, 'bagging_freq': 7, 'min_child_samples': 60}. Best is trial 34 with value: 0.993006993006993.
Trial 71 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.8649682213959036, 'bagging_freq': 1, 'min_child_samples': 57}. Best is trial 34 with value: 0.993006993006993.
Trial 72 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.9207614582768401, 'bagging_freq': 1, 'min_child_samples': 85}. Best is trial 34 with value: 0.993006993006993.
Trial 73 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.8044095925710554, 'bagging_freq': 0, 'min_child_samples': 55}. Best is trial 34 with value: 0.993006993006993.
Trial 74 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.9515350800305495, 'bagging_freq': 1, 'min_child_samples': 63}. Best is trial 34 with value: 0.993006993006993.
Trial 75 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.7060731687517339, 'bagging_freq': 0, 'min_child_samples': 59}. Best is trial 34 with value: 0.993006993006993.
Trial 76 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.7614757350126367, 'bagging_freq': 2, 'min_child_samples': 47}. Best is trial 34 with value: 0.993006993006993.
Trial 77 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.8338496680323342, 'bagging_freq': 4, 'min_child_samples': 74}. Best is trial 34 with value: 0.993006993006993.
Trial 78 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.8576091579102583, 'bagging_freq': 1, 'min_child_samples': 19}. Best is trial 34 with value: 0.993006993006993.
Trial 79 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.8811753491330904, 'bagging_freq': 6, 'min_child_samples': 50}. Best is trial 34 with value: 0.993006993006993.
Trial 80 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.5333114406431032, 'bagging_freq': 5, 'min_child_samples': 66}. Best is trial 34 with value: 0.993006993006993.
Trial 81 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.7058001228531242, 'bagging_freq': 0, 'min_child_samples': 58}. Best is trial 34 with value: 0.993006993006993.
Trial 82 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.724505011432927, 'bagging_freq': 0, 'min_child_samples': 53}. Best is trial 34 with value: 0.993006993006993.
Trial 83 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.6910688524762375, 'bagging_freq': 0, 'min_child_samples': 61}. Best is trial 34 with value: 0.993006993006993.
Trial 84 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6862345919423284, 'bagging_freq': 1, 'min_child_samples': 36}. Best is trial 34 with value: 0.993006993006993.
Trial 85 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.7901820162143982, 'bagging_freq': 3, 'min_child_samples': 29}. Best is trial 34 with value: 0.993006993006993.
Trial 86 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.9079037279237518, 'bagging_freq': 7, 'min_child_samples': 24}. Best is trial 34 with value: 0.993006993006993.
Trial 87 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6654681209378389, 'bagging_freq': 0, 'min_child_samples': 11}. Best is trial 34 with value: 0.993006993006993.
Trial 88 finished with value: 0.9790209790209791 and parameters: {'bagging_fraction': 0.6372986304201603, 'bagging_freq': 2, 'min_child_samples': 61}. Best is trial 34 with value: 0.993006993006993.
Trial 89 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7356828392166599, 'bagging_freq': 4, 'min_child_samples': 40}. Best is trial 34 with value: 0.993006993006993.
Trial 90 finished with value: 0.958041958041958 and parameters: {'bagging_fraction': 0.6181164855989274, 'bagging_freq': 7, 'min_child_samples': 44}. Best is trial 34 with value: 0.993006993006993.
Trial 91 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6747235147632097, 'bagging_freq': 0, 'min_child_samples': 59}. Best is trial 34 with value: 0.993006993006993.
Trial 92 finished with value: 0.993006993006993 and parameters: {'bagging_fraction': 0.6979391272433741, 'bagging_freq': 0, 'min_child_samples': 32}. Best is trial 34 with value: 0.993006993006993.
Trial 93 finished with value: 0.9440559440559441 and parameters: {'bagging_fraction': 0.7539206550723351, 'bagging_freq': 0, 'min_child_samples': 26}. Best is trial 34 with value: 0.993006993006993.
Trial 94 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.6905788866045115, 'bagging_freq': 1, 'min_child_samples': 32}. Best is trial 34 with value: 0.993006993006993.
Trial 95 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.47802674248437804, 'bagging_freq': 0, 'min_child_samples': 36}. Best is trial 34 with value: 0.993006993006993.
Trial 96 finished with value: 0.972027972027972 and parameters: {'bagging_fraction': 0.7755033327112355, 'bagging_freq': 1, 'min_child_samples': 55}. Best is trial 34 with value: 0.993006993006993.
Trial 97 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.7266934171613126, 'bagging_freq': 0, 'min_child_samples': 28}. Best is trial 34 with value: 0.993006993006993.
Trial 98 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.8179301609016051, 'bagging_freq': 5, 'min_child_samples': 46}. Best is trial 34 with value: 0.993006993006993.
Trial 99 finished with value: 0.951048951048951 and parameters: {'bagging_fraction': 0.5846801244531846, 'bagging_freq': 6, 'min_child_samples': 68}. Best is trial 34 with value: 0.993006993006993.
Trial 100 finished with value: 0.986013986013986 and parameters: {'bagging_fraction': 0.7940230875224689, 'bagging_freq': 2, 'min_child_samples': 31}. Best is trial 34 with value: 0.993006993006993.
Trial 101 finished with value: 0.965034965034965 and parameters: {'bagging_fraction': 0.701257826320385, 'bagging_freq': 0, 'min_child_samples': 64}. Best is trial 34 with value: 0.993006993006993.