深度学习 / 默认 · 2月 19, 2022 0

自动调参利器BayesianOptimization贝叶斯优化测试

内容纲要

BayesianOptimization贝叶斯优化方案测试

https://github.com/fmfn/BayesianOptimization

“python
!pip install bayesian-optimization


    Requirement already satisfied: bayesian-optimization in /usr/local/lib/python3.7/dist-packages (1.2.0)
    Requirement already satisfied: scikit-learn>=0.18.0 in /usr/local/lib/python3.7/dist-packages (from bayesian-optimization) (1.0.1)
    Requirement already satisfied: scipy>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from bayesian-optimization) (1.4.1)
    Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from bayesian-optimization) (1.19.5)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.18.0->bayesian-optimization) (1.1.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.18.0->bayesian-optimization) (3.0.0)

```python
from bayes_opt import BayesianOptimization

# 黑盒参数
def black_box_function(x, y):
    """Function with unknown internals we wish to maximize.

    This is just serving as an example, for all intents and
    purposes think of the internals of this function, i.e.: the process
    which generates its output values, as unknown.
    """
    return (-x ** 2 - (y - 1) ** 2 + 1)*1

# 参数候选
# Bounded region of parameter space
pbounds = {'x': (200000, 400), 'y': (-30000, 100000003)}

optimizer = BayesianOptimization(
    f=black_box_function,
    pbounds=pbounds,
    random_state=1,
)

The BayesianOptimization object will work out of the box without much tuning needed. The main method you should be aware of is maximize, which does exactly what you think it does.

There are many parameters you can pass to maximize, nonetheless, the most important ones are:

n_iter: How many steps of bayesian optimization you want to perform. The more steps the more likely to find a good maximum you are.
init_points: How many steps of random exploration you want to perform. Random exploration can help by diversifying the exploration space.

optimizer.maximize(
    init_points=20,
    n_iter=5,
)
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 1       [0m | [0m-5.187e+1[0m | [0m 1.168e+0[0m | [0m 7.202e+0[0m |
| [95m 2       [0m | [95m-9.128e+1[0m | [95m 2e+05   [0m | [95m 3.021e+0[0m |
| [95m 3       [0m | [95m-8.479e+1[0m | [95m 1.707e+0[0m | [95m 9.207e+0[0m |
| [0m 4       [0m | [0m-1.193e+1[0m | [0m 1.628e+0[0m | [0m 3.454e+0[0m |
| [0m 5       [0m | [0m-2.902e+1[0m | [0m 1.208e+0[0m | [0m 5.387e+0[0m |
| [0m 6       [0m | [0m-4.694e+1[0m | [0m 1.163e+0[0m | [0m 6.851e+0[0m |
| [0m 7       [0m | [0m-7.71e+15[0m | [0m 1.592e+0[0m | [0m 8.781e+0[0m |
| [0m 8       [0m | [0m-4.494e+1[0m | [0m 1.945e+0[0m | [0m 6.704e+0[0m |
| [0m 9       [0m | [0m-3.12e+15[0m | [0m 1.167e+0[0m | [0m 5.586e+0[0m |
| [0m 10      [0m | [0m-3.915e+1[0m | [0m 1.72e+05[0m | [0m 1.979e+0[0m |
| [0m 11      [0m | [0m-9.375e+1[0m | [0m 4.017e+0[0m | [0m 9.683e+0[0m |
| [0m 12      [0m | [0m-4.792e+1[0m | [0m 1.374e+0[0m | [0m 6.922e+0[0m |
| [0m 13      [0m | [0m-8.003e+1[0m | [0m 2.507e+0[0m | [0m 8.946e+0[0m |
| [95m 14      [0m | [95m-1.506e+1[0m | [95m 1.83e+05[0m | [95m 3.877e+0[0m |
| [0m 15      [0m | [0m-7.711e+1[0m | [0m 1.661e+0[0m | [0m 8.781e+0[0m |
| [0m 16      [0m | [0m-1.772e+1[0m | [0m 1.804e+0[0m | [0m 4.209e+0[0m |
| [0m 17      [0m | [0m-2.841e+1[0m | [0m 8.805e+0[0m | [0m 5.33e+07[0m |
| [0m 18      [0m | [0m-9.942e+1[0m | [0m 6.19e+04[0m | [0m 3.153e+0[0m |
| [0m 19      [0m | [0m-6.965e+1[0m | [0m 6.297e+0[0m | [0m 8.346e+0[0m |
| [0m 20      [0m | [0m-5.626e+1[0m | [0m 1.963e+0[0m | [0m 7.501e+0[0m |
| [0m 21      [0m | [0m-3.46e+14[0m | [0m 400.0   [0m | [0m 1.86e+07[0m |
| [95m 22      [0m | [95m-1.443e+1[0m | [95m 400.0   [0m | [95m 3.799e+0[0m |
| [95m 23      [0m | [95m-1.397e+1[0m | [95m 400.0   [0m | [95m 3.738e+0[0m |
| [0m 24      [0m | [0m-6.304e+1[0m | [0m 400.0   [0m | [0m 7.94e+07[0m |
| [0m 25      [0m | [0m-1.42e+13[0m | [0m 400.0   [0m | [0m 3.769e+0[0m |
=================================================

The best combination of parameters and target value found can be accessed via the property optimizer.max.

print(optimizer.max)
{'target': 12.635001220531157, 'params': {'x': 3.6921342392926375, 'y': 0.9439109674344994}}
for i in range(10):
  optimizer.maximize(
    init_points=2,
    n_iter=3,
  ) 
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [95m 6       [0m | [95m 16.13   [0m | [95m 2.294   [0m | [95m-2.446   [0m |
| [0m 7       [0m | [0m 8.341   [0m | [0m 2.373   [0m | [0m-0.9266  [0m |
| [95m 8       [0m | [95m 22.76   [0m | [95m 2.989   [0m | [95m-2.85    [0m |
| [95m 9       [0m | [95m 31.0    [0m | [95m 4.0     [0m | [95m-3.0     [0m |
| [0m 10      [0m | [0m 29.56   [0m | [0m 3.998   [0m | [0m-2.817   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 11      [0m | [0m 7.392   [0m | [0m 2.794   [0m | [0m 0.2329  [0m |
| [0m 12      [0m | [0m 7.069   [0m | [0m 2.838   [0m | [0m 1.111   [0m |
| [0m 13      [0m | [0m 7.069   [0m | [0m 2.838   [0m | [0m 1.111   [0m |
| [0m 14      [0m | [0m 7.069   [0m | [0m 2.838   [0m | [0m 1.111   [0m |
| [0m 15      [0m | [0m 7.069   [0m | [0m 2.838   [0m | [0m 1.111   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 16      [0m | [0m 6.412   [0m | [0m 2.409   [0m | [0m 2.269   [0m |
| [0m 17      [0m | [0m 3.223   [0m | [0m 2.055   [0m | [0m 1.023   [0m |
| [0m 18      [0m | [0m 3.223   [0m | [0m 2.055   [0m | [0m 1.023   [0m |
| [0m 19      [0m | [0m 31.0    [0m | [0m 4.0     [0m | [0m-3.0     [0m |
| [0m 20      [0m | [0m 31.0    [0m | [0m 4.0     [0m | [0m-3.0     [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 21      [0m | [0m 7.455   [0m | [0m 2.835   [0m | [0m 0.3521  [0m |
| [0m 22      [0m | [0m 12.11   [0m | [0m 2.281   [0m | [0m-1.811   [0m |
| [0m 23      [0m | [0m 12.11   [0m | [0m 2.281   [0m | [0m-1.811   [0m |
| [0m 24      [0m | [0m 12.11   [0m | [0m 2.281   [0m | [0m-1.811   [0m |
| [0m 25      [0m | [0m 12.11   [0m | [0m 2.281   [0m | [0m-1.811   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 26      [0m | [0m 15.25   [0m | [0m 3.601   [0m | [0m 2.81    [0m |
| [0m 27      [0m | [0m 5.924   [0m | [0m 2.627   [0m | [0m 1.154   [0m |
| [0m 28      [0m | [0m 5.924   [0m | [0m 2.627   [0m | [0m 1.154   [0m |
| [0m 29      [0m | [0m 5.924   [0m | [0m 2.627   [0m | [0m 1.154   [0m |
| [0m 30      [0m | [0m 5.924   [0m | [0m 2.627   [0m | [0m 1.154   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 31      [0m | [0m 14.95   [0m | [0m 3.753   [0m | [0m 2.368   [0m |
| [0m 32      [0m | [0m 17.89   [0m | [0m 2.17    [0m | [0m-2.766   [0m |
| [0m 33      [0m | [0m 17.89   [0m | [0m 2.17    [0m | [0m-2.766   [0m |
| [0m 34      [0m | [0m 17.89   [0m | [0m 2.17    [0m | [0m-2.766   [0m |
| [0m 35      [0m | [0m 17.89   [0m | [0m 2.17    [0m | [0m-2.766   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 36      [0m | [0m 6.084   [0m | [0m 2.34    [0m | [0m 2.269   [0m |
| [0m 37      [0m | [0m 5.996   [0m | [0m 2.197   [0m | [0m-0.4734  [0m |
| [0m 38      [0m | [0m 5.996   [0m | [0m 2.197   [0m | [0m-0.4734  [0m |
| [0m 39      [0m | [0m 5.996   [0m | [0m 2.197   [0m | [0m-0.4734  [0m |
| [0m 40      [0m | [0m 5.996   [0m | [0m 2.197   [0m | [0m-0.4734  [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 41      [0m | [0m 14.97   [0m | [0m 3.916   [0m | [0m 0.199   [0m |
| [0m 42      [0m | [0m 14.89   [0m | [0m 3.384   [0m | [0m-1.107   [0m |
| [0m 43      [0m | [0m 14.89   [0m | [0m 3.384   [0m | [0m-1.107   [0m |
| [0m 44      [0m | [0m 14.89   [0m | [0m 3.384   [0m | [0m-1.107   [0m |
| [0m 45      [0m | [0m 14.89   [0m | [0m 3.384   [0m | [0m-1.107   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 46      [0m | [0m 11.39   [0m | [0m 3.373   [0m | [0m 2.008   [0m |
| [0m 47      [0m | [0m 3.399   [0m | [0m 2.037   [0m | [0m 1.501   [0m |
| [0m 48      [0m | [0m 3.399   [0m | [0m 2.037   [0m | [0m 1.501   [0m |
| [0m 49      [0m | [0m 3.399   [0m | [0m 2.037   [0m | [0m 1.501   [0m |
| [0m 50      [0m | [0m 3.399   [0m | [0m 2.037   [0m | [0m 1.501   [0m |
=================================================
|   iter    |  target   |     x     |     y     |
-------------------------------------------------
| [0m 51      [0m | [0m 15.06   [0m | [0m 3.978   [0m | [0m 1.489   [0m |
| [0m 52      [0m | [0m 6.099   [0m | [0m 2.561   [0m | [0m 1.736   [0m |
| [0m 53      [0m | [0m 6.099   [0m | [0m 2.561   [0m | [0m 1.736   [0m |
| [0m 54      [0m | [0m 6.099   [0m | [0m 2.561   [0m | [0m 1.736   [0m |
| [0m 55      [0m | [0m 6.099   [0m | [0m 2.561   [0m | [0m 1.736   [0m |
=================================================
print(optimizer.max)
{'target': 31.0, 'params': {'x': 4.0, 'y': -3.0}}
dir(optimizer)
['__class__',
 '__delattr__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 '_bounds_transformer',
 '_events',
 '_gp',
 '_prime_queue',
 '_prime_subscriptions',
 '_queue',
 '_random_state',
 '_space',
 '_verbose',
 'dispatch',
 'get_subscribers',
 'max',
 'maximize',
 'probe',
 'register',
 'res',
 'set_bounds',
 'set_gp_params',
 'space',
 'subscribe',
 'suggest',
 'unsubscribe']
%d 博主赞过: