A Whetlab tuning experiment.
A name and description for the experiment must be specified.
A Whetlab access token must also be provided.
The parameters to tune in the experiment are specified by
parameters. It should be a dict, where keys are
the parameters (str) and values are dict that
provide information about these parameters. Each of these
dict should contain the appropriate keys to properly describe
the parameter:
- 'min': minimum value of the parameter
- 'max': maximum value of the parameter
- 'scale': scale to use when exploring parameter values (default: 'linear')
- 'units': units (str) in which the parameter is measured (default: '')
- 'type': type of the parameter (default: 'float')
- 'size': size of parameter (default: 1)
outcome should also be a dict, describing the outcome. It
should have the keys:
- 'name': name (str) for the outcome being optimized
- 'type': type of the parameter, either 'float', 'int' or 'enum' (default: 'float')
- 'units': units (str) in which the parameter is measured (default: '')
If name and description match a previously created experiment,
that experiment will be resumed (in this case, parameters and outcoume are ignored).
This behavior can be avoided by setting the argument resume
to False (in which case an error will be raised is an experiment
with the same name and description is found).
Parameters: |
- access_token (str) – Access token for your Whetlab account.
- name (str) – Name of the experiment.
- description (str) – Description of the experiment.
- parameters (dict) – Parameters to be tuned during the experiment.
- outcome (dict) – Description of the outcome to maximize.
- resume (bool) – Whether to allow the resuming of a previously executed experiment.
|
A Whetlab experiment instance will have the following variables:
Variables: |
- parameters – Parameters to be tuned during the experiment.
- outcome – Description of the outcome to maximize.
- experiment_id – ID of the experiment (useful for resuming).
|
-
best(*args, **kwargs)[source]
Return job with best outcome found so far.
Returns: | Parameter values with best outcome. |
Return type: | dict |
-
cancel(*args, **kwargs)[source]
Cancel a job, by removing it from the jobs recorded so far in the experiment.
Parameters: | param_values (dict) – Values of the parameters for the job to cancel. |
-
clear_pending(*args, **kwargs)[source]
Cancel jobs (results) that are marked as pending.
-
pending(*args, **kwargs)[source]
Return the list of jobs which have been suggested, but for which no
result has been provided yet.
Returns: | List of parameter values. |
Return type: | list |
-
report(*args, **kwargs)[source]
Plot a visual report of the progress made so far in the experiment.
-
suggest(*args, **kwargs)[source]
Suggest a new job.
Returns: | Values to assign to the parameters in the suggested job. |
Return type: | dict |
-
update(*args, **kwargs)[source]
Update the experiment with the outcome value associated with some parameter values.
Parameters: |
- param_values (dict) – Values of parameters.
- outcome_val (type defined for outcome) – Value of the outcome.
|