Which two sampling methods can you use?

You plan to use the Hyperdrive feature of Azue Machine Learning to determine the optimal hyperparameter values when training a model.

You must use Hyperdrive to try combinations of the following hyperparameter values.

You must not apply an early termination policy.

– learning_rate: any value between 0.001 and 0.1

– batch_size: 16, 32, or 64

You need to configure the sampling method for the Hyperdrive experiment.

Which two sampling methods can you use? Each correct answer is a complete solution. NOTE: Each correct selection is worth one point.
A . No sampling
B . Grid sampling
C . Bayesian sampling
D . Random sampling

View Answer

Answer: CD

Explanation:

C: Bayesian sampling is based on the Bayesian optimization algorithm and makes intelligent choices on the hyperparameter values to sample next. It picks the sample based on how the previous samples performed, such that the new sample improves the reported primary metric.

Bayesian sampling does not support any early termination policy

Example:

from azureml.train.hyperdrive import BayesianParameterSampling

from azureml.train.hyperdrive import uniform, choice

param_sampling = BayesianParameterSampling( {

"learning_rate": uniform(0.05, 0.1),

"batch_size": choice(16, 32, 64, 128)

}

)

D: In random sampling, hyperparameter values are randomly selected from the defined search space. Random sampling allows the search space to include both discrete and continuous hyperparameters.

Incorrect Answers:

B: Grid sampling can be used if your hyperparameter space can be defined as a choice among discrete values and if you have sufficient budget to exhaustively search over all values in the defined search space. Additionally, one can use automated early termination of poorly performing runs, which reduces wastage of resources.

Example, the following space has a total of six samples:

from azureml.train.hyperdrive import GridParameterSampling

from azureml.train.hyperdrive import choice

param_sampling = GridParameterSampling( {

"num_hidden_layers": choice(1, 2, 3),

"batch_size": choice(16, 32)

}

)

Reference: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-tune-hyperparameters