Beyond Default Settings

The performance of your model is highly sensitive to its hyperparameters: learning_rate, dropout_rate, number_of_layers, etc. Manually tuning these is slow and inefficient. Vertex AI provides a service called Vizier for automated hyperparameter tuning.

---

1. Search Algorithms

Vertex AI supports three main algorithms for hyperparameter tuning:

• Grid Search: Exhaustively tries every combination of parameters you specify. This is very slow and generally not recommended.
• Random Search: Randomly samples combinations of parameters. Surprisingly effective and a good baseline.
• Bayesian Optimization: This is the default and most recommended algorithm. It uses the results of previous trials to intelligently choose the next set of parameters to try. This is much more efficient than Grid or Random search.

Exam Tip: If you see a question about efficient hyperparameter tuning, the answer is almost always Bayesian Optimization.

---

2. Defining the Search Space

You need to tell Vertex AI which hyperparameters to tune and what range of values to try. You do this by defining a search space in a config.yaml file.

studySpec:
  metrics:
  - metricId: accuracy
    goal: MAXIMIZE
  parameters:
  - parameterId: learning_rate
    doubleValueSpec:
      minValue: 0.0001
      maxValue: 0.1
    scaleType: UNIT_LOG_SCALE
  - parameterId: num_layers
    integerValueSpec:
      minValue: 1
      maxValue: 5
  - parameterId: optimizer
    categoricalValueSpec:
      values:
      - adam
      - sgd

Key Configuration Points

• Metrics: You need to specify the metric you want to optimize (e.g., accuracy) and the goal (e.g., MAXIMIZE).
• Parameters: For each hyperparameter, you need to specify its name, type (double, integer, categorical, or discrete), and the range of values to search.
• Scale Type: For floating-point numbers, using a LOG_SCALE is often more effective, especially for learning rates.

---

3. Integrating with Custom Training

You can easily add a hyperparameter tuning job to your custom training script. The training script needs to be able to:

1. Accept the hyperparameters as command-line arguments.
2. Report the final metric value back to Vertex AI.

The cloudml-hypertune library is a simple way to do this.

import hypertune

# ... train your model ...

# Report the metric
hpt = hypertune.HyperTune()
hpt.report_hyperparameter_tuning_metric(
    hyperparameter_metric_tag='accuracy',
    metric_value=accuracy
)

---

4. Summary

• Automated hyperparameter tuning is more efficient and effective than manual tuning.
• Bayesian Optimization is the preferred algorithm for hyperparameter tuning in Vertex AI.
• You need to define a search space to tell Vertex AI which hyperparameters to tune and what values to try.
• Your training script needs to be able to accept hyperparameters as arguments and report the final metric value.

---

Knowledge Check