Your Model Is Not Static

The world is constantly changing, and your model needs to be able to adapt to these changes. If you don't monitor your model's performance over time, you may not realize that it's no longer making accurate predictions.

Vertex AI Model Monitoring is a tool that helps you monitor your model's performance in production and detect any issues that may arise.

---

1. Feature Drift

Feature drift occurs when the statistical distribution of a feature in your serving data changes over time. For example, if you are training a model to predict the price of a house, the distribution of the 'interest_rate' feature may change over time as the economy changes.

Feature drift can have a significant impact on your model's performance. If your model is trained on data with one distribution but is then served on data with a different distribution, it may not be able to make accurate predictions.

---

2. Model Performance

In addition to monitoring for feature drift, you should also monitor your model's performance on key business metrics, such as:

• Click-through rate: The percentage of users who click on an ad.
• Conversion rate: The percentage of users who make a purchase.
• Customer satisfaction: A measure of how satisfied customers are with your product or service.

---

3. Vertex AI Model Monitoring

Vertex AI Model Monitoring can be used to monitor for both feature drift and changes in your model's performance. To use Vertex AI Model Monitoring, you will need to:

1. Create a baseline: A snapshot of your model's performance on a representative dataset.
2. Configure a monitoring job: Specify the metrics that you want to monitor and the thresholds that will trigger an alert.
3. Receive alerts: If any of the metrics exceed the thresholds that you have set, you will receive an alert.

---

Knowledge Check