Deploying with Confidence

You've trained a new model that you think is better than the current one. How do you deploy it to production without breaking anything? The answer is A/B testing and model staging.

---

1. Traffic Splitting

Vertex AI Prediction allows you to split traffic between multiple models on the same endpoint. This is the foundation of A/B testing and model staging.

You can specify the percentage of traffic that you want to send to each model. For example, you could send 90% of the traffic to the current model and 10% to the new model.

---

2. A/B Testing

A/B testing is a way to compare the performance of two or more models in a live production environment. To run an A/B test, you would:

1. Deploy the new model to the same endpoint as the current model.
2. Split the traffic between the two models.
3. Monitor the performance of both models on key business metrics (e.g., click-through rate, conversion rate).
4. If the new model performs better, you can gradually increase the traffic to it until it is handling 100% of the traffic.

---

3. Model Staging (Canary Deployments)

Model staging, also known as a canary deployment, is a way to safely roll out a new model to production. The process is similar to A/B testing, but the goal is to validate the stability and performance of the new model before sending a large amount of traffic to it.

To stage a new model, you would:

1. Deploy the new model to the same endpoint as the current model.
2. Send a small percentage of traffic (e.g., 1%) to the new model.
3. Monitor the new model for errors and performance issues.
4. If the new model is stable, you can gradually increase the traffic to it until it is handling 100% of the traffic.

---

Knowledge Check