Validating Data and Models
How to ensure data quality and model performance across training and serving. A guide to TensorFlow Data Validation (TFDV) and TensorFlow Model Analysis (TFMA).
Garbage In, Garbage Out
The quality of your model is only as good as the quality of your data. It's crucial to validate your data and models throughout the ML lifecycle. TensorFlow provides two powerful libraries for this: TensorFlow Data Validation (TFDV) and TensorFlow Model Analysis (TFMA).
1. TensorFlow Data Validation (TFDV)
TFDV helps you understand and validate your data. It can:
- Generate descriptive statistics: Get a quick overview of your data, including the number of missing values, the distribution of values, and the number of unique values.
- Infer a schema: Automatically generate a schema that describes the expected properties of your data, such as the data type of each feature and the expected range of values.
- Detect anomalies: Compare your data to the schema and identify any anomalies, such as missing values, incorrect data types, or out-of-range values.
TFDV in a Pipeline
You can use TFDV in a TFX pipeline to:
- Validate your training data: Ensure that your training data is clean and consistent.
- Detect training-serving skew: Compare the statistics of your training data to the statistics of your serving data to detect any differences that could impact your model's performance.
2. TensorFlow Model Analysis (TFMA)
TFMA helps you evaluate your model's performance. It can:
- Compute a wide range of metrics: Calculate metrics such as accuracy, precision, recall, and AUC.
- Slice your data: Evaluate your model's performance on different slices of your data (e.g., by country, by age group). This is useful for identifying any fairness or bias issues.
- Compare models: Compare the performance of different models or different versions of the same model.
TFMA in a Pipeline
You can use TFMA in a TFX pipeline to:
- Evaluate your model after training: Ensure that your model meets your performance requirements before deploying it to production.
- Continuously monitor your model's performance: Track your model's performance over time and trigger an alert if it drops below a certain threshold.
Knowledge Check
?Knowledge Check
You are training a model to predict customer churn. You want to make sure that your model is fair and that it performs equally well for customers in different countries. What should you do?