Model Monitoring (Data Drift and Concept Drift) Model monitoring encompasses the continuous monitoring of a trained model's performance and adherence to its...
Model Monitoring (Data Drift and Concept Drift)
Model monitoring encompasses the continuous monitoring of a trained model's performance and adherence to its intended functionality. Detecting changes in the data the model is trained on allows us to identify potential drifts in the model's behavior, indicating a shift from its intended purpose. These drifts can be caused by various factors, including noise in the data, changes in the underlying data distribution, or even deliberate manipulation of the data.
Data drift: A gradual change in the data that the model is trained on. For instance, if the model is trained on a dataset of customer transactions, a gradual increase in the average order value could indicate data drift due to an increase in customer spending.
Concept drift: When the model's internal representations (such as the learned features and their relationships) drift away from their initial values. This can happen due to various factors, including the introduction of new features, changes in the data distribution, or the gradual decay of model parameters.
Examples:
A model trained on financial news data might experience data drift due to fluctuations in stock prices.
A model trained on a dataset of text documents might experience concept drift as the language evolves and new topics and concepts emerge over time.
A model trained on a dataset of sensor readings might experience both data drift and concept drift as the readings become corrupted or the sensor technology changes