An artificial intelligence system that performs well in tests may start delivering less accurate predictions months later. This phenomenon is known as model drift. The cause isn’t always the machine itself—often, the environment in which it operates changes, such as input data, user behavior, or the relationships between information.
How model drift manifests
Drift occurs when production conditions diverge from development or training conditions. The NIST defines this situation as an AI system’s loss of ability to meet its original assumptions as the environment evolves. A classic example is a demand forecasting model trained on historical purchase data that becomes inaccurate after a sudden change in consumer habits.
There are two main types of drift. Data drift occurs when the distribution of input data changes. Concept drift, on the other hand, occurs when the relationship between input data and the expected outcome changes.
Why January can be different from August
Imagine a company that uses AI to classify resumes. The model was trained on certain job titles, terms, and professional profiles. Over time, new professions emerge, skills become more prominent, and candidates describe their experiences differently. The system continues to operate, but its accuracy decreases.
Similar situations can affect sales forecasting systems, fraud detection, product recommendations, and other professional applications. In the case of generative AI, it’s also important to monitor response quality, relevance, consistency, the frequency of hallucinations, and changes in system behavior, as recommended by IBM.
How to identify signs of model drift
The first step is to establish a baseline. Before deploying a model, it’s essential to record its behavior under ideal conditions. Without this reference, it’s difficult to detect when performance deteriorates.
In practice, monitoring can include:
- changes in data distribution;
- decrease in forecast accuracy;
- increase in error rate;
- changes in response quality;
- variations in latency or resource consumption;
- differences between expected and observed results.
NIST recommends monitoring system behavior during operation and comparing production metrics with those observed before deployment, even if there’s no apparent failure.
What to do when performance drops
Detecting the problem is just the beginning. The team must investigate the source of the change before deciding on corrective action. In some cases, the cause lies in the data; in others, it may be the model itself, the infrastructure, or how users interact with the system.
Depending on the diagnosis, solutions may include updating training data, recalibrating or retraining the model, adjusting monitoring metrics, or revising the system architecture. IBM advocates for continuous monitoring, root cause analysis, and model updates when necessary.
The issue has also gained institutional attention. In March 2026, NIST published a report on the challenges of monitoring AI systems in production, highlighting the detection of degradation and drift as still critical issues for organizations.