Imagine a machine learning model that, six months ago, predicted with 92% accuracy which customers would cancel a service, but today gets fewer than 70% of those predictions right. Nothing in the code has changed. What changed was the world around it, and this is precisely the kind of situation Rolando Bonaccorsi, a leader in AI and data science applied to business and operations, uses to explain why the model training phase tends to receive all the attention, while its ongoing maintenance receives almost none.
The phenomenon has a technical name: data drift, which occurs when the statistical patterns of the real world gradually move away from those that existed when the model was trained. Consumer behavior changes, new products are added to the catalog, an economic crisis alters purchasing habits, and the model, trained on a snapshot of the past, continues making decisions as if that past were still the present.
The Blind Spot Between the Lab and Production
Data science teams often validate a model with impressive metrics in a controlled environment, but the validation process ends there, without any structured system for monitoring the model after it enters production and begins dealing with real-world data that is unpredictable and constantly changing.
Rolando Bonaccorsi points out that a production model without continuous monitoring is essentially a silent bet that the world will stop changing—a bet that almost never pays off. Therefore, every model deployment should include, from day one, specific metrics for tracking performance over time, rather than relying solely on the initial validation report.
Retraining Is Not the Same as Fixing the Problem!
When performance declines, the automatic response of many teams is simply to retrain the model using more recent data, without investigating whether the root cause is data drift, a change in the definition of the business problem itself, or a silent error in the data collection process feeding the pipeline.
Faced with this dilemma, Rolando Bonaccorsi summarizes the issue by saying that retraining without diagnosis is a generic remedy for an unidentified symptom. Therefore, any performance degradation should first go through a structured investigation capable of distinguishing between a temporary data problem and a structural change that requires completely rethinking the variables used by the model.
The Governance Gap Between Versions
Every new model training cycle creates a different version, with its own weights, hyperparameters, and training dataset—an entire history that many companies fail to document rigorously enough, making it difficult to understand, months later, why a particular decision was made by a previous version of the system.
Rolando Bonaccorsi illustrates this reasoning with a simple example: without version traceability, a future audit into why a customer was denied a particular service may simply have no answer, because no one can reconstruct which model, using which data, made that specific decision at that particular point in time.
Infrastructure Costs That Grow Under the Radar
Maintaining multiple models in production, each requiring monitoring, periodic retraining, and dedicated computing capacity, generates infrastructure costs that grow less visibly than the initial development investment but accumulate significantly over months of continuous operation.
This is where Rolando Bonaccorsi’s observation becomes particularly relevant: the true cost of a machine learning model is never simply the cost of training it once, but the cumulative cost of keeping it relevant and reliable for as long as it remains in production—an expense that rarely appears in full in the initial budget of an applied artificial intelligence project.
The discipline known as MLOps exists precisely to address this gap with the same level of rigor that has been applied to traditional software operations for decades: continuous monitoring, strict version control, and clear retraining processes make the difference between a model that ages gracefully and one that silently deteriorates until someone realizes, too late, that its automated decisions no longer make sense.
Companies that treat model production as the beginning of a continuous life cycle, rather than the finish line of a project, build more reliable artificial intelligence systems that are also easier to audit when questions arise about any specific automated decision.
