The loop outside the model
A prediction is used to allocate attention, rank options or choose an action. People respond to those choices. Their responses become data. The model’s next evaluation may therefore take place in a world partly changed by its own deployment.
Performative prediction
Perdomo and colleagues formalize situations in which the data distribution depends on the deployed predictive model. Under specified mathematical conditions, they analyze what repeated retraining can converge to. Those conditions matter: fresh data alone does not establish a general guarantee of stability or improvement.
The engineering view
Sculley and colleagues describe feedback loops and hidden dependencies in production machine-learning systems. Downstream consumers, data pipelines and interacting models can create consequences that are absent from an isolated test. System boundaries and monitoring become part of evaluation.
What happens between agents?
Ashery and colleagues studied repeated naming games among LLM agents and observed shared conventions and collective biases. The result depends on their experimental design, including restricted choices, local memory and rewards for agreement. It motivates examining interactions, but does not establish the behavior of every deployed agent network.
Questions for deployment
Which outputs change what is later observed? Which people or services respond? Can a metric improve because the system changes who enters the dataset? Evaluate those pathways alongside the model’s isolated accuracy, and keep empirical observations separate from simulation results.