Distribution shift
- Failures
- Standard term
Distribution shift is a difference between the data an AI system was built or tested on and the data it meets once people use it.
A model learns whatever patterns separated right answers from wrong ones in its training data, including some that held only there, such as a familiar layout or a common phrasing. When the incoming data changes, some of those patterns stop holding, and accuracy on the new cases can fall, hold, or rise.
Where you’ll meet it
Your company's expense tool reads receipts with AI and filled in totals correctly all last year. This quarter the company moved all travel booking to a new service whose receipts are laid out differently, and now the tool misreads the totals on most of your team's travel claims.
What to do
For now, check every field on your travel claims before you submit them. Tell whoever runs the tool that the errors began with the new service's receipts, and attach a few, so they can add that layout to the tool's tests.
Easy to mix up
Out-of-distribution describes one input unlike anything in training. Distribution shift describes the whole mix of inputs drifting away from what the system was built and tested on.
Check yourself
Your company extends its return window from 30 days to 60 and updates the help bot's documents. Customers keep asking the same questions, and the next morning the bot passes its full test suite. What does the pass tell you about its answers on returns?
Go deeperResearchers sort shifts by which part of the data changed
In covariate shift the inputs change and the right answer for each one stays the same, as with photos from a new camera. In label shift, how common each answer is changes, as when a fraud checker meets a season with far more fraud. In concept shift the right answer itself changes, as when a tax rule is rewritten.
Each kind needs a different repair. New inputs the model can't handle may need examples of them added to training, while a shift in how common each answer is may need only a new cutoff for flagging a payment as fraud. Many large language models' training data is undisclosed, so outsiders can't check whether inputs like theirs were covered.