Workflow drift
- Failures
- Industry term
Workflow drift is a gradual gap that opens between how an AI workflow was meant to behave and how it behaves now, after changes to its prompts, tools, data, model, or inputs.
Some of the changes are edits that seem small and sensible when they're made, and some happen with no edit at all. Each output still looks fine on its own, so the gap shows only when recent results sit beside what the workflow was built to do, a comparison that daily use rarely calls for.
Where you’ll meet it
Your team's AI assistant sorts incoming support emails and flags urgent ones for a same-day reply. Over a year, several people edited its instructions to sound friendlier. This week a customer's “our whole site is down” email sat in the normal queue for two days.
What to do
- Collect a few dozen past emails, including borderline urgent ones, and label each urgent or not.
- Run them through the assistant and compare its flags with your labels.
- Rerun the set monthly and after any edit to the assistant's instructions, adding a few recent emails each time.
Easy to mix up
Context rot usually clears up in a fresh session, which starts with little context. Workflow drift carries into every new session, because the changes live in the workflow itself.
Check yourself
Your meeting-notes assistant has run for months. Yesterday its action-item list left out a task your manager handed out. Is that workflow drift?
Go deeperFinding the cause takes a record of what each run used
Workflow drift parallels configuration drift in infrastructure management, where servers set up the same way slowly diverge through one-off fixes. Machine learning names two related cases. Data drift is when incoming data moves away from what a system was built on, and concept drift is when the right output for the same input changes over time.
Comparing prompt versions shows edits but misses a model update or a change in the data's format. So teams log, for each run, the version of the prompt, the tools, the model, and the data it used, which lets them line up the day results changed with the change that caused it.