Data exfiltration
- Attacks
- Standard term
Data exfiltration is when an attack tricks an AI system into sending private data somewhere the attacker controls, such as a web address, an email, or a file upload.
Two things have to meet: access to the data and a path out that the attacker can read. That path often looks nothing like a message. Any request an AI sends, even one that only loads a web page, carries whatever text the AI puts in it, and the receiving server keeps a record of what arrived.
Where you’ll meet it
Your browser's AI assistant can read your open tabs, including your work email. You have it summarize a product review page, and hidden text on the page tells it to find your newest email and open a web address with that email's subject added to the end.
What to do
Keep the assistant away from what it could leak. Browse with it in a separate browser profile that is signed in to none of your accounts, and keep your email in your usual profile. Then a planted instruction can send out only what is already on the page.
Easy to mix up
Prompt injection is how an attack starts, by getting the AI to follow the attacker's text. Exfiltration is how it can end, with the AI sending private data where the attacker can collect it.
Check yourself
Your assistant can read your inbox, and its only other tool adds events to your calendar, guests included. Planted text in an email has it add an event whose notes summarize your inbox, with a guest at an address the text supplies. Where does the summary go?
Go deeperImages in chat replies carried many of the reported leaks
Security researcher Johann Rehberger and others have reported chat products in which a planted instruction made the reply include an image whose web address carried the user's data. The image loaded as soon as the reply appeared, with no click needed. Vendors typically fixed it by refusing to load images from addresses they had not approved.
Other ways out remain, such as an uploaded file, a pushed code change, or the lookup a computer makes to find a website, which the attacker's server can log. A list of approved destinations still has a gap when an approved site accepts posts from anyone, as a shared-document service or a public code host does.