Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Data exfiltration

  • Attacks
  • Standard term
4 steps

Data exfiltration is when an attack tricks an AI system into sending private data somewhere the attacker controls, such as a web address, an email, or a file upload.

Two things have to meet: access to the data and a path out that the attacker can read. That path often looks nothing like a message. Any request an AI sends, even one that only loads a web page, carries whatever text the AI puts in it, and the receiving server keeps a record of what arrived.

Where you’ll meet it

Your browser's AI assistant can read your open tabs, including your work email. You have it summarize a product review page, and hidden text on the page tells it to find your newest email and open a web address with that email's subject added to the end.

What to do

Keep the assistant away from what it could leak. Browse with it in a separate browser profile that is signed in to none of your accounts, and keep your email in your usual profile. Then a planted instruction can send out only what is already on the page.

Easy to mix up

Prompt injection is how an attack starts, by getting the AI to follow the attacker's text. Exfiltration is how it can end, with the AI sending private data where the attacker can collect it.

Check yourself

Your assistant can read your inbox, and its only other tool adds events to your calendar, guests included. Planted text in an email has it add an event whose notes summarize your inbox, with a guest at an address the text supplies. Where does the summary go?

Go deeperImages in chat replies carried many of the reported leaks

Security researcher Johann Rehberger and others have reported chat products in which a planted instruction made the reply include an image whose web address carried the user's data. The image loaded as soon as the reply appeared, with no click needed. Vendors typically fixed it by refusing to load images from addresses they had not approved.

Other ways out remain, such as an uploaded file, a pushed code change, or the lookup a computer makes to find a website, which the attacker's server can log. A list of approved destinations still has a gap when an approved site accepts posts from anyone, as a shared-document service or a public code host does.

Practice Modern AI

Or take the learning quiz on a mix of every set