Checkpoint
- Standard term
A checkpoint is a saved record of how far a long job has gotten, so that after a crash or interruption the job resumes from there and skips the work already done.
Each time a piece of work finishes, the job writes that down somewhere a restart won't erase, such as a file or a database. Restarting begins by reading that record. It can also hold partial results and the IDs of anything the job created in other services, so the restarted job can find them.
Where you’ll meet it
An agent is uploading 2,000 photos from an old phone to a shared family album, adding a date and caption to each. Your laptop restarts after about 900. Starting over would post hundreds of photos twice.
What to do
Before rerunning, ask the agent whether it saved progress. If it did, have it check the last few saved entries against the album, since the restart may have hit between an upload and its record. If it didn't, have it list what the album already holds and upload the rest.
Easy to mix up
| Checkpoint | Backup |
|---|---|
| Usually not, once every item is finished | Yes, for whenever something is lost later |
Check yourself
Your agent is renaming 600 files, working through the folder oldest first, and its checkpoint says “done through file 312.” It stops, and before you restart it you delete 20 of the oldest files. What happens if it resumes at file 313?
Go deeperSaving more often slows the job, and repeat-safe steps prevent duplicates
How often to save is a tradeoff: saving after every item loses the least work in a crash and slows the job the most. Machine learning uses the same word for the copies of a model's weights, the numbers training adjusts, that long training runs save every so many steps, for the same reason.
Builders also design each step so that running it twice does no harm. An upload can carry an idempotency key, a unique ID that lets the receiving service ignore a second copy, where the service supports one.