Concepts
Your files
How /kaggle/working survives every session, and how kdev picks the version to restore.
/kaggle/working behaves like a disk that is always there.
Kaggle saves it. When a “Save & Run All” session ends, Kaggle keeps
/kaggle/working as that version’s output — and it does so however the
session ends. A clean exit, a crash and a cancel were all measured to save
every file up to the moment the container stopped. Older versions stay
readable (v1, v2, …), and they belong to the notebook, not to an account.
kdev puts it back. Kaggle starts every batch session with an empty
/kaggle/working; its “Persistence” setting only covers the web editor. So
each kdev up works out which saved version you left off at, and the new box
downloads it before you connect.
| how the last session ended | what the next kdev up gives you |
|---|---|
kdev down |
everything |
kdev down, then kdev up straight away |
everything: up waits for the save first (about 2 min per GB) |
| Stop session in the Kaggle UI | everything (a cancel still saves) |
| the session’s time ran out | everything (the box stops itself 2 min early) |
| the tunnel process died | everything |
| the box crashed | everything, up to the crash |
| your laptop or network dropped | the box is still running: kdev up connects to it |
| Kaggle’s machine itself failed | not testable; a git remote covers it |
What kdev adds on top of Kaggle
Kaggle’s saved output keeps every regular file — thousands of them, dotfiles,
.git, large binaries. It drops symlinks and empty directories, and loses
executable bits, which would break a Python virtualenv. The box records
those in .kdev/meta.json every 30 seconds, and the restore puts them back.
Picking the version to restore
A version with no files is a source-only save (group setup, a Quick Save), so
kdev walks past it. A session that was stopped before its restore finished —
or that you worked in before restoring — is not skipped: it gets stacked on
the versions it was meant to be built on, so nothing done in it is lost. The
layers underneath fill in only what that session’s restore never delivered
(the box logs each file as it lands, in .kdev/fetched), so a file you deleted
there stays deleted.
A restore never overwrites a file written in the current session, so it is safe to run again at any time:
kdev restore # finish or repeat the restore on the running box
kdev restore --from v12 # add files v12 has that the box doesn't (not a rollback)
kdev workspace files # what `kdev up` would restore (--version v12 for another)
kdev backup # optional: an exact copy of the box on this machine
Your own cells in the notebook
kdev’s cell sits on top, and nothing below it runs. When a session ends,
later cells are blanked for that run, so a training cell can’t start after
kdev down and use quota. Your cells stay in the notebook, untouched.
Files made in the Kaggle web editor
The editor’s interactive session is the one place kdev can’t reach: its files
only reach the API when a version is saved. Kaggle describes Quick Save as
saving the notebook “as displayed”, which isn’t a clear promise to save the
working directory, so check with kdev workspace files. Everything made
through kdev is covered.