--archive-spill-max-bytes

LicensePRO

Cap the total bytes a crawl or sitemap walk spills to disk while capturing. On reaching the cap the walk stops gracefully and writes a successful partial archive, the same way the page budget does. Captured bodies stream to per-request spill files, so a walk scales to disk rather than memory. Requires WARC output.

Example

zshot -t warc -f site.warc.gz --crawl-links --archive-spill-max-bytes 1073741824 https://example.com