--archive-body-size-limit | Maximum per-resource body size captured into an archive Standard |
--archive-cdx | Path to write a CDXJ index of the WARC's records Standard |
--archive-spill-max-bytes | Cap the total bytes a crawl or sitemap walk spills to disk Pro |
--crawl-allow-url | Re-permit a URL that a --crawl-deny-url glob blocked Pro |
--crawl-concurrency | Browsers a crawl or sitemap walk loads pages in at once Pro |
--crawl-delay | Least time in seconds between page loads on one host Pro |
--crawl-deny-url | Skip URLs matching this glob during a sitemap walk or link crawl Pro |
--crawl-link-depth | Hops to follow from the seed when crawling links Pro |
--crawl-link-selector | Additional CSS selector for crawlable links Pro |
--crawl-links | Crawl links discovered on each captured page into the same WARC Pro |
--crawl-max-links | Cap on total pages fetched by a crawl or sitemap walk Pro |
--crawl-media-sources | Fetch <audio>/<video> source URLs on each walked page so deferred media is archived Pro |
--crawl-page-timeout | Per-page capture budget in seconds for WARC output Pro |
--crawl-sitemap-max-depth | Levels of <sitemapindex> to follow when capturing a sitemap Pro |
--crawl-url-is-sitemap | Treat the target as a manifest of URLs to capture into one WARC Pro |
--har | Path to write a diagnostic HAR (HTTP Archive) file Standard |
--har-capture-bodies | HAR will contain response bodies Standard |
--har-captures-navigation | HAR will contain navigator session Pro |
--warc | Path to write a WARC for this request Standard |
--warc-captures-navigation | WARC will contain navigator session Standard |
--warc-no-gzip | Disable gzip compression for output WARC Standard |