zshot/cliDownload

--crawl-delay-cap

ValueSECS
LicensePRO

Ceiling in seconds for a robots.txt Crawl-delay during a sitemap walk or link crawl. The default is 30. A value of 0 disables the ceiling and honors the declared delay in full.

Example

zshot -t warc -f site.warc.gz --crawl-url-is-sitemap xml \
  --crawl-delay-cap 5 https://example.com/sitemap.xml