--crawl-allow-url

CLI: --crawl-allow-url
REST: crawl_allow_url

ValueGLOB

LicensePro

Permits URLs matching this glob: those a --crawl-deny-url glob blocked, and in a link crawl those beyond the seed’s registrable domain. Repeatable.

Example

Capture only the chapters a sitemap lists:

Run locally

zshot -f house-at-pooh-corner.wacz \
  --crawl-deny-url "*" \
  --crawl-allow-url "*/**/chapter-*" \
  https://zshot-cli.com/example_assets/house-at-pooh-corner/sitemap.xml

{
  "url": "https://zshot-cli.com/example_assets/house-at-pooh-corner/sitemap.xml",
  "output_type": "wacz",
  "crawl_deny_url": ["*"],
  "crawl_allow_url": ["*/**/chapter-*"]
}

curl -X POST http://127.0.0.1:3000/ \
  -H "Content-Type: application/json" \
  -d '{"url":"https://zshot-cli.com/example_assets/house-at-pooh-corner/sitemap.xml","output_type":"wacz","crawl_deny_url":["*"],"crawl_allow_url":["*/**/chapter-*"]}' \
  -o house-at-pooh-corner.wacz