--skip-robot-instructions

CLI: --skip-robot-instructions
REST: skip_robot_instructions

Captures pages that robots.txt disallows for the ZShot user agent, and ignores the robot directives pages give: crawls follow, index and archive every page, and a model may read a page that opts out of AI use.

Example

Run locally

zshot -f page.png \
  --skip-robot-instructions \
  https://zshot-cli.com/example_assets/hello_zshot/

{
  "url": "https://zshot-cli.com/example_assets/hello_zshot/",
  "output_type": "png",
  "skip_robot_instructions": true
}

curl -X POST http://127.0.0.1:3000/ \
  -H "Content-Type: application/json" \
  -d '{"url":"https://zshot-cli.com/example_assets/hello_zshot/","output_type":"png","skip_robot_instructions":true}' \
  -o page.png