# --crawl-delay

Source: https://zshot-cli.com/reference/flags/archive/crawl-delay/
Markdown: https://zshot-cli.com/reference/flags/archive/crawl-delay/index.md
Section: [Reference](https://zshot-cli.com/reference/) › [Flags](https://zshot-cli.com/reference/flags/) › [Archive](https://zshot-cli.com/reference/flags/archive/)
CLI: `--crawl-delay` · REST: [`crawl_delay`](https://zshot-cli.com/reference/server/api/schema/#crawl_delay)
Value: `SECS` · Default: `1`
License: [Pro](https://zshot-cli.com/support/licenses/pro)

Requires page loads on one host wait at least this many seconds, across every browser of a link crawl or sitemap walk. `0` loads pages without any delay.

A host whose robots.txt asks for a longer `Crawl-delay` is paced by that instead, with a warning. With an Enterprise license, a `--crawl-delay` is followed even where robots.txt asks for longer.

## Example

**CLI**

```sh
zshot -f house-at-pooh-corner.wacz \
  --crawl-delay 3 \
  https://zshot-cli.com/example_assets/house-at-pooh-corner/sitemap.xml
```

**JSON** — `POST /` body for the [API](/reference/server/api), Pro license

```json
{
  "url": "https://zshot-cli.com/example_assets/house-at-pooh-corner/sitemap.xml",
  "output_type": "wacz",
  "crawl_delay": 3
}
```

**curl**

```sh
curl -X POST http://127.0.0.1:3000/ \
  -H "Content-Type: application/json" \
  -d '{"url":"https://zshot-cli.com/example_assets/house-at-pooh-corner/sitemap.xml","output_type":"wacz","crawl_delay":3}' \
  -o house-at-pooh-corner.wacz
```

## Related

Flags:

- [`--crawl-concurrency`](https://zshot-cli.com/reference/flags/archive/crawl-concurrency/)
- [`--crawl-links`](https://zshot-cli.com/reference/flags/archive/crawl-links/)
- [`--crawl-url-is-sitemap`](https://zshot-cli.com/reference/flags/archive/crawl-url-is-sitemap/)
- [`--skip-robots-txt`](https://zshot-cli.com/reference/flags/browser/skip-robots-txt/)

Topics:

- [robots.txt on Wikipedia](https://en.wikipedia.org/wiki/Robots.txt)

## Navigation

- [Examples](https://zshot-cli.com/examples/)
- [Download](https://zshot-cli.com/download/)
- [Purchase](https://zshot-cli.com/purchase/)
- [Reference](https://zshot-cli.com/reference/)
  - [Formats](https://zshot-cli.com/reference/formats/)
  - [Flags](https://zshot-cli.com/reference/flags/)
    - [Common](https://zshot-cli.com/reference/flags/common/)
    - [Agents](https://zshot-cli.com/reference/flags/agents/)
    - [Archive](https://zshot-cli.com/reference/flags/archive/)
      - [--archive-body-size-limit](https://zshot-cli.com/reference/flags/archive/archive-body-size-limit/)
      - [--archive-cdx](https://zshot-cli.com/reference/flags/archive/archive-cdx/)
      - [--archive-spill-max-bytes](https://zshot-cli.com/reference/flags/archive/archive-spill-max-bytes/)
      - [--crawl-allow-url](https://zshot-cli.com/reference/flags/archive/crawl-allow-url/)
      - [--crawl-concurrency](https://zshot-cli.com/reference/flags/archive/crawl-concurrency/)
      - **--crawl-delay**
      - [--crawl-deny-url](https://zshot-cli.com/reference/flags/archive/crawl-deny-url/)
      - [--crawl-link-depth](https://zshot-cli.com/reference/flags/archive/crawl-link-depth/)
      - [--crawl-link-selector](https://zshot-cli.com/reference/flags/archive/crawl-link-selector/)
      - [--crawl-links](https://zshot-cli.com/reference/flags/archive/crawl-links/)
      - [--crawl-max-links](https://zshot-cli.com/reference/flags/archive/crawl-max-links/)
      - [--crawl-media-sources](https://zshot-cli.com/reference/flags/archive/crawl-media-sources/)
      - [--crawl-page-timeout](https://zshot-cli.com/reference/flags/archive/crawl-page-timeout/)
      - [--crawl-sitemap-max-depth](https://zshot-cli.com/reference/flags/archive/crawl-sitemap-max-depth/)
      - [--crawl-url-is-sitemap](https://zshot-cli.com/reference/flags/archive/crawl-url-is-sitemap/)
      - [--har](https://zshot-cli.com/reference/flags/archive/har/)
      - [--har-capture-bodies](https://zshot-cli.com/reference/flags/archive/har-capture-bodies/)
      - [--har-captures-navigation](https://zshot-cli.com/reference/flags/archive/har-captures-navigation/)
      - [--warc](https://zshot-cli.com/reference/flags/archive/warc/)
      - [--warc-captures-navigation](https://zshot-cli.com/reference/flags/archive/warc-captures-navigation/)
      - [--warc-no-gzip](https://zshot-cli.com/reference/flags/archive/warc-no-gzip/)
    - [Browser](https://zshot-cli.com/reference/flags/browser/)
    - [Diagnostics](https://zshot-cli.com/reference/flags/diagnostics/)
    - [Document](https://zshot-cli.com/reference/flags/document/)
    - [Image](https://zshot-cli.com/reference/flags/image/)
    - [Navigator](https://zshot-cli.com/reference/flags/navigator/)
    - [Security](https://zshot-cli.com/reference/flags/security/)
    - [Server](https://zshot-cli.com/reference/flags/server/)
    - [Video](https://zshot-cli.com/reference/flags/video/)
  - [Agents](https://zshot-cli.com/reference/agents/)
  - [Configuration](https://zshot-cli.com/reference/configuration/)
  - [Crawling](https://zshot-cli.com/reference/crawling/)
  - [Docker](https://zshot-cli.com/reference/docker/)
  - [Exit Codes](https://zshot-cli.com/reference/exit_codes/)
  - [Linux](https://zshot-cli.com/reference/linux/)
  - [LLM Connections](https://zshot-cli.com/reference/llm_connections/)
  - [LLM Formatting](https://zshot-cli.com/reference/llm_formatting/)
  - [Local Inputs](https://zshot-cli.com/reference/inputs/)
  - [Navigator](https://zshot-cli.com/reference/navigator/)
  - [Security](https://zshot-cli.com/reference/security/)
  - [Server](https://zshot-cli.com/reference/server/)
  - [Shell completions](https://zshot-cli.com/reference/shell_completions/)
  - [Storage](https://zshot-cli.com/reference/storage/)
  - [Timeouts](https://zshot-cli.com/reference/timeouts/)
  - [Web Bot Auth](https://zshot-cli.com/reference/web_bot_auth/)
- [Support](https://zshot-cli.com/support/)
- [Changelog](https://zshot-cli.com/changelog/)

Site index: https://zshot-cli.com/llms.txt