Brooklake Tools

Incremental Site Crawl Monitor

Run Cloudflare Browser Rendering crawl jobs from a static frontend while a Worker keeps persistent crawl state in KV. Repeat runs automatically reuse the last successful crawl time as `modifiedSince`.

Crawl Target

Target URLWorker API URL

Worker connection is configured for this site.

Crawl Settings

Page LimitDepthmaxAge (seconds)Source

Use incremental modeReuse the last successful crawl timestamp as modifiedSince on the next start.Render pagesEnable browser rendering for JavaScript-heavy pages.Include subdomainsLet the crawl cross into subdomains when the upstream API allows it.Include external linksAllow the crawl to continue outside the starting host.

Formats

Include PatternsExclude Patterns

Limits

Cloudflare crawl jobs are asynchronous. The Worker starts the job and the UI polls status.
KV holds the last successful crawl timestamp so `modifiedSince` survives refreshes and later runs.
Cloudflare plan limits vary. Start with conservative page limits and poll sparingly.

Ready.

Status

idle

Pages

—

Last Success

Never

Browser Time

—

Crawl State

Current Worker-backed state for the selected crawl target.

idle

Load a target or start a crawl to create persistent state in KV.

Latest Completed Samples

No completed page samples returned yet.

Skipped Samples

No skipped page samples were returned.

Recent Runs

No run history saved for this target yet.