Compare
SiteRune vs Botify, Oncrawl, and enterprise log crawlers
· Configuration notes, not a newsroom.
Enterprise crawlers ingest logs and millions of URLs. SiteRune is a senior-engineer brief for a host you paste today. Different budget, different output.
BOScale
Botify and Oncrawl are how a retailer watches crawl budget across a million SKUs. SiteRune walks a handful of URLs on purpose. If you need log-based crawl budget, you are not our customer for that job.
What they rarely attach
A paste-ready CSP, a WordPress mu-plugin, and an llms.txt draft in the CMS dialect. Use the enterprise crawler for the estate. Use SiteRune for the origin the brand actually ships this week — including AEO, which log crawlers still treat as a footnote.
Capability table
| Capability | SiteRune | Botify / Oncrawl |
|---|---|---|
| Log-based crawl budget | No | Yes |
| Million-URL estates | No | Yes |
| Hosted brief for one origin today | Yes | Heavy setup |
| AEO + paste-ready files | Yes | Uncommon |
FAQ
Does SiteRune replace botify?
No. Use the comparison table. SiteRune is a hosted configuration brief with CMS dialect and ship files. It does not replace rank tracking, backlink graphs, lab Core Web Vitals, or a desktop site-wide crawl.
What does a SiteRune scan actually fetch?
HTML and headers from this vantage point, robots.txt, sitemap, llms.txt, security.txt, and a polite walk of same-origin interiors. No headless Chrome. No exploits.
Run it on a live URL
Same scan engine. Guest scans stay free.