Discover every page on your site.
EdgeRank's crawler maps your full site, categorizes pages by type, and feeds the rest of the platform — audits, content analysis, optimizations, and rank tracking.
We crawl from the edge with respect for your robots.txt, sitemap, and rate limits. Pages are classified into types — homepage, product, category, blog post, location, doc — so every downstream view (audit results, content briefs, reports) is grouped in a way that matches how you actually think about your site.
Re-crawls run on a schedule (or on demand) and detect changes since the last run: new pages, removed pages, content updates, redirect chains. You see what changed without re-reading the whole site.
Discovery is the foundation. The Audit surfaces issues per page; the Optimizer applies fixes to selected pages from the crawl. Without an accurate page inventory, the rest of the pipeline misfires.
What's included
- — Full-site discovery with sitemap and link-graph crawls
- — Automatic page-type classification
- — Change detection across re-crawls (new, removed, modified, redirected)
- — Configurable depth, frequency, and exclusion rules
- — Respects robots.txt and rate limits
Start with a free crawl.
See your site mapped end-to-end before you commit to anything.