Crawl scheduling - the problem of deciding what to crawl, when, and how often - has become a central optimization challenge for modern web data pipelines. In 2025, the explosion of JavaScript-heavy sites, aggressive anti-bot defenses, and increasing compliance requirements means that naive breadth‑first or fixed-interval crawls are no longer viable for serious applications.