Glossary
Crawl budget
The set of URLs Google can and wants to crawl on a site. A real constraint on very large sites, and rarely the reason pages aren’t indexed on small ones.
Crawl budget is the set of URLs Google can and wants to crawl on a site. It comes from two things: the crawl capacity limit, or how much your server can take, and crawl demand, or how much Google wants to fetch.
Google’s guide is written for sites over about a million pages updated weekly, or 10,000 pages changing daily. On a smaller site, a page left out is rarely a budget problem: look at its internal links and its quality first.