What is crawl budget and do I need to worry about it?

Crawl budget is the number of pages / amount of crawling Googlebot (and other search engine bots) allocates to your site over time. It depends on site size, server capacity, and “importance” of pages.

Do you need to worry? Yes, especially for large sites / e‑commerce platforms / sites that publish lots of content. For small sites, seldom.

How to manage crawl budget:

1. Block non‑important pages via robots.txt or noindex (eg. admin pages, tag pages, duplicate content).

2. Use efficient site structure and sitemap.xml to highlight priority URLs.

3. Ensure server responds fast (low response time).

4. Fix broken links and redirect chains.

5. Monitor “Crawl Stats” in GSC.

Further reading:

https://support.google.com/webmasters/community-guide/354590602/why-isn-t-google-indexing-pages-in-discovered-currently-not-indexed-status?

https://www.outrank.so/blog/how-to-fix-crawled-but-not-indexed