Crawl Budget Is Why Your 200-Post Shopify Blog Has 40 Pages Indexed

Why does your Shopify blog have 200 posts but only a fraction indexed?

A crawl budget shopify blog problem is almost always the culprit. Googlebot gives every site a finite number of page visits per day, and on Shopify that budget gets quietly consumed by duplicate product URLs, filter pages, internal search results, and tag archives before it ever reaches your blog section. The posts are published, they're in your sitemap, but Googlebot is busy elsewhere. The good news: each of the causes is fixable without touching your products or collections.

What crawl budget actually means for a Shopify store

Two factors shape how many of your pages Googlebot visits in any given period. The first is the crawl rate limit — how fast Googlebot can fetch pages without overloading your server. The second is crawl demand — how much Google actually wants to visit a given page, based on its popularity and how recently it changed.

On Shopify specifically, the crawl budget problem is more acute than on other platforms because of how the platform generates URLs. Shopify serves every product at two separate URL paths: /products/product-handle and /collections/collection-name/products/product-handle. A store with 500 products across 10 collections can generate up to 5,500 crawlable product URLs when only 500 are unique. Shopify does add canonical tags, but Googlebot still visits both paths in many cases, consuming crawl budget that could otherwise reach your blog.

The result is predictable: blog posts sit in Google Search Console under "Discovered, currently not indexed," meaning Google found them in the sitemap but hasn't gotten around to actually fetching them yet.

How to confirm your blog has a crawl budget problem

Before touching anything, spend five minutes confirming the diagnosis.

• Check Crawl Stats in Google Search Console. Go to Settings in the left sidebar and open the Crawl Stats report. It shows total crawl requests per day, average response time, and which file types Googlebot is hitting. A flat or declining pages-crawled-per-day trend while your blog grows is a clear signal.
• Run a site: search for your blog path. Type site:yourstore.com/blogs/ into Google. The count of results is roughly how many blog posts are currently indexed. Compare that number to your published post count in Shopify admin.
• Filter the Pages report by blog URLs. In Search Console, open the Pages (Index Coverage) report and filter by your /blogs/ path. Posts showing "Discovered, currently not indexed" indicate a crawl prioritization issue. Posts showing "Crawled, currently not indexed" have a content quality issue instead.

It matters which status you're looking at, because the fix is completely different. "Discovered, not indexed" means Googlebot hasn't visited yet (crawl budget). "Crawled, not indexed" means Googlebot visited and decided the content didn't meet its quality threshold for storage (quality). This guide focuses on the crawl budget side, but if you're seeing a mix of both, check our step-by-step indexing diagnosis guide for the quality side as well.

Cause 1: The duplicate product URL problem (most common)

When Shopify renders a product card inside a collection, the default Liquid theme code often generates links using the within: collection filter, creating /collections/tops/products/blue-t-shirt instead of the canonical /products/blue-t-shirt. Googlebot follows those links, discovers the duplicate paths, and crawls them, often instead of your blog posts.

The fix is a theme-level edit. Locate your product-card.liquid or product-grid-item.liquid snippet and remove the within: collection filter from product anchor tags. This forces every internal link on collection pages to point strictly to the canonical /products/ URL, permanently closing the duplicate routing path.

Cause 2: Internal search pages burning crawl budget

Shopify's native search at /search?q= generates a unique URL for every query typed on your store. These pages have no ranking value. They're dynamically generated, session-specific, and produce infinite URL variants. Yet Shopify's default robots.txt does not block the /search path, so Googlebot visits them freely.

The fix is a single line in your theme's robots.txt.liquid file:

Disallow: /search

Add this to the Googlebot section. It takes effect within a few crawl cycles and stops what can amount to hundreds or thousands of wasted crawl requests per week.

Cause 3: Blog tag archive pages diluting crawl attention

Every tag you assign to a Shopify blog post creates a unique archive URL at /blogs/blog-name/tagged/your-tag. If you have 200 posts tagged across 30 tags, that's 30 extra pages Googlebot has to evaluate. Those pages contain no unique editorial content, just a filtered list of posts it could find via the sitemap anyway.

If those tag pages get indexed, they can dilute crawl attention and make your blog look like it has a high proportion of low-unique URLs. Adding a conditional noindex meta tag to tag archive views in your blog.liquid template removes these pages from contention. The tag navigation can stay on your site for users, but it just shouldn't be indexed.

Worth reading alongside this: our article on pruning weak posts to lift the whole site covers how cleaning up low-quality URLs — including tag archives — tends to improve indexing rates for the content you want ranked.

Cause 4: Orphaned blog posts with no internal links

Googlebot discovers pages by following links. A blog post that has no internal links pointing to it from other pages on your site is essentially invisible to Googlebot's link-following behavior. Even if it's in your sitemap, Googlebot treats sitemap-only URLs as lower priority than pages discovered through internal link traversal.

For every new post you publish, make sure at least two existing posts or pages link to it. When you update an older, high-authority post, scan it for natural places to add links to newer related articles. This is also why your blog index page, the page that lists all posts, matters: if it's buried in your navigation or has no links pointing to it, the crawl signal flowing through it is weak.

Internal linking also directly helps with the kind of long-tail content that Shopify blogs are well-positioned to rank for — those posts often need link equity from stronger pages to get crawled and indexed reliably.

Cause 5: Redirect chains consuming crawl slots

Every redirect Googlebot follows uses a crawl slot. A chain like 301 → 301 → 200 uses two slots to reach one page. On a store that has accumulated years of product updates, URL changes, and app installations, redirect chains are common. Use Screaming Frog (free up to 500 URLs, as of October 2026) to identify chains, then update internal links to point directly to the final destination URL rather than through intermediaries.

Cause 6: Slow page speed capping your crawl rate limit

Google's crawl rate limit scales with how fast your server responds. Pages loading under 200ms earn more aggressive crawling than pages averaging 800ms or more. Third-party Shopify apps that inject scripts on every page, such as live chat widgets, review carousels, and upsell popups, add load time that directly shrinks your crawl budget. Compress images, defer non-critical JavaScript, and audit which app scripts load globally versus only on the pages that need them.

Cause 7: Thin or near-duplicate blog posts

When Googlebot visits a blog post and decides not to index it, that visit still consumed crawl budget. If you've published short posts under 500 words, posts that largely repeat product descriptions, or posts on nearly identical topics, Googlebot will crawl them and then decline to store them, costing you crawl budget with no return.

In Search Console, filter the Pages report for "Crawled, currently not indexed" within your blog path. These posts are costing you crawl slots while delivering nothing. The options are to substantially expand and differentiate them, merge overlapping posts into a single stronger article, or remove them entirely and redirect to a related post. Each of these outcomes improves the average quality signal across your blog and frees future crawl budget for posts that meet Google's threshold for indexing.

If you're publishing at scale with AI tools, this is the most common failure mode. Quantity without quality leaves Googlebot less willing to crawl the whole blog section promptly. See our indexing diagnosis guide for a full triage workflow.

Fixing your sitemap so it guides, not misleads, Googlebot

Your sitemap should function as a curated list of pages you want indexed, not an exhaustive dump of every URL the store has ever generated. Visit yourstore.com/sitemap.xml and confirm that sitemap_blogs_1.xml is listed in the sitemap index. If your posts are split across multiple blog sections in Shopify admin, there may be multiple blog sitemaps. Check that all are present.

Open the blog sitemap file directly and scan for:

• Tag archive URLs that have crept in
• URLs returning 404 or carrying noindex tags
• Draft or hidden posts that were temporarily published
• Posts you've since deleted without redirecting

A sitemap that lists unindexable URLs sends a confusing signal to Google and can slow down its processing of the valid entries. The sitemap tells Googlebot where to go; your robots.txt tells it where not to go. Both need to point in the same direction.

How to monitor crawl budget on an ongoing basis

Crawl budget optimization is not a one-time project. As your Shopify store grows, with new products, new collections, and new apps, new crawl drains appear. Set a calendar reminder to run a quarterly audit:

• Check Crawl Stats in Search Console for any increase in total requests without a corresponding increase in indexed pages (that gap is crawl waste growing).
• Run a fresh Screaming Frog crawl to catch new duplicate URL patterns introduced by recently installed apps.
• Review the "Discovered, not indexed" count in the Pages report for your blog path. If it's growing faster than your published post count, something new is competing for crawl budget.

A useful benchmark: when your "days to full crawl" metric drops below seven days, Googlebot can efficiently discover and process every page on your site within a week of publication. At that point, slow indexing shifts from a budget problem to a content quality problem, and content quality is entirely within your control.

A note on crawl budget and content publishing cadence

If you're publishing several posts a week, especially with AI assistance, the crawl budget shopify blog problem can compound quickly. Each thin or unfocused post adds a URL Googlebot must evaluate, and each evaluation that ends in "not indexed" is a small vote of reduced confidence in your blog section's overall quality.