Many websites have a long list under "Crawled - currently not indexed" in the Search Console Page indexing report. The status sounds like a technical error, but it is not one. Google fetched the page, looked at it and decided not to add it to the index for now. So the question is not how to get Google to crawl the page. The question is why Google did not want it after the visit, and whether you should change that.
After reading, you can sort the affected URLs into groups, name the likely cause for each group and decide whether to improve, consolidate, exclude or remove it.
What the status means and what it does not
Google describes the status briefly: the page was crawled but not indexed. It may or may not be indexed in the future, and there is no need to resubmit it. That is what the Page indexing report help says.
Two things follow from this. First, crawling is done. Robots.txt, server errors or a blocked URL are usually not the explanation here, otherwise the page would appear under a different reason. Second, Google made an assessment. Indexing is not something every reachable URL is entitled to. Google explicitly does not guarantee that all pages of a site will make it into the index.
And what does "Discovered - currently not indexed" mean?
The second status in the same list looks similar but has a different cause. Google knows the URL, for example from the sitemap or a link, but has not fetched it yet. According to Google, the crawl was usually rescheduled because it would otherwise have overloaded the site.
On small sites this status often clears up on its own. If it persists for weeks or affects many pages, look at four things:
- Too many unimportant URLs. Filters, parameters and generated pages use up crawling capacity that important pages then lack. The glossary covers this under crawl budget.
- Slow server responses. If the server responds slowly under load, Google crawls more cautiously.
- Weak internal linking. Pages that only appear in the sitemap have little priority for Google.
- An untidy sitemap. If it contains redirected, broken or unimportant URLs, it loses value as a signal.
So "Discovered" is about priority and capacity in crawling. "Crawled" is about the assessment of the page itself. The rest of this guide focuses on the second case.
Sort first: is this a problem at all?
Before you change anything, look at which URLs are affected. Often a large share of them belongs outside the index:
- URLs with parameters from filters, sorting or tracking
- paginated archive pages and thin tag or author pages
- internal search result pages
- feeds, print versions and similar technical variants
These pages do not need to rank. If Google leaves them out, the assessment is working.
It only becomes critical for URLs you consider important yourself: product and category pages, guides, landing pages, new articles. Keep in mind that the list in the report is limited to 1,000 rows and does not necessarily show every affected URL. On larger sites it is a sample, not a complete inventory.
Export the list and assign every URL to a group, ideally by directory or page type. Three hundred individual URLs become five or six groups, and you make one decision per group. Fixing URLs one by one takes time and rarely addresses the real cause.
The most common causes
Google does not give a reason per URL. You have to infer the cause from the pattern. In practice, affected page groups almost always fall into one of five categories.
The content offers too little of its own. Pages with little text, bare product lists without context, generated descriptions or content that already exists elsewhere on the site in better form stand out here. This is often called thin content. What matters is not the word count but whether the page answers a query better than the pages already in the index.
The page is too similar to another one. Variants of the same product page, location pages with only the city name swapped, or two articles on the same topic create duplicate content. Google picks one version and leaves the other. Check in URL Inspection which canonical Google selected and whether it matches the one in your canonical tag.
The page is poorly connected internally. A URL that is only reachable through the XML sitemap or deeply nested archive pages looks unimportant to Google. Few internal links, a high click depth and no links from topically related pages are a common pattern. The glossary entry on internal linking explains the logic behind it.
The technical signals contradict each other. A page is listed in the sitemap but its canonical points to another URL. Or it was set to noindex for a while. Or the main content only appears after JavaScript runs and is missing from the rendered HTML. A crawl makes these contradictions visible, especially when JavaScript rendering is involved.
The page is new or demand is low. Freshly published pages sometimes show this status first and get indexed later. The same applies to topics with very little search demand. Patience often helps here, as long as the page is well linked.
Step by step to a decision
Work through the groups from your export like this.
- Inspect a sample. Take three to five URLs from each group and run a live test in URL Inspection. Look at the rendered HTML, the Google-selected canonical and whether the main content appears in the rendered code at all.
- Assess the page honestly. Would you want to see this page as its own search result? Is there another page on the site that answers the same question better? This question settles more than any metric.
- Check the technical signals. Compare canonical, meta robots, status code, redirects and the entries in your XML sitemap. The sitemap should only list URLs you want indexed.
- Count internal links. How many internal links point to the page, and from where? An important page that is only linked from the footer or an archive page needs links from relevant content.
- Decide per group. There are four options, described in the next section.
- Start validation and wait. Once the changes are live, start validation in the report. Google checks a few URLs first and then works through the list. This typically takes up to about two weeks, sometimes longer.
Four options for every page group
Improve. The page should rank but offers too little. Add what searchers need: concrete answers, comparisons, examples, your own data or experience. Then improve internal links from relevant pages. More text alone does not help if it says nothing new.
Consolidate. Two pages compete for the same topic. Merge them into one stronger page and redirect the weaker one with a 301. For technical variants that have to stay, a clean canonical to the main version is enough.
Exclude on purpose. Users need the page, search does not. Set it to noindex, remove it from the sitemap and link it internally only where users need it. That cleans up the report and nobody wastes time on these URLs again.
Remove. The page serves neither users nor search. Delete it and return a 404 or 410, or redirect it to a relevant page if a genuine equivalent exists.
What does not help
Some reactions feel productive but do not change the assessment:
- Requesting indexing again and again. The feature has a quota and does not replace improvement. Use it selectively for individual pages you have actually reworked.
- Resubmitting the sitemap. Google already knows the URLs. A resubmitted sitemap does not change the quality question.
- Using the Indexing API. Google only allows the Indexing API for pages with job postings or livestream videos. It is the wrong tool for regular content pages.
- Padding the text. Filler does not make a page more useful. If anything it makes reading worse.
How a crawl shortens the search for causes
Search Console tells you that a page is not indexed. It does not tell you why. This is where a crawl of your own site helps, because it puts every signal of a URL side by side: status code, meta robots, canonical, click depth, number of internal links and the amount of content.
With Crawl Foundry Site Audit you crawl the site and filter the affected directories. Findings such as noindex, conflicting canonicals, thin content, missing internal links in body copy or redirect chains appear per URL and per page group. You can quickly see whether a whole group shares the same pattern, such as category pages with very little text of their own. The URL detail page in the documentation shows what evidence is collected for each page.
After the changes, you compare a new crawl with the previous one and see which findings have actually disappeared. Our article on technical SEO issues in Site Audit describes how to put crawl findings into a useful order.
A practical example
An online shop finds several hundred URLs with this status in the report. After sorting, three groups remain: filter URLs with parameters, tag pages of the magazine and about forty second-level category pages.
The filter URLs get a canonical to the unfiltered category and are removed from the sitemap. The tag pages are set to noindex because they only list articles. For the category pages, the crawl shows two patterns: hardly any text of their own above the product list and only one internal link from the navigation. The team writes a short, helpful introduction with buying guidance for each category, links the categories from relevant guides and then starts validation.
Google still decides which categories get indexed. But the list now only contains pages that are worth waiting for, and every other page has been decided on purpose.
From a long list to a clear decision per URL group
"Crawled - currently not indexed" is Google's verdict on a page, not a crawl error. Sort the affected URLs into groups, assess a sample honestly and decide per group: improve, consolidate, exclude or remove. Comparing the reasons in the Page indexing report with a crawl of your own saves you weeks of guesswork.
Start by exporting the affected URLs from the Page indexing report and grouping them by template or directory.