Crawled — Currently Not Indexed: A Coverage Status Guide

Posted by cml6 3

Google’s Index Coverage report is absolutely fantastic because it dedicates SEOs clearer penetrations into Google’s crawling and indexing decisions. Since its roll-out, we use it almost daily at Go Fish Digital to diagnose technical issues at proportion for our clients.

Within the report, there are many different “statuses” that support webmasters with informed on how Google is handling their site content. While many of the ranks cater some situation around Google’s crawling and indexation decisions, one was uncertain: “Crawled — currently not indexed”.

Since seeing the “Crawled — currently not indexed” status reported, we’ve heard from various site proprietors querying about its meaning. One of the benefits of working at an agency is being able to get in front of a lot of data, and because we’ve seen this message across numerou chronicles, we’ve begun to pick up on vogues from reported URLs.

Google’s description

Let’s start with the official definition. According to Google’s official documentation, this status makes: “The page was crawled by Google, but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling.”

So, essentially what we know is that:

Google is able to access the pageGoogle made time to crawl the pageAfter crawling, Google decided not to include it in the index

The key to understanding this status is to think of reasons why Google would “consciously” decide against indexation. We know that Google isn’t having trouble finding the sheet, but for some reason it feels customers wouldn’t benefit from finding it.

This can be quite frustrating, as you might not know why your content isn’t getting indexed. Below I’ll detail some of the most common rationales our squad has met to explain why this mysterious status might be affecting your website.

1. False positives Priority: Low

Our first step is to always perform a few spot checks of URLs flagged in the “Crawled — currently not indexed” section for indexation. It’s not uncommon to find URLs that are getting reported as omitted but turn out to be in Google’s index after all.

For example, here’s a URL that’s getting signalled in the report for our website: https :// meetup/

However, when using a site rummage hustler, we can see that the URL is actually included in Google’s index. You can do this by appending the verse “site: ” before the URL.

If you’re understand URLs reported under this status, I recommend starting by using the site search operator to determine whether the URL is indexed or not. Sometimes, these turn out to be false positives.

Solution: Do nothing! You’re good. 2. RSS feed URLs Priority: Low

This is one of the most common patterns that we insure. If your locate exercises an RSS feed, you might be finding URLs appearing in Google’s “Crawled — currently not indexed” report. Many times these URLs will have the “/ feed/ ” string appended to the end. They can appear in the report like this:

Google finding these RSS feed URLs related from the primary sheet. They’ll often be linked to using a “rel= alternate” point. WordPress plugins such as Yoast can automatically make these URLs.

Solution: Do nothing! You’re good.

Google is likely selectively choice not to indicator these URLs, and for the right reasons. If you steer to an RSS feed URL, you’ll consider an XML document like the one below:

While this XML document is useful for RSS feeds, there’s no need for Google to include it in the index. This would support a very poor experience as the content is not intended for users.

3. Paginated URLs Priority: Low

Another extremely common reasonablenes for the “Crawled — currently not indexed” exclusion is pagination. We will often receive a good number of paginated URLs appear in this report. Here we can see some paginated URLs performing from a very large e-commerce site 😛 TAGEND

Solution: Do nothing! You’re good.

Google will need to crawl through paginated URLs to get a terminated crawling of the site. This is its pathway to content such as deeper category pages or product description pages. Nonetheless, while Google uses the pagination as a pathway to access the content, it doesn’t necessarily need to index the paginated URLs themselves.

If anything, make sure that you don’t do anything to impact the creeping of the individual pagination. Ensure that all of your pagination contains a self-referential canonical labeland is free of any “nofollow” tags. This pagination acts as an avenue for Google to crawl other key pages on your area so you’ll surely require Google to continue crawling it.

4. Expired commodities Priority: Medium

When spot-checking individual sheets that are listed in the report, a common question we assure across patrons is URLs that contain text remark “expired” or “out of stock” products. Peculiarly on e-commerce areas, it appears that Google checks to see the availability of a particular product. If it having determined that a produce has no access, it continues to eliminate that make from the indicator. This induces smell from a UX perspective as Google might not want to include material in the index that users aren’t able to purchase.

However, if these products are actually available on your place, this could result in a lot of missed SEO opportunity. By excluding the pages from the index, your content isn’t given a chance to rank at all.

In addition, Google doesn’t really check the visible content on the page. There have been instances where we’ve discovered no clue within the noticeable content that the produce has no access. However, when checking the structured data, we can see that the “availability” property is set to “OutOfStock”.

It appears that Google is taking evidences from both the observable content and structured data about a particular product’s availability. Thus, it’s important that you check both the contents and schema.

Solution: Check your inventorying availability.

If you’re obtain products that is really available getting listed in this report, you’ll want to check all of your makes that may be incorrectly rolled as unavailable. Act a creep of your site and use a custom extraction tool like Screaming Frog’s to rub data regarding your make pages.

For instance, if you want to see at proportion all of your URLs with schema set to “OutOfStock”, you can provided the “Regex” to: “availability”: “

Read more: