Large Google Index Increase - according to site: operator - Google Search and SEO forum at WebmasterWorld

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Large Google Index Increase - according to site: operator

Joshmc

7:51 pm on Nov 22, 2010 (gmt 0)

Now, off the bat, I realize that the "site:" is not a great recognition of the amount of pages Google actually has indexed for a site. However, I keep track of this as well as other statistics weekly for my site and my competitors. Over the last two weeks I have seen this index increase 10 fold for my site and all the competitor sites I monitor for. With some indexes jumping from 200,000 (large ecommerce sites) to up to 1 million pages in the index. I thought it was a fluke but this is the second week in the a row that the "site:" has returned such high results.

Was wondering what you guys thought about these changes?

tedster

7:56 pm on Nov 22, 2010 (gmt 0)

This sounds like Google has repaired some of the inaccuracies that the site: operator has been showing for a while. In other words, the index does not really include more pages but the site: operator results are not quite so buggy.

Some detail: when the site: operator took a nose dive, I did some research for a few client websites and quickly saw that the drop was just buggy information - many pages were still in the index and drawing traffic, even though site: did not show them any more.

I just spot checked a few website, and it does look like they're back to reasonably accurate levels again.

Joshmc

8:05 pm on Nov 22, 2010 (gmt 0)

Yea I agree it is more in tune with what I see in webmaster tools and landing pages with traffic in analytics. Just thought it was interesting Google would be doing this as I haven't seen them even close to this high in over a year that i have been tracking it. Far be it for Google to allow the site command to provide non buggy information...

Robert Charlton

9:10 pm on Nov 22, 2010 (gmt 0)

At PubCon, I got the strong sense from Matt Cutts' comments that resources have been strained of late as Google has been making major changes, and, as a result, areas of lower priority haven't been getting full attention.

Matt said this with regard to spam in particular, but clearly the effects rippled through a lot of Google's reporting functions. Perhaps this correction is a sign that things will quiet down for a while.

aristotle

2:10 am on Nov 23, 2010 (gmt 0)

areas of lower priority haven't been getting full attention.

Matt said this with regard to spam in particular

Off-topic, but doesn't this imply that the spam problem is low priority for Google?

Robert Charlton

3:28 am on Nov 23, 2010 (gmt 0)

...doesn't this imply that the spam problem is low priority for Google?

No.

It is meant to suggest that Google has more resources available now after a period of some major changes.

Joshmc

4:46 pm on Nov 23, 2010 (gmt 0)

Posted by "Matt Cutts" on SEO round table "Interesting interpretation, the number of urls that are 'indexed' in webmaster tools is the number of URLS that are going to get cached. The URLS that are shown when using the "site:" operator could potentially be old URLS that are not going to be re-indexed. These URLS can be removed in webmaster tools."

tedster

5:00 pm on Nov 23, 2010 (gmt 0)

Thanks for that, joshmc - here's a link to Matt's comment: [seroundtable.com...]

[edited by: Robert_Charlton at 7:09 pm (utc) on Nov 23, 2010]
[edit reason] fixed link [/edit]

SEOPTI

7:25 am on Nov 29, 2010 (gmt 0)

< moved from another location >

Just submitted a fresh sitemap, they seem to definitely report more URLs now in Webmaster Tools:

802,401 submitted
647,067 URLs in web index

[edited by: Robert_Charlton at 8:32 am (utc) on Nov 29, 2010]

Joshmc

7:18 pm on Nov 29, 2010 (gmt 0)

Another interesting thing I just saw, Google is actually showing me a different amount of results with the "site:" search when personalization is on and off.

tedster

8:17 pm on Nov 29, 2010 (gmt 0)

I never tested that - it's a very bizarre result, isn't it?