Do Multiple Sitemaps Damage Indexing?

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Do Multiple Sitemaps Damage Indexing?

Tom_Cash

12:40 pm on Mar 17, 2011 (gmt 0)

Hey folks,
I have recently split our sitemap into two, to allow more products into the Google Index.

However, making a seperate sitemap just for products I have come across an issue...

For example...

GWT used to say:

sitemap.php: 50000 urls submitted and 1000 indexed.

Whereas, the new file now says:

sitemap.php: 150 urls submitted and 150 indexed.
product-sitemap.php: 50000 urls submitted and 0 indexed.

Will splitting our sitemaps impact our indexing at all?

Very worried,
please advise.

Thanks in advance,
Tom.

mark_roach

2:13 pm on Mar 17, 2011 (gmt 0)

I have about 6 or 7 different sitemaps and a sitemap index which references all the sitemaps. I just submit the sitemap index using webmaster tools and gbot has no problem indexing the content in all of my sitemaps.

Tom_Cash

2:41 pm on Mar 17, 2011 (gmt 0)

Awesome, thanks for your input! :D

tedster

2:52 pm on Mar 17, 2011 (gmt 0)

Sitemaps are a supplement to Google's regular crawl, so that report you see does not mean those are the only pages Google is crawling and indexing.

More than that, it is common to see the "0 indexed" message for a new sitemap submission. There seems to be a reporting lag before the crawl of the sitemap shows up in Webmaster Tools.

[edited by: tedster at 2:53 pm (utc) on Mar 17, 2011]

almighty monkey

2:52 pm on Mar 17, 2011 (gmt 0)

Ahhh, that's how you handle sites with big sitemaps. I got asked the other day and, having never ran into that issue yet, really didn't know the answer.

Cheers Mark.

tedster

2:55 pm on Mar 17, 2011 (gmt 0)

Google suggests using a sitemap index file when there are too many URLs for a single xml sitemap. Here's the reference: [google.com...]

netmeg

2:58 pm on Mar 17, 2011 (gmt 0)

Yea I just moved to a sitemap index file with three separate sitemaps, and have more URLs indexed than ever before. I dunno if that's why, but either way, I'm happy with it.

mark_roach

3:08 pm on Mar 17, 2011 (gmt 0)

More than that, it is common to see the "0 indexed" message for a new sitemap submission. There seems to be a reporting lag before the crawl of the sitemap shows up in Webmaster Tools.

I have seen that reporting lag as well. I manually submitted one of my sitemaps recently after I had corrected an error in it. It showed up with 0 pages indexed for about a day before the real number of pages indexed was shown in webmaster tools.

Tom_Cash

3:39 pm on Mar 17, 2011 (gmt 0)

Thanks for some excellent feedback!

Google suggests using a sitemap index file when there are too many URLs for a single xml sitemap. Here's the reference: [google.com...]

Yea I just moved to a sitemap index file with three separate sitemaps, and have more URLs indexed than ever before. I dunno if that's why, but either way, I'm happy with it.

Can anyone else confirm this?

indyank

4:48 pm on Mar 17, 2011 (gmt 0)

netmeg, how many urls do you maintain in a sitemap?

indyank

4:52 pm on Mar 17, 2011 (gmt 0)

Tom-cash, tedster had posted a link to a google page. Do you need any other confirmation? ;)

sitemap index file is the only way to handle sitemaps for large sites. Google suggests 50K urls per sitemap.

[edited by: indyank at 4:53 pm (utc) on Mar 17, 2011]

netmeg

4:52 pm on Mar 17, 2011 (gmt 0)

Nowhere near the limit. We're just using different technologies across the sites, so we don't have a "one size fits all" sitemap solution. So we have to aggregate.

zerillos

12:14 am on Mar 18, 2011 (gmt 0)

It seems good practice to me to use sitemaps and a sitemap index for scalability issues. If you get to have too many pages for a single sitemap to handle, you won't have to make the change then.

tedster

12:45 am on Mar 18, 2011 (gmt 0)

If you take care with the technical accuracy of the sitemap URLs, it provides another quick check on your indexation level - and that's a slippery number for sure.

In fact, it's so slippery that I don't lean on it very hard. Pages that get search traffic can be a much more actionable metric.

Tom_Cash

2:48 pm on Mar 18, 2011 (gmt 0)

Tom-cash, tedster had posted a link to a google page. Do you need any other confirmation? ;)

Sorry, I should be more clear with my message...

I meant, can anyone else confirm that using a index file will result in more of your pages being indexed?

Some interesting reading so far, however.

netmeg

4:01 pm on Mar 18, 2011 (gmt 0)

I don't think anybody can guarantee that; again, anecdotal evidence. I've just seen it work over more than one site, but that doesn't mean a sitemap index is the reason.

I'm pretty sure it wouldn't *hurt* you, though, so there's no reason really not to do it, if you're inclined.

zerillos

2:29 am on Mar 19, 2011 (gmt 0)

I meant, can anyone else confirm that using a index file will result in more of your pages being indexed?

i don't think there's any connection between the sitemap and the number of indexed pages. G says the sitemap helps them find more pages. they don't say they're also going to get indexed.

Tom_Cash

9:55 am on Mar 21, 2011 (gmt 0)

Cheers guys. :)

spiritparse

4:04 pm on May 27, 2011 (gmt 0)

The "indexed" count always starts at 0 when deleting/adding new sitemap. The count is really "URLs in submitted sitemap verified to be index".