Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Adding content in bulk without penalty

         

Sgt_Kickaxe

6:40 am on Jul 26, 2011 (gmt 0)



It's been discussed here on webmasterworld as far back as 2006...

http://www.webmasterworld.com/google/3070792.htm [webmasterworld.com]

...but I'd like to see a current best practice list with tips from people who've done this recently, along with the result.

I've read the posts, including the one linked above, and although the message from Google seems to be related to getting the new content indexed the issues seem to pop-up for webmasters when that new content impacts existing content. Let me explain...

Internal links are the easiest to get and if you suddenly create 50,000 new pages on a 500 page site, all with the same header and sidebar with respective navigation links, you severely unbalance the internal link structure you are currently ranked with. Checks and balances are in place.

This got me thinking that if I want/need to add bulk content to a site, perhaps an entire new section or new product line etc, I should do so in a way that minimizes the impact to the rest of the site. But how can that be done?

- Making the new content available from a single deep link within the site, at first.
- Easing the new content into your similar pages links slowly.
- Introducing the new content slowly over time.

That first suggestion intrigues me, has anyone added a LOT of content that was only linked from perhaps a single lowly ranking page found deep within your archives? If so... what happened? I suspect the new content didn't rank well but did it cause a traffic reduction or penalty? Getting it all on your domain is a big scary first step.

[edited by: Robert_Charlton at 6:44 am (utc) on Jul 26, 2011]
[edit reason] fixed link [/edit]

goodroi

10:21 am on Jul 26, 2011 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I haven't recently greatly expanded a website but I did want to share another option.

Upload all the content and block it with robots.txt and then gradually unblock. This would give your users immediate access and also address the concerns about potentially shocking Google.

wheel

12:41 pm on Jul 26, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've added 5-10k pages in the last couple of years without adverse affect. One link from the homepage down to all the pages, it was a seperate section of the website.

I don't know that I got any competitive traffic from doing so, but I certainly saw traffic coming in from long tail terms.

Last I recall, Matt Cutts said something like 10K pages should be OK, more than that be careful.

Planet13

6:49 pm on Jul 26, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@ goodroi:


Upload all the content and block it with robots.txt and then gradually unblock.


Is there a reason why robots.txt should be used instead of something else, like a meta noindex tag?

Thanks in advance.

goodroi

8:03 pm on Jul 26, 2011 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



@Planet13

Using meta noindex is another option. I personally would prefer robots.txt since it would prevent Google from even realizing how much content you uploaded. Using meta noindex would allow Google to crawl and see that you just increased your content by 1,000%.

Planet13

8:39 pm on Jul 26, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@goodroi:

Thanks for the reply!