Hiding ad pages from bots

Forum Moderators: open

Message Too Old, No Replies

Hiding ad pages from bots

codeman

7:38 pm on Sep 24, 2008 (gmt 0)

I have developed a web application which will conditionally show ads if the url is suffixed by a "/1". For example:

http://www.example.com/product/baby-gift/1

If a visitor enters the site this way, a session variable is set that will cause all pages to show these ads, even if there is no "/1" appended.

At the same time that the ads are shown, I have added this:

The goal is to not have search engines index pages with ads, so that those pages will not have a canonical problem with duplicate content.

So, my question is, is this a fool-proof way to make sure that Google, and other search engines, can never index pages that show ads, and that we will not risk duplicate content penalties?

Ajaxunion

4:47 am on Sep 26, 2008 (gmt 0)

Using javascript for the ads may help sometimes. Look how google adsense works. Doesnt get indexed as content. FYI i have seen yahoo index google ads! and sometimes google indexes yahoo ads. But that will also depend on how they are being displayed. Some webmasters have found a way to make the JS ads appear as HTML and get indexed but thats a different topic.