Does the AdSense bot obey robots.txt?

Forum Moderators: martinibuster

Message Too Old, No Replies

Does the AdSense bot obey robots.txt?

Hillbilly1

3:27 am on Jul 14, 2006 (gmt 0)

I'm a newbe, I have heard that a robot txt will, or might foil adsense from putting relevant links and ads on my pages, is this true?

celgins

3:38 am on Jul 14, 2006 (gmt 0)

If you're blocking the Mediapartners bot, it will.

Other than that, Adsense should place relevant links just fine.

jatar_k

4:15 am on Jul 14, 2006 (gmt 0)

does the mediapartners bot obey robots.txt?

given the nature of it's job it actually shouldn't

I don't actually believe it does, if G made it, then that is rather foolish

It should spider any page with adsense regardless of whether someone doesn't want a page with adsense on it spidered, it should spider it anyway. If the site owner blocks it's IP then ban their account.