Forum Moderators: open

Message Too Old, No Replies

Any list of bad bots to block on site?

         

born2run

9:15 pm on Sep 28, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hi is there any advise or list of bad bots to block for website? Please advise thanks!

keyplyr

9:26 pm on Sep 28, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hi born2run,

Again, all that info can be found in the Search Engine Spider & User Agent ID Forum [webmasterworld.com]

There is no "list of bad bots" since that so-called list would be different for each website. What may be "bad" for you might be "good" for me.

I would recommend never taking anyone's advice on what to block or allow. Do the research and make that decision yourself.

There is no quick fix to website security. You need to do the leg work yourself. Blocking without full understanding of what you're blocking could have a significant negative affect on your website.

Read here: Blocking Methods [webmasterworld.com]

born2run

10:34 pm on Sep 28, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks! I'm analysing my logs and it seems the most active bots are stuff like SemrushBot and AhrefsBot :-(

keyplyr

11:14 pm on Sep 28, 2017 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Semrush - [webmasterworld.com...]

AhrefsBot - [webmasterworld.com...]


Note: If you publish ads, it may be to your benefit to allow bots that offer products about marketing data or advertising. These products might be used by advertisers who bid for spots on your pages. If you block those bots, the advertisers may never know those ad spots are available.