Forum Moderators: open

Message Too Old, No Replies

semrush/whorush

web marketing tools

         

Dijkgraaf

1:03 am on Oct 29, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



UA: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3

IP: 88.214.193.166
rDNS: [No reverse DNS entry per ns1.ipipe.net.]
Robots.txt: No

Referrers: (my domain replaced with example.com and space added)
http:// www.semrush.com/info/example.com
http:// www.whorush.com/search/?q=example.com

Behaviour: Only requests root page

A I haven't subscribed to their marketing tools, they don't request robots.txt and don't declare themselves in the UA I consider it a bot.

jdMorgan

2:17 am on Oct 29, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Seen them too. Same IP address, same behavior.

Since it was the exact same IP address, this makes it unlikely that we are seeing a user of the service checking out our sites, as opposed to the service's own 'bot. The chances of someone being interested in your site and mine are pretty low, no matter what our sites are about...

The HTTP Connection, Accept, Accept-Language, Accept-Encoding, and Accept-Charset headers are incorrect for Firefox.

Jim

Pfui

4:39 am on Oct 29, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Same thing here on at least two different days on two disparate sites -- same UA, same bot-runner IP, same log-spamming refs containing the sites' names, etc. Also no robots.txt, no images, no nothing but root.

FWIW, SEMRush is a competitor research-checker and its domain is registered to a Russia-based SEO endeavor. The bot-runner IP hails from a UK-based server farm (Hqhost), ditto kin SeoQuake, SeoDigger, SeoPivot, and SEMIsland. WHORush and PicsDigger are also connected to the same data mining/selling mix, and who knows how many more.

ANYway, all of the above earned 88.214.193.166 a spot in my firewall's killfile and 403s for the self-refs.

(I'm weary of mod_rewrite-403'ing bot-runners only to see them try and try again anyway.)