Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
66.232.17.78
fell in bot trap...and a question
9 coyote 5:46 pm Nov 3, 2003
Mac Finder / rima-tde.net
Spider from Spanish spam friendly ISP?
2 jmccormac 4:57 pm Nov 3, 2003
vang.net spider 1.6 (Spider 1.7/site@vang.net)
vang.net + soegning.dk - aggressive new spider
2 claus 11:44 am Nov 3, 2003
Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com)
218.145.25.105 (Kornet, Naver)
2 claus 11:21 am Nov 3, 2003
P.Arthur 1.1
Are you Bot or not?
10 Gopi_Sundaram 10:57 am Nov 3, 2003
Blank UA and referer with googlebot IPs
all asking for image files
2 nancyb 6:56 am Nov 3, 2003
206.16.1.x (cnet) killing us
5 papamaku 2:03 am Nov 3, 2003
Expired Domain Sleuth
Nuther gnew won?
5 pendanticist 2:19 pm Nov 2, 2003
How NOT to circulate your resume
(tsk, tsk, tsk.)
4 pendanticist 12:42 pm Nov 2, 2003
38.112.195.45 -- fell in bot trap
3 BlueSky 12:16 pm Nov 2, 2003
Strange Googlebot IP
203.199.124.xxx
5 plasma 6:55 am Nov 2, 2003
216.185.57.106
fell in bot trap
3 coyote 5:44 am Nov 2, 2003
Google log: StripIt 0.2
Who is this?
5 Jesse_Smith 4:31 am Nov 2, 2003
Unknown bot ...
Anyone know who is this?
6 ideavirus 4:21 am Nov 2, 2003
YahooSeeker/1.0
19 akogo 3:28 am Nov 2, 2003
da 4.0
2 wilderness 10:12 pm Nov 1, 2003
207.193.137.16
Comes with no UA or referer.
2 addvisors 11:27 am Nov 1, 2003
UA simply "Mozilla"
...and a question
6 coyote 5:41 am Nov 1, 2003
Ocelli/1.0
Hit site 122 times - cannot find in records
2 vincevincevince 1:39 am Nov 1, 2003
sv-fw.looksmart.com
5 addvisors 7:54 pm Oct 31, 2003
Microsoft-WebDAV-MiniRedir/5.1.2600
148,000 hits from the above, but what is it?
4 creepychris 12:30 pm Oct 31, 2003
Scooter
5 Vermont 9:02 am Oct 31, 2003

3 BlueSky 5:55 am Oct 31, 2003
Slurp - Very odd requests
requesting pages that dont exist
5 highman 8:22 am Oct 30, 2003
24.153.166.189
fell in bot trap
2 coyote 2:40 am Oct 30, 2003