Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Covert Spider Activity
They can hide, but we still see them
8 uncle_bob 2:06 pm Jun 24, 2004
SE Spider ID Forum is *BACK*
11 Brett_Tabke 8:47 am Jun 24, 2004
German National Research
spider is new to me, you?
4 kwasher 4:31 pm Jun 23, 2004
Cowdog Bot
Head 'em up and move 'em out. Rawhide
2 Josk 2:53 pm Nov 4, 2003
alexa
new spider to me
6 dgdclynx 12:08 pm Nov 4, 2003
UA "MSIE 6.0" (and nothing else)
hit the trap, masquerading bot?
12 WebJoe 8:39 am Nov 4, 2003
Cowbot 0.1
5 BlueSky 8:21 am Nov 4, 2003
Is this an invalid UA?
Mozilla/6.0 (compatible; MSIE 6.0; Windows NT 5.2)
7 Wizcrafts 6:39 am Nov 4, 2003
Spider masking as a UCMore toolbar?
"GET /favicon.ico HTTP/1.1" 200 - "-" "UCmore"
21 cyberkat 4:48 am Nov 4, 2003
infomine.ucr.edu
at apnic ip?
6 bull 2:55 am Nov 4, 2003
Can I ban unwanted bots without Apache?
My ISP tells me they run Zeus
5 Michael_Anthony 10:12 pm Nov 3, 2003
66.232.17.78
fell in bot trap...and a question
9 coyote 5:46 pm Nov 3, 2003
Mac Finder / rima-tde.net
Spider from Spanish spam friendly ISP?
2 jmccormac 4:57 pm Nov 3, 2003
vang.net spider 1.6 (Spider 1.7/site@vang.net)
vang.net + soegning.dk - aggressive new spider
2 claus 11:44 am Nov 3, 2003
Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com)
218.145.25.105 (Kornet, Naver)
2 claus 11:21 am Nov 3, 2003
P.Arthur 1.1
Are you Bot or not?
10 Gopi_Sundaram 10:57 am Nov 3, 2003
Blank UA and referer with googlebot IPs
all asking for image files
2 nancyb 6:56 am Nov 3, 2003
206.16.1.x (cnet) killing us
5 papamaku 2:03 am Nov 3, 2003
Expired Domain Sleuth
Nuther gnew won?
5 pendanticist 2:19 pm Nov 2, 2003
How NOT to circulate your resume
(tsk, tsk, tsk.)
4 pendanticist 12:42 pm Nov 2, 2003
38.112.195.45 -- fell in bot trap
3 BlueSky 12:16 pm Nov 2, 2003
Strange Googlebot IP
203.199.124.xxx
5 plasma 6:55 am Nov 2, 2003
216.185.57.106
fell in bot trap
3 coyote 5:44 am Nov 2, 2003
Google log: StripIt 0.2
Who is this?
5 Jesse_Smith 4:31 am Nov 2, 2003
Unknown bot ...
Anyone know who is this?
6 ideavirus 4:21 am Nov 2, 2003