Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Maybe a spider? "Crawl_Application"
Has not been brought up for a few months.
2 werty 5:48 pm Jul 9, 2003
GornKer Crawler - Anyone know what it does?
It did 1600 pageviews on my site in a week!
3 ir8Rtist 5:11 pm Jul 9, 2003
FYI: New Googlebot IPs
5 Marcia 1:11 pm Jul 9, 2003
Net Depot, Inc.
216.180.241.154
3 photoman 12:55 am Jul 9, 2003
UA Change in 12 seconds
4 photoman 10:37 pm Jul 7, 2003
Under the Rainbow?
13 SeanL 8:56 pm Jul 7, 2003
Batik/1.0
Floral
4 DavidT 6:42 pm Jul 7, 2003
GalaxyBot
I have never seen this before
8 quotations 3:26 pm Jul 7, 2003
The Big List
Identify the good guys...
3 hanuman 9:09 am Jul 7, 2003
217.155.46.193
3 wilderness 6:50 am Jul 7, 2003
Seeker.lookseek.com
Seeker.lookseek.com
4 keyplyr 12:39 am Jul 7, 2003
Scooter
Not Good
14 wilderness 9:05 pm Jul 6, 2003
what spider is this!
and what is it seeing
4 madmax1 5:46 pm Jul 6, 2003
alta vista 24 hr update spider?
2 chompy 4:57 pm Jul 6, 2003
MediaPartners
JavaScript
6 DavidT 3:15 am Jul 5, 2003
New network?
64.68.88.134
4 pontifex 6:50 pm Jul 4, 2003
Banning IP for Tripod / Lycos sites
Banning IP for Tripod / Lycos sites
7 HardKnox 3:21 pm Jul 4, 2003
New Crawler?
Today a lot of visits from 66.123.16.67
3 angiolo 1:37 pm Jul 4, 2003
Dirty Agent Tricks
How to deny
3 nuhkweb 3:15 am Jul 4, 2003
Did anybody try Allow in robots.txt?
Allow only Googlebot but no others
4 HitProf 8:26 pm Jul 3, 2003
Google hardly!
just a heads up
2 wilderness 5:53 pm Jul 3, 2003
tovero
6 wilderness 6:13 am Jul 3, 2003
Netscape/PICgrabber
4 keyplyr 12:23 am Jul 3, 2003
Zao/0.1
Another resurrected spyder?
14 pendanticist 10:58 pm Jul 2, 2003
Univ of Toyko
Just a heads up
2 wilderness 10:53 pm Jul 2, 2003