Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Mozilla/3.01 (compatible;) |
4 | JAB_Creations | 7:39 pm Dec 7, 2004 | ||||
| Ocelli/1.1 One aggressive spider! |
6 | jam13 | 9:35 am Dec 6, 2004 | ||||
| Requests for the same 23 pages not sure if it is a 'bot or what... |
9 | carfac | 3:47 am Dec 6, 2004 | ||||
| Looking for data on internet filters |
12 | volatilegx | 8:07 pm Dec 4, 2004 | ||||
| Hotbot-Lycos Robot |
4 | jdancing | 2:49 pm Dec 4, 2004 | ||||
| "GoogleBot Cloak" Robot showed up in my tracking, "GoogleBot Cloak", what is it? |
4 | aghill | 12:21 pm Dec 3, 2004 | ||||
| Ripley's "Believe it or not" |
2 | wilderness | 3:44 am Dec 3, 2004 | ||||
| HTtrack and Webpix Both his my site, rapid fire. |
5 | larryhatch | 10:21 pm Nov 29, 2004 | ||||
| pcnbot? |
5 | sqlgod | 9:18 pm Nov 29, 2004 | ||||
| SearchByUsa/2 Robots.txt -yes. |
4 | pendanticist | 2:10 am Nov 29, 2004 | ||||
| gazz/5.0 Japan |
3 | Staffa | 11:49 am Nov 28, 2004 | ||||
| CreativeCommons |
3 | wilderness | 3:50 am Nov 28, 2004 | ||||
| User-Agent: OmniWeb Disallowed |
4 | guitaristinus | 12:20 pm Nov 27, 2004 | ||||
| mozilla/4.0 (compatible; cerberian drtrs version-3.1-build-16) Again! |
6 | DoppyNL | 10:27 pm Nov 26, 2004 | ||||
| Stealth bots How to detect them? |
16 | Umbra | 7:35 am Nov 25, 2004 | ||||
| Please check my .htaccess bad bot list ... |
6 | tigertom | 3:43 pm Nov 24, 2004 | ||||
| Anyone explain Alexa Bot |
9 | blaketar | 6:53 pm Nov 21, 2004 | ||||
| NextGenSearchBot 1 Did not obey robots.txt. Is it new? |
6 | cybertime | 2:04 am Nov 21, 2004 | ||||
| Verizon Directory Sales - East Co. |
7 | wilderness | 7:21 pm Nov 19, 2004 | ||||
| Blue Earth Valley |
2 | wilderness | 6:20 am Nov 18, 2004 | ||||
| Jetbot and Travelbot |
7 | Scooter24 | 4:24 am Nov 17, 2004 | ||||
| pipeLiner/0.3a pipeline-search.com |
4 | pendanticist | 5:09 pm Nov 16, 2004 | ||||
| NokodoBot doesn't obey robots.txt and prides itself on the fact |
10 | volatilegx | 6:02 pm Nov 15, 2004 | ||||
| Program Shareware 1.0.0 Can't find *ANY* info on Program Shareware 1.0.0 |
5 | JAB_Creations | 11:48 pm Nov 14, 2004 | ||||
| google shows my old cached site? shows my old landing page, but has been to my site recently from raw logs |
2 | willis1480 | 10:38 pm Nov 13, 2004 |