Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| What is readwebpage? |
2 | BlueSky | 4:30 pm Aug 11, 2003 | ||||
| InternetLinkAgent/3.1 |
4 | rainborick | 1:48 pm Aug 11, 2003 | ||||
| Who or what is WebHiker/1.0 |
3 | viggen | 12:38 pm Aug 11, 2003 | ||||
| 68.192.98.81 |
6 | wilderness | 3:20 pm Aug 10, 2003 | ||||
| Cityreview Robot strange one. no robots. |
6 | bull | 2:58 pm Aug 10, 2003 | ||||
| Being used as a proxy? appears something is passing off a URL through my site |
19 | anchordesk | 2:57 pm Aug 10, 2003 | ||||
| Useragent "Mozilla" just fetches robots.txt |
2 | viggen | 9:52 am Aug 10, 2003 | ||||
| 198.210.1.1 fell in bt trap |
2 | coyote | 11:39 pm Aug 9, 2003 | ||||
| SearchSpider.com/1.1 24.90.240.54 |
4 | jon80 | 5:42 pm Aug 8, 2003 | ||||
| Martini anyone? I've been left a martini... |
5 | Goober | 9:47 pm Aug 7, 2003 | ||||
| webbot bot include - what is this |
4 | Megatech | 4:18 pm Aug 7, 2003 | ||||
| Girafabot does it disobey robots.txt? and a question... |
5 | davelms | 8:22 am Aug 7, 2003 | ||||
| SpiderKU/0.9 158.108.35.123 |
4 | volatilegx | 2:57 pm Aug 6, 2003 | ||||
| libwww-perl/5.53 |
2 | Raymond | 6:31 pm Aug 5, 2003 | ||||
| WebRACE/1.1 (University of Cyprus, Distributed Crawler) |
2 | jdMorgan | 6:20 pm Aug 5, 2003 | ||||
| about pay how much do i get,if i am CCNA,CCIE |
3 | nighteye | 4:21 pm Aug 5, 2003 | ||||
| newskies.net Bad bot? |
3 | dorward | 3:03 pm Aug 5, 2003 | ||||
| Who is Savvis.net with IP 216.88.158.142? |
2 | akogo | 3:15 pm Aug 4, 2003 | ||||
| Linkman Referrer spam bot? |
3 | dorward | 12:59 pm Aug 4, 2003 | ||||
| How To Block trek6.sv.av.com from my site trek6.sv.av.com takes up too much bandwidth |
6 | akogo | 8:59 pm Aug 2, 2003 | ||||
| Exalead NG/MimeLive Client (convert/http/0.123) New bot or former NG/1.0? |
2 | claus | 10:21 pm Aug 1, 2003 | ||||
| WebSearchBench no robots either |
2 | bull | 6:30 pm Aug 1, 2003 | ||||
| Zibie Spider 0.1 Java/1.4.2 no robots. |
2 | bull | 5:38 pm Aug 1, 2003 | ||||
| Good guy spiders and the Bad ones! Is there a list |
7 | atlinx | 3:50 pm Aug 1, 2003 | ||||
| My list is outdated |
2 | rainborick | 2:21 pm Aug 1, 2003 |
