Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Help ip address |
6 | Gerry | 5:11 am Apr 19, 2001 | ||||
| Surfairy another one for you DNS junkies |
6 | vis | 12:21 pm Apr 18, 2001 | ||||
| file not found |
4 | stanleytk | 2:58 am Apr 18, 2001 | ||||
| First-search.com |
4 | stn24 | 9:27 am Apr 17, 2001 | ||||
| Cyveillance notorious bandwidth wasters just got some more ips, new to me at least |
3 | jeremy_goodrich | 9:41 pm Apr 16, 2001 | ||||
| Exclusion of bots from dynamic pages A flexible alternative to robots.txt |
2 | Everyman | 5:15 pm Apr 16, 2001 | ||||
| Internet Archive and Thunderstone? Who are they |
2 | Justin | 5:09 pm Apr 16, 2001 | ||||
| Any Body? What is this dpcatalog1.go2net.com |
10 | drbill | 6:11 pm Apr 13, 2001 | ||||
| I have not seen this one before. |
6 | miles | 5:13 pm Apr 13, 2001 | ||||
| Ip: 199.245.188.102 |
4 | double0seven | 4:52 am Apr 13, 2001 | ||||
| RPT-HTTPClient/0.3-2 what is this spider doing |
5 | stanleytk | 7:29 pm Apr 12, 2001 | ||||
| Yahoo who? just found this, todays log -- what's up? |
2 | Marcia | 4:23 am Apr 12, 2001 | ||||
| Indy Library 211.101.236.29 Mozilla/3.0 (compatible; Indy Library) |
9 | Son_House | 5:03 pm Apr 11, 2001 | ||||
| 007/v7 spider |
3 | stanleytk | 2:03 pm Apr 11, 2001 | ||||
| This? I've never seen this before.. |
4 | BoneHeadicus | 6:29 pm Apr 9, 2001 | ||||
| is this a spider or google employee |
3 | nube | 6:27 pm Apr 9, 2001 | ||||
| New Excite Bot? storm.excite.com |
3 | Son_House | 5:02 pm Apr 9, 2001 | ||||
| Bot or not? h-206-132-131-163.globalcenter.net |
4 | Son_House | 3:50 am Apr 7, 2001 | ||||
| User Agent: Pizilla++ ver 2.45 IP 216.218.197.2 |
3 | volatilegx | 9:29 pm Apr 6, 2001 | ||||
| Spiders how can I make one / find one (cgi or perl) |
5 | luckynh | 5:34 pm Apr 5, 2001 | ||||
| How can I easily view all incomers? |
5 | Eric_Lander | 1:55 pm Apr 5, 2001 | ||||
| What |
2 | drbill | 5:31 am Apr 5, 2001 | ||||
| Anybody know this one? User-Agent: Mozilla/5.0 URL-Spider |
2 | volatilegx | 8:17 pm Apr 4, 2001 | ||||
| eSense (Chimera) Mozzila/4.0 (Compatible); www.vigil.com/esensedisclaim.html |
9 | mattur | 1:42 pm Apr 4, 2001 | ||||
| Freaky spiders |
3 | Froggyman | 9:55 pm Apr 3, 2001 |
