Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Googlebots being spoofed? How can you tell? |
14 | pendanticist | 9:41 pm Apr 10, 2003 | ||||
| List of 'BAD' bots? Which one are they? |
3 | le_gber | 8:57 pm Apr 10, 2003 | ||||
| Mozilla/3.0 (compatible) Again Pulling strange urls |
5 | jimbeetle | 8:17 pm Apr 10, 2003 | ||||
| 65.92.240.232 - Is this a Googlebot? Googlebot/1.0 |
8 | Jesse_Smith | 4:04 pm Apr 10, 2003 | ||||
| URL_Spider_Pro/3.0 Update to URL Spider Pro/1.92 |
2 | pendanticist | 5:42 am Apr 10, 2003 | ||||
| proxy.google.com |
2 | coffeelover | 12:49 am Apr 10, 2003 | ||||
| timboBot/0.9 weblog indexer |
2 | frontpage | 10:33 pm Apr 9, 2003 | ||||
| 'index.html' and 'index.htm' Will spiders see both if they are both in root directory? |
2 | TWhalen | 9:51 pm Apr 9, 2003 | ||||
| bad bot or bad self made spider trap? |
6 | matthias | 6:30 pm Apr 9, 2003 | ||||
| snorkel-x.markwatch.com Is this a spider? |
5 | scone | 6:04 pm Apr 9, 2003 | ||||
| AlkalineBOT behaving a bit aggressive... |
7 | RonPK | 9:17 am Apr 9, 2003 | ||||
| what's this (PHP/4.2.3) |
5 | fandangood | 7:58 am Apr 9, 2003 | ||||
| What to do with names and I.P. addresses of spiders? why spend a ton of time gathering info? |
11 | Brad1ey | 1:06 am Apr 9, 2003 | ||||
| Windows NT 5.2? I wasn't aware of this type of browser... |
15 | kewlbeezer | 5:11 pm Apr 8, 2003 | ||||
| Weak bot 80.8.120.54 |
3 | wilderness | 12:45 am Apr 8, 2003 | ||||
| Larbin-experimental Rude bot |
5 | herb | 6:43 pm Apr 7, 2003 | ||||
| Google Visit |
11 | DavidT | 5:31 pm Apr 7, 2003 | ||||
| net-sweeper Ignored Robots.txt |
10 | frontpage | 5:15 pm Apr 7, 2003 | ||||
| Microsoft Spider poking around? seems to be behaving itself |
7 | WitchLars | 9:11 am Apr 7, 2003 | ||||
| fresh vs deep ips |
22 | Brett_Tabke | 4:43 am Apr 7, 2003 | ||||
| WebStripper I know, you probably already have it banned... |
4 | carfac | 7:11 pm Apr 6, 2003 | ||||
| WebFilter Robot 1.0 Good/Bad or just plain Ugly? |
8 | pendanticist | 4:59 pm Apr 5, 2003 | ||||
| Four visited in one day Is it unusual? |
3 | jaybee | 4:33 pm Apr 5, 2003 | ||||
| NCSA_Beta_1 The Supercomputer is after me! |
3 | bcolflesh | 4:48 am Apr 5, 2003 | ||||
| Cloaked Spider!? ...something strange went through my site! |
3 | frogg | 12:41 am Apr 4, 2003 |
