Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| grub-crawler - bad manners good one for an .htaccess ban |
11 | mivox | 5:30 pm Aug 15, 2001 | ||||
| Spider List Is this available? |
3 | phrequency | 12:16 am Aug 15, 2001 | ||||
| Weird Log Entries weird GET requests |
5 | patchwork | 8:02 am Aug 14, 2001 | ||||
| moget - goo.ne.jp Took alot of pages - v nice |
2 | zero6 | 11:20 pm Aug 11, 2001 | ||||
| Rainbot1.1 Any ideas? |
2 | themoff | 6:58 pm Aug 11, 2001 | ||||
| Bizzare Patterns - A New Bot? 63.127.106.243 - Can anyone identify this ip? |
5 | digitalgirl | 7:56 pm Aug 10, 2001 | ||||
| DFusionBot I've searched, but I can't find it... |
4 | BDC | 4:12 pm Aug 10, 2001 | ||||
| What are they doing? |
2 | toolman | 2:00 pm Aug 10, 2001 | ||||
| AV Spider?! Don't know |
8 | circuitjump | 11:23 pm Aug 9, 2001 | ||||
| gazz/2.1 any info on this spider? |
4 | volatilegx | 5:43 pm Aug 9, 2001 | ||||
| How do I find out the UA Name ? |
6 | circuitjump | 8:07 am Aug 9, 2001 | ||||
| Mozilla/3.01 (compatible;) Crawler or Browser |
6 | joshie76 | 4:38 pm Aug 8, 2001 | ||||
| p95-169.unl.ac.uk cache or spider? |
2 | roadxrunnerx128 | 4:14 pm Aug 8, 2001 | ||||
| "pavuk/0.9pl27 i686-pc-linux-gnu" What is this? |
2 | Ulrike | 9:46 am Aug 8, 2001 | ||||
| odd hit from Lycos |
5 | Key_Master | 9:55 am Aug 5, 2001 | ||||
| Googlebot with a query? |
6 | grnidone | 12:03 am Aug 5, 2001 | ||||
| Alexa - what a pain |
3 | Jill | 1:43 pm Aug 3, 2001 | ||||
| can anyone id this spider |
5 | nube | 12:16 am Aug 3, 2001 | ||||
| Heads up, new Lycos spider |
2 | Air | 11:18 pm Aug 2, 2001 | ||||
| Marvin v0.5 any info on this UA? |
7 | volatilegx | 10:02 pm Aug 2, 2001 | ||||
| SSI and Spiders |
2 | Benj | 3:33 am Aug 1, 2001 | ||||
| Normal spider pattern what is it? |
8 | grnidone | 12:20 am Aug 1, 2001 | ||||
| 216.221.224.148 - Funny access pattern |
2 | mark_roach | 10:36 pm Jul 27, 2001 | ||||
| GET x HTTP/1.0 strange request |
2 | bobriggs | 7:16 pm Jul 27, 2001 | ||||
| A spider with a referring URL I knew I'd seen this before |
8 | volatilegx | 5:31 pm Jul 27, 2001 |