Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| carrier-enabler |
3 | Pfui | 4:42 pm Aug 8, 2010 | ||||
| Yandex bots - incorrect rDNS? |
7 | dstiles | 9:40 pm Aug 6, 2010 | ||||
| How to Identify bad bots from legitimate users Need help on hints and tips that give away a bad bot |
11 | liam_85 | 3:43 pm Aug 6, 2010 | ||||
| Google%20Quick%20Search%20Box |
4 | Pfui | 12:29 pm Aug 4, 2010 | ||||
| CustomUserAgent Apple surprise? |
12 | Pfui | 6:16 pm Aug 1, 2010 | ||||
| Teleport Ultra |
3 | Pfui | 11:14 pm Jul 31, 2010 | ||||
| ARIN Whois |
11 | wilderness | 1:24 pm Jul 31, 2010 | ||||
| How to trap bots? That crawls disallowed pages. |
25 | Sunnz | 2:16 pm Jul 26, 2010 | ||||
| Botnet Attempting Zen Cart Attack Blank UAs Attacking |
13 | incrediBILL | 3:08 am Jul 24, 2010 | ||||
| wtv bot |
2 | Pfui | 3:40 am Jul 23, 2010 | ||||
| categorizers.bdbrandprotect.com referer bdbrandprotect http referer |
3 | a24061 | 8:51 pm Jul 21, 2010 | ||||
| acquia |
7 | Pfui | 8:18 pm Jul 19, 2010 | ||||
| panscient with own IP range |
4 | Staffa | 5:48 pm Jul 19, 2010 | ||||
| Interesting site ripper/bot like critter Try blocking this one (other than IP...) |
3 | tangor | 6:47 am Jul 18, 2010 | ||||
| Yahoo Pipes 1.0 as UA Seems like RSS Bot? |
2 | blend27 | 8:44 pm Jul 16, 2010 | ||||
| KiwiStatus/0.3 New Zealand Search |
2 | Dijkgraaf | 8:25 pm Jul 16, 2010 | ||||
| Gentoo UA - to block or not to block |
2 | zerillos | 7:22 pm Jul 15, 2010 | ||||
| How do you decide on an IP ban? |
13 | Status_203 | 1:47 pm Jul 11, 2010 | ||||
| imrd |
7 | Pfui | 9:52 pm Jul 10, 2010 | ||||
| User-Agent: in User Agent string Strange undeclared bot |
5 | Dijkgraaf | 7:58 pm Jul 9, 2010 | ||||
| SlugBug Spider a new bot on the block |
2 | Dijkgraaf | 2:30 am Jul 9, 2010 | ||||
| SearchBlox |
2 | keyplyr | 1:02 am Jul 6, 2010 | ||||
| Facebook gets value of robots.txt Not a Spider Report, just info |
7 | tangor | 8:39 am Jul 2, 2010 | ||||
| bloggsi (incl. link to post-Tweet bot recap) |
3 | Pfui | 11:22 pm Jul 1, 2010 | ||||
| WordPress/2.9.2 What kind of UA is this? |
3 | tangor | 4:50 pm Jun 30, 2010 |