Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Nook |
2 | Pfui | 7:22 pm Jan 8, 2012 | ||||
| Servers using open proxies |
4 | dstiles | 3:21 pm Jan 7, 2012 | ||||
| boia.org tools request, ignore robots.txt |
2 | Pfui | 7:06 am Jan 7, 2012 | ||||
| Googlebot-richsnippets |
2 | Pfui | 2:37 am Jan 7, 2012 | ||||
| MFE expand |
3 | Pfui | 12:44 am Jan 7, 2012 | ||||
| Vocus Hocus Pocus |
2 | Pfui | 11:17 pm Jan 6, 2012 | ||||
| Kindle & Amazon IPs? |
28 | keyplyr | 8:45 pm Jan 5, 2012 | ||||
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) |
9 | Pfui | 8:41 pm Jan 5, 2012 | ||||
| Automatically detect crawlers / bots |
13 | Globetrotter | 3:03 pm Jan 5, 2012 | ||||
| Semrushbot/0.9 |
5 | tangor | 11:12 pm Jan 4, 2012 | ||||
| yolinkBot |
4 | keyplyr | 7:06 pm Jan 3, 2012 | ||||
| MSNBot has become a constant Fast-Scraper 7 IPs crawling at max 12 pages / sec - this is out of order[2] ( 1 2 ) |
42 | AlexK | 8:28 pm Jan 1, 2012 | ||||
| GoogleDocs |
4 | Pfui | 3:09 am Dec 31, 2011 | ||||
| DDDD DDDD human? |
8 | lucy24 | 2:39 am Dec 29, 2011 | ||||
| Packet One |
2 | keyplyr | 5:07 pm Dec 24, 2011 | ||||
| Awstats Scrapers / Injectors |
19 | dstiles | 11:59 pm Dec 19, 2011 | ||||
| New scraper? portalimage.org |
9 | brokaddr | 11:27 am Dec 17, 2011 | ||||
| Use Spamhaus PBL to Decode Mixed Use Space? |
10 | incrediBILL | 10:28 pm Dec 15, 2011 | ||||
| Blekko Scoutjet new IP range |
7 | dstiles | 10:58 pm Dec 14, 2011 | ||||
| CloudFlare and spammers? How can CloudFlare be used by spammers? |
8 | Hope_Fowl | 7:43 pm Dec 13, 2011 | ||||
| opera/9.80 |
3 | topr8 | 8:20 am Dec 13, 2011 | ||||
| AdBot |
4 | keyplyr | 9:52 pm Dec 12, 2011 | ||||
| GoogleProducer New UA from Bare G IPs |
16 | Pfui | 10:05 pm Dec 11, 2011 | ||||
| TwitterBot |
3 | incrediBILL | 1:25 pm Dec 11, 2011 | ||||
| orange you sorry you asked? |
6 | lucy24 | 8:06 am Dec 10, 2011 |