Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| What do we do with all the IP blocks gathered? What technology to block crawlers, .htaccess or scripts? |
22 | dougwilson | 7:06 pm Oct 15, 2014 | ||||
| WebTarantula Tools They have their own nasty browser, toolbar, and bots |
9 | Angonasec | 7:11 pm Oct 12, 2014 | ||||
| stremorbot |
7 | wilderness | 10:14 am Oct 6, 2014 | ||||
| 141 Class A |
5 | wilderness | 1:13 am Oct 6, 2014 | ||||
| Don't understand this log entry |
15 | aristotle | 10:36 am Sep 21, 2014 | ||||
| ContextAdbot IP updates |
6 | lucy24 | 1:31 am Sep 3, 2014 | ||||
| GTT bots You may want to fine tune your blocks: I don't |
5 | Angonasec | 12:22 pm Aug 27, 2014 | ||||
| strange googlebot referer googlebot referral from odd site |
9 | dupres01 | 7:06 am Aug 27, 2014 | ||||
| xovibot |
7 | lucy24 | 7:05 pm Aug 25, 2014 | ||||
| odd apple requests strange requests coming from apple ramge |
6 | dupres01 | 6:20 pm Aug 24, 2014 | ||||
| IPv6 enabled Bots / Server Farm |
8 | Ocean10000 | 6:24 pm Aug 18, 2014 | ||||
| Googlebot and iPhone |
30 | wilderness | 1:41 pm Aug 17, 2014 | ||||
| Manticore (more AWS spawn) |
5 | Pfui | 6:06 pm Aug 16, 2014 | ||||
| Enterprise Search Appliances for Scrapers? Thunderstone |
20 | not2easy | 9:56 am Aug 14, 2014 | ||||
| apache pb2.gif why do bots ask for this[2] ( 1 2 ) |
44 | dupres01 | 4:46 am Aug 12, 2014 | ||||
| Aug 8 Amazon UPDATED public IPs Aws ip |
4 | miscbyproduct | 4:00 pm Aug 11, 2014 | ||||
| A6 on its Knees They are taking notice |
9 | Angonasec | 12:06 pm Aug 11, 2014 | ||||
| sub-semalt |
10 | wilderness | 7:05 pm Aug 10, 2014 | ||||
| BrandProtect |
5 | lucy24 | 12:44 am Aug 2, 2014 | ||||
| Server Farms - April 2014 Tracking and Reporting Data Center IP Ranges [7] ( 1 2 3 4 5 6 7 ) |
193 | incrediBILL | 12:39 am Jul 31, 2014 | ||||
| Facebot from Facebook[2] ( 1 2 ) |
40 | Pfui | 2:12 pm Jul 29, 2014 | ||||
| Blocked IP Appends Data on URL |
3 | Angonasec | 11:09 pm Jul 27, 2014 | ||||
| Botnets How to recognise & block[2] ( 1 2 ) |
43 | keyplyr | 9:37 pm Jul 27, 2014 | ||||
| Trustwave |
3 | wilderness | 7:25 pm Jul 27, 2014 | ||||
| Bing Bot hiccup |
5 | wilderness | 10:43 pm Jul 26, 2014 |
