Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Server Farms - August 2014 Tracking and Reporting Data Center IP Ranges [10] ( 1 2 3 4 5 6 7 8 9 10 ) |
285 | incrediBILL | 5:53 am Feb 17, 2015 | ||||
| WorldBrewBot |
3 | keyplyr | 6:06 pm Feb 14, 2015 | ||||
| Scansafe ranges |
5 | dstiles | 8:07 pm Feb 12, 2015 | ||||
| At Home with the Robots: 2015 edition |
12 | lucy24 | 8:27 am Feb 12, 2015 | ||||
| How can bots know the internal structure of a password protected site? |
7 | lamati | 10:35 am Feb 7, 2015 | ||||
| antiquated robots |
3 | lucy24 | 10:24 pm Feb 5, 2015 | ||||
| acapbot/0.1 |
4 | keyplyr | 8:11 pm Feb 4, 2015 | ||||
| Block everything from 54. except certain bots |
18 | physics | 8:51 pm Feb 1, 2015 | ||||
| Automating the server farm identification an alternative approach[3] ( 1 2 3 ) |
63 | trintragula | 12:55 am Feb 1, 2015 | ||||
| Android, Frames and Google Can android handle frames? |
4 | dstiles | 7:33 pm Jan 31, 2015 | ||||
| Looks like spiders from China? Receiving responsecode 500 errors from China visitors |
6 | Anders | 6:49 pm Jan 29, 2015 | ||||
| AdvBot |
9 | lucy24 | 9:32 pm Jan 24, 2015 | ||||
| Odd User Agent |
13 | wilderness | 8:32 pm Jan 22, 2015 | ||||
| "Mozilla/5.0 ()" - Is this used by any valid browsers? |
7 | physics | 12:15 am Jan 19, 2015 | ||||
| Named distributed crawler list |
6 | trintragula | 11:03 am Jan 9, 2015 | ||||
| HTTP header fields ACCEPT and ACCEPT-CHARSET |
3 | dstiles | 5:01 pm Jan 8, 2015 | ||||
| Opera Mini IP ranges |
5 | Xpat | 5:22 pm Jan 7, 2015 | ||||
| Chinese exploiter out of Kansas City IP |
10 | slipkid | 6:34 am Jan 6, 2015 | ||||
| Bad Bot Blackhole - Blocking Ranges |
14 | SerpsGuy | 1:21 am Jan 1, 2015 | ||||
| Baidu UA |
5 | wilderness | 10:05 am Dec 31, 2014 | ||||
| VisiStat Misbehaving San Jose, CA Brat blocking itself |
3 | Hobbs | 10:01 am Dec 31, 2014 | ||||
| Google-Fake Directories testing 404 |
23 | wilderness | 5:10 am Dec 18, 2014 | ||||
| another one for the profilers [2] ( 1 2 ) |
38 | lucy24 | 3:30 am Dec 14, 2014 | ||||
| Ddos attack - what to do |
21 | doc_z | 12:37 pm Dec 13, 2014 | ||||
| Why so many China bots? |
13 | roshaoar | 11:57 pm Dec 8, 2014 |