Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Amazon aws, dedicatedcpanel, serverloft Issues These domains are dedicatedly screwing my site - Need Help. |
10 | Future | 7:59 pm Nov 23, 2013 | ||||
| TECH UPDATE: Bots as Browsers Using JavaScript The Bot Detection Game Has Changed[2] ( 1 2 ) |
48 | incrediBILL | 9:18 pm Nov 11, 2013 | ||||
| google-xrawler Unknown User agent "google-xrawler" |
15 | DutchTheWiz | 7:51 pm Nov 7, 2013 | ||||
| Zylon |
4 | lucy24 | 8:32 pm Nov 6, 2013 | ||||
| Is CompSpyBot really an NSASpyBot? |
17 | evsiz | 7:55 pm Oct 30, 2013 | ||||
| 50.16.178.xx Amazon of course linkCheck |
2 | Bewenched | 8:21 pm Oct 24, 2013 | ||||
| Who/what is ns4006554.ip-198-27-80.net? |
7 | iomfan | 2:01 am Oct 22, 2013 | ||||
| another one for the profilers |
6 | lucy24 | 6:52 am Oct 19, 2013 | ||||
| What is GABot/1.0 |
6 | woodpecker | 11:33 pm Oct 12, 2013 | ||||
| "DAPPER-HOST-IP" why Baidu ? "DAPPER-HOST-IP" usrer agent baidu |
18 | aboshakeeb | 9:20 am Oct 10, 2013 | ||||
| CrawlDaddy abot Site Harvester |
3 | WarmGlow | 11:37 pm Sep 28, 2013 | ||||
| meanpath |
10 | lucy24 | 6:42 am Sep 21, 2013 | ||||
| International Data Centers |
6 | incrediBILL | 7:31 pm Sep 16, 2013 | ||||
| Dot Exe in User Agent String |
10 | lucy24 | 9:55 pm Sep 10, 2013 | ||||
| same old same old YisouSpider Yisou back from the dead? |
4 | lucy24 | 1:23 am Sep 6, 2013 | ||||
| msn bot has lost its mind 1000+ parallel connections |
7 | amznVibe | 11:27 am Sep 5, 2013 | ||||
| Best method of blocking? Apache conf, iptables or other |
30 | jmccormac | 9:56 pm Sep 4, 2013 | ||||
| Server Farms - May 2013 Ongoing Hosting Data Center Discussion[4] ( 1 2 3 4 ) |
103 | incrediBILL | 8:09 pm Sep 4, 2013 | ||||
| YandexImages |
11 | lucy24 | 8:21 pm Aug 31, 2013 | ||||
| Scraper Test Drive Open Discussion of Scraper Tools and Success Rates |
3 | incrediBILL | 3:41 am Aug 31, 2013 | ||||
| First bot on IPv6 for me |
7 | bhukkel | 8:11 am Aug 28, 2013 | ||||
| Brand Affinity Tech |
6 | wilderness | 1:22 pm Aug 26, 2013 | ||||
| Did delegated-arin-latest vanish on 8.13.13? It seems to have disappeared from their server |
9 | incrediBILL | 4:54 am Aug 24, 2013 | ||||
| the plainclothes bingbot |
4 | lucy24 | 1:59 pm Aug 23, 2013 | ||||
| UP.Browser |
13 | lucy24 | 7:57 am Aug 14, 2013 |
