Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Exabot Thumbnails Block or not to block? |
2 | tangor | 1:07 pm Feb 12, 2011 | ||||
| WordPress |
4 | Umbra | 8:36 pm Feb 11, 2011 | ||||
| What is the ADM token for? |
4 | GaryK | 9:24 pm Feb 10, 2011 | ||||
| RockMelt |
6 | TheMadScientist | 11:42 pm Feb 9, 2011 | ||||
| Bot Blocking in IPV6 what's your strategy? |
7 | incrediBILL | 11:07 pm Feb 4, 2011 | ||||
| Googlebot acting strangely Finding it in error logs |
6 | grandma_genie | 3:52 am Feb 4, 2011 | ||||
| KKman2.0 With Strange Characters in Logs |
6 | OnThePike | 4:59 pm Feb 3, 2011 | ||||
| Is this IP google? |
4 | zerillos | 9:28 am Feb 3, 2011 | ||||
| Yellow Pages USA what is its bot UA? |
10 | dstiles | 4:37 pm Feb 2, 2011 | ||||
| BigPanda new search site crawler |
6 | Mokita | 7:30 pm Feb 1, 2011 | ||||
| Blocking Navigator 98 User Agent and NT 4.0 |
8 | incrediBILL | 10:30 pm Jan 30, 2011 | ||||
| GlueText Spider/Crawler Located |
7 | incrediBILL | 5:29 pm Jan 29, 2011 | ||||
| Did The Wayback Machine Die in 2010? |
4 | incrediBILL | 5:14 pm Jan 24, 2011 | ||||
| Whitelisting just the good bots |
12 | Dexie | 3:11 am Jan 23, 2011 | ||||
| Can anyone add anything else to this ? |
12 | Dexie | 9:59 pm Jan 22, 2011 | ||||
| How to block Unwanted Traffic Generators? Traffic jumped 10 fold in 2 months -- I need to tackle this :| |
9 | MaxGrenk | 7:32 am Jan 19, 2011 | ||||
| Mozilla/4.0 (compatible; ICS) What does ICS stand for? |
5 | SteveWh | 11:02 pm Jan 17, 2011 | ||||
| ARIN-WHOIS again |
4 | wilderness | 10:41 pm Jan 9, 2011 | ||||
| Suggesting an improvement to Majestic's validation keys |
4 | Clark | 10:19 pm Jan 5, 2011 | ||||
| Ning coming with Feedburner UA |
2 | zerillos | 2:26 am Jan 5, 2011 | ||||
| Is this Googlebot's ip? |
7 | tabish | 12:17 am Jan 5, 2011 | ||||
| New bot from Google? |
9 | Dijkgraaf | 12:40 pm Dec 28, 2010 | ||||
| bah humbug bot also known as grinch bot |
2 | incrediBILL | 3:20 am Dec 27, 2010 | ||||
| Facebook Referer Trapped Is uiserver a common referer situation? |
7 | dstiles | 5:09 am Dec 22, 2010 | ||||
| Is this Yahoo crawling via Google? |
6 | GaryK | 5:24 am Dec 15, 2010 |
