Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| SetEnvIf Remote-Addr |
12 | wilderness | 12:01 am May 26, 2011 | ||||
| Google as the referrer but no query string |
9 | agent_x | 9:14 pm May 25, 2011 | ||||
| And Now Google's Doing It. JS Stats Show GoogleBot [3] ( 1 2 3 ) |
74 | TheMadScientist | 10:04 pm May 18, 2011 | ||||
| not sure |
6 | wilderness | 9:53 pm May 11, 2011 | ||||
| Mikrotik HttpProxy Misconfigured headers /scraper |
3 | caribguy | 10:03 pm May 6, 2011 | ||||
| Adaptive Spider Technology |
3 | topr8 | 11:02 pm May 4, 2011 | ||||
| Microsoft Bot Processing jQuery? |
6 | TheMadScientist | 11:56 pm May 1, 2011 | ||||
| Chinese scrapers caught ya |
2 | Bewenched | 6:43 pm Apr 30, 2011 | ||||
| Exalead - massive spidering I know this one has been around a while |
6 | Bewenched | 3:25 pm Apr 29, 2011 | ||||
| Geo-Targetting by IP and Search engines spiders |
4 | CrazyBigGaz | 7:40 pm Apr 28, 2011 | ||||
| literate robots? |
14 | lucy24 | 1:40 am Apr 27, 2011 | ||||
| who are marshal dot com and what do they want? |
7 | lucy24 | 6:48 pm Apr 20, 2011 | ||||
| Indonesian scraper bad scraper go away. |
3 | Bewenched | 7:12 am Apr 20, 2011 | ||||
| more MS |
3 | wilderness | 11:59 pm Apr 19, 2011 | ||||
| Odd |
4 | wilderness | 9:07 pm Apr 15, 2011 | ||||
| Comment spammer? |
5 | CyberShadow | 7:54 pm Apr 15, 2011 | ||||
| the return of spbot continuing the spbot / seoprofiler discussion |
3 | lucy24 | 10:09 pm Apr 9, 2011 | ||||
| Google spidering with new user agent |
5 | zerillos | 10:01 pm Apr 9, 2011 | ||||
| How to find all the IP ranges of a hosting co |
15 | Megaclinium | 3:15 am Apr 8, 2011 | ||||
| Googlebot with Referer |
5 | dstiles | 10:11 pm Apr 5, 2011 | ||||
| Bingbot Full Trip DNS Validation is BROKEN! |
9 | incrediBILL | 9:31 pm Apr 2, 2011 | ||||
| Microsoft launches Azure |
2 | wilderness | 8:13 pm Apr 1, 2011 | ||||
| Bork Edition Not the Swedish chef |
2 | caribguy | 9:35 pm Mar 29, 2011 | ||||
| Spider / Bit / Agent rippers 0 in stats |
2 | jim_knopf | 2:14 am Mar 29, 2011 | ||||
| LeaseWeb? |
6 | Bewenched | 10:06 pm Mar 28, 2011 |