Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Are users visiting my site from Google.com? Some browser like agents are visiting my site from Google.com |
17 | schnee | 4:12 am Apr 13, 2008 | ||||
| Open ISearch/1.2.1 |
3 | Bewenched | 4:01 am Apr 13, 2008 | ||||
| cameleon scraper bot Different UA for every hit |
9 | Hobbs | 9:28 am Apr 12, 2008 | ||||
| Cidr |
2 | wilderness | 8:01 pm Apr 11, 2008 | ||||
| Who or what is ip-78-137-163-***.dedi.digiweb.ie |
5 | Drreggae | 8:38 am Apr 11, 2008 | ||||
| YodaoBot Beijing,China |
4 | Bewenched | 3:39 am Apr 10, 2008 | ||||
| baidu |
4 | wilderness | 6:05 am Apr 8, 2008 | ||||
| verifying ia archiver & Yahoo Bot via RDNS |
4 | Hobbs | 9:59 am Apr 7, 2008 | ||||
| Is this really a PlayStation 3 How can you tell if the browser is really a PlayStation 3? |
12 | Ocean10000 | 3:25 am Apr 7, 2008 | ||||
| Quick primer on identifying bot activity. And a how to guide to slow and stop scraping |
21 | Ocean10000 | 12:48 pm Apr 4, 2008 | ||||
| Bot or malware? I'm seeing strange 'msoffice' access to my system |
4 | Megaclinium | 4:48 am Apr 4, 2008 | ||||
| Please welcome our new Moderator incrediBILL |
29 | engine | 4:41 am Apr 4, 2008 | ||||
| Bravo MSN |
2 | wilderness | 9:57 pm Apr 2, 2008 | ||||
| Yahoo image grabber |
2 | newbie6 | 8:08 pm Apr 2, 2008 | ||||
| identifying user views resulting from spiders |
8 | newbie6 | 2:28 pm Apr 2, 2008 | ||||
| Snoopy v1.2.3 Source Identified PHP Class |
3 | incrediBILL | 5:46 pm Mar 28, 2008 | ||||
| CCBot another Nutch Variant |
7 | Ocean10000 | 1:39 pm Mar 28, 2008 | ||||
| Sap |
6 | wilderness | 11:48 pm Mar 27, 2008 | ||||
| User Agent: "WordPress/2.1.1" coming from Russia and UK Is this an RSS aggragator, or a scraper bot? |
6 | Wizcrafts | 2:32 am Mar 27, 2008 | ||||
| Readrun tag crawl showing up lately annoyance |
4 | mrjones | 9:52 am Mar 26, 2008 | ||||
| Slashers wondering about site slashers |
3 | Megaclinium | 6:22 am Mar 26, 2008 | ||||
| what is this bot? 72.244.103.zz |
3 | Megaclinium | 11:42 pm Mar 25, 2008 | ||||
| ConveraCrawler Is ConveraCrawler a malicious robot? |
17 | mattie | 1:05 pm Mar 25, 2008 | ||||
| Boitho ignoring robots.txt |
7 | Megaclinium | 8:17 pm Mar 24, 2008 | ||||
| Strange UA |
4 | wilderness | 1:14 am Mar 21, 2008 |