Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| pflab |
3 | keyplyr | 12:50 am Sep 3, 2011 | ||||
| Netvigator |
5 | keyplyr | 11:31 pm Sep 1, 2011 | ||||
| ScooperBot |
3 | Pfui | 2:46 pm Sep 1, 2011 | ||||
| who is Covario and what do they want from me? |
3 | lucy24 | 7:27 pm Aug 31, 2011 | ||||
| 74.125.52.93 in my logs today |
6 | apauto | 9:11 pm Aug 30, 2011 | ||||
| AIM7 WinHTTP/1.0 |
2 | Pfui | 7:00 pm Aug 29, 2011 | ||||
| YodaoBot |
7 | Pfui | 1:16 am Aug 29, 2011 | ||||
| BOTW Spider Jurgle LLC in New York |
2 | grandma_genie | 6:50 pm Aug 24, 2011 | ||||
| jikespider Chinese bot shape-shifts on the fly. |
4 | Pfui | 7:29 pm Aug 23, 2011 | ||||
| Google IPs but MSIE UA Huh? |
2 | Pfui | 4:47 pm Aug 22, 2011 | ||||
| NerdByNature.Bot |
13 | keyplyr | 10:18 pm Aug 21, 2011 | ||||
| Level3 and more |
5 | wilderness | 8:51 pm Aug 21, 2011 | ||||
| AhrefsBot née SiteBot |
5 | Pfui | 4:57 pm Aug 21, 2011 | ||||
| Butterfly |
2 | Pfui | 7:30 pm Aug 20, 2011 | ||||
| integrity/3 |
3 | Pfui | 4:30 pm Aug 18, 2011 | ||||
| CydralSpider |
2 | Pfui | 7:58 pm Aug 17, 2011 | ||||
| Strange User Header |
3 | yaashul | 4:30 pm Aug 15, 2011 | ||||
| baidu Transcoder |
5 | Pfui | 8:23 pm Aug 14, 2011 | ||||
| SISTRIX Crawler sistrix.com/net/de SEO ignores robots.txt |
2 | Pfui | 8:53 pm Aug 9, 2011 | ||||
| static.cloud-ips |
4 | keyplyr | 7:49 am Aug 8, 2011 | ||||
| Hurricane Electric. Bot? Good or bad? |
7 | Bewenched | 1:54 am Aug 7, 2011 | ||||
| Windows-Live-Social-Object-Extractor-Engine |
5 | dstiles | 10:03 pm Aug 6, 2011 | ||||
| Bergdorf Group |
7 | keyplyr | 8:47 pm Aug 5, 2011 | ||||
| Block Search Engines From Indexing Member Profiles |
10 | anthonyon | 7:48 pm Jul 30, 2011 | ||||
| Google's Fox Offers to Guard Your Henhouse Whither thou goest, local access controls? |
2 | Pfui | 4:11 pm Jul 29, 2011 |