Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| integrity/3 |
3 | Pfui | 4:30 pm Aug 18, 2011 | ||||
| CydralSpider |
2 | Pfui | 7:58 pm Aug 17, 2011 | ||||
| Strange User Header |
3 | yaashul | 4:30 pm Aug 15, 2011 | ||||
| baidu Transcoder |
5 | Pfui | 8:23 pm Aug 14, 2011 | ||||
| SISTRIX Crawler sistrix.com/net/de SEO ignores robots.txt |
2 | Pfui | 8:53 pm Aug 9, 2011 | ||||
| static.cloud-ips |
4 | keyplyr | 7:49 am Aug 8, 2011 | ||||
| Hurricane Electric. Bot? Good or bad? |
7 | Bewenched | 1:54 am Aug 7, 2011 | ||||
| Windows-Live-Social-Object-Extractor-Engine |
5 | dstiles | 10:03 pm Aug 6, 2011 | ||||
| Bergdorf Group |
7 | keyplyr | 8:47 pm Aug 5, 2011 | ||||
| Block Search Engines From Indexing Member Profiles |
10 | anthonyon | 7:48 pm Jul 30, 2011 | ||||
| Google's Fox Offers to Guard Your Henhouse Whither thou goest, local access controls? |
2 | Pfui | 4:11 pm Jul 29, 2011 | ||||
| where's the browser? |
19 | lucy24 | 7:36 pm Jul 27, 2011 | ||||
| Bender the web crawler anyone know what this new googlebot is about? |
12 | Kelowna | 1:57 pm Jul 26, 2011 | ||||
| UAs that asked for robots.txt |
11 | incrediBILL | 6:02 am Jul 18, 2011 | ||||
| Yellow Pages bot impersonating Googlebot? |
11 | mrtonyg | 9:57 pm Jul 15, 2011 | ||||
| m160.cs.uwaterloo.ca block? |
4 | santapaws | 7:40 pm Jul 15, 2011 | ||||
| evrinid |
6 | Pfui | 8:37 pm Jul 13, 2011 | ||||
| Garlik? A UK outfit |
5 | tangor | 2:52 am Jul 13, 2011 | ||||
| back in the Ukraine if the User Agent fits, wear it |
23 | lucy24 | 8:22 pm Jul 9, 2011 | ||||
| a legitimate blank? |
26 | lucy24 | 3:52 pm Jul 9, 2011 | ||||
| Re-visiting Data URI Unexpected data:image URI calls to web sites |
6 | dstiles | 9:24 pm Jul 8, 2011 | ||||
| the faviconbot's new clothes |
3 | lucy24 | 11:45 pm Jul 7, 2011 | ||||
| SiteSpeedBot and misinformation, bad bot |
5 | incrediBILL | 5:22 am Jul 2, 2011 | ||||
| q1 |
2 | Pfui | 10:03 pm Jun 29, 2011 | ||||
| where old browsers go to die Netscape hits bottom |
2 | lucy24 | 9:53 am Jun 28, 2011 |
