Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Diigo caching pages in violation of meta tags & cloaking bot |
4 | KenB | 6:18 pm Jan 7, 2010 | ||||
| MJ12bot v1.3.0 Implements Ground Breaking Validation Capability [4] ( 1 2 3 4 ) |
99 | System | 11:49 pm Jan 3, 2010 | ||||
| Orgbybot/OrgbyBot v1.2 |
7 | GaryK | 6:15 am Jan 1, 2010 | ||||
| Linguee Bot |
19 | Pfui | 6:58 pm Dec 31, 2009 | ||||
| aiHitBot |
4 | Pfui | 3:47 am Dec 30, 2009 | ||||
| Updating bot ban list & cleaning out obsolete entries |
11 | KenB | 8:15 pm Dec 29, 2009 | ||||
| Twiceler/cuil.com craziness (FWIW) |
7 | Pfui | 7:05 am Dec 29, 2009 | ||||
| Zscho.de Crawler Nutch Redux |
2 | Pfui | 9:22 pm Dec 27, 2009 | ||||
| TwitterReseach [sic] |
5 | Pfui | 10:01 pm Dec 22, 2009 | ||||
| Google non-bots What to do about rDNS? |
2 | dstiles | 11:04 pm Dec 21, 2009 | ||||
| Mozilla/4.0 (compatible) Greasemonkey |
2 | Pfui | 8:23 pm Dec 20, 2009 | ||||
| Variable IP proxy |
3 | dstiles | 7:11 pm Dec 20, 2009 | ||||
| GazoPabot & HTMLParser Dual-hit bots |
3 | dstiles | 6:01 pm Dec 20, 2009 | ||||
| Facebook share follower |
6 | GaryK | 9:34 pm Dec 18, 2009 | ||||
| Stinky crawler proxy Googlebot fed through a proxy |
3 | jdMorgan | 4:57 pm Dec 18, 2009 | ||||
| DuckDuckBot |
9 | keyplyr | 4:54 pm Dec 18, 2009 | ||||
| NSmith / NSmitm / NutSmith / Jane Smith |
2 | Pfui | 9:28 pm Dec 17, 2009 | ||||
| Spinn3r Still ban-worthy |
2 | Pfui | 9:06 am Dec 17, 2009 | ||||
| Mozilla/5.0 (compatible; LegalAnalysisAgent/1.0; http://www.#*$! |
4 | GaryK | 5:14 pm Dec 14, 2009 | ||||
| baypup/colbert (Baypup; http://sf.baypup.com/webmasters; jason@baypup. |
2 | GaryK | 4:46 pm Dec 14, 2009 | ||||
| Netvibes favicons proxy |
3 | GaryK | 4:45 pm Dec 14, 2009 | ||||
| Stealth bot? Same-site referers lack post-suffix slash |
12 | Pfui | 4:40 pm Dec 14, 2009 | ||||
| 208.43.205.234:80:::pscan |
3 | GaryK | 4:30 pm Dec 14, 2009 | ||||
| On Dasher? |
4 | Megaclinium | 3:10 am Dec 14, 2009 | ||||
| Facebook share follower |
3 | dstiles | 8:45 pm Dec 12, 2009 |
