Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Purge Information from domain tools and aboutus.org? [3] ( 1 2 3 ) |
68 | doni | 2:30 am Jul 21, 2009 | ||||
| TailsweepBlogCrawler/Tailsweep |
5 | JAB_Creations | 3:33 pm Jul 17, 2009 | ||||
| Microsoft launches Azure, an AWS competitor Expect more unverifiable bots |
8 | incrediBILL | 12:59 am Jul 16, 2009 | ||||
| Slurp Gone Wild Took only disallowed files |
2 | GaryK | 9:43 am Jul 15, 2009 | ||||
| pythumbnail.py taking screenshots? |
3 | GaryK | 9:42 am Jul 15, 2009 | ||||
| Qryos |
5 | keyplyr | 9:36 am Jul 15, 2009 | ||||
| Log spam or crawler? mtrack.nl |
7 | GaryK | 6:40 am Jul 15, 2009 | ||||
| Fake Googlebots - why are they here? Asking about why there are so many fake Googlebots accessing our site. |
8 | wizboy | 3:12 am Jul 15, 2009 | ||||
| User-Agent "GbPlugin" Annoying HTTP requests for a non-existent .SWF file |
2 | jaynl | 8:44 pm Jul 14, 2009 | ||||
| Visited by http://tools.geek-tools.org Apparent log spam |
5 | GaryK | 4:49 am Jul 14, 2009 | ||||
| /(null) |
22 | Pfui | 4:12 pm Jul 13, 2009 | ||||
| New MSN bot? adidxbot |
12 | dstiles | 6:54 pm Jul 6, 2009 | ||||
| ovale.ru's: "ovalebot3.ovale.ru facepage" and "search new sites" 1 host, 2 bots, 0 robots.txt |
2 | Pfui | 6:18 pm Jul 5, 2009 | ||||
| solfo-linkchecker If it looks like a bot... |
2 | Pfui | 6:57 pm Jul 3, 2009 | ||||
| Tagoobot/3.0 http://www.tagoo.ru |
4 | tpeacock | 1:48 am Jul 1, 2009 | ||||
| "slurp, yahoo! slurp, slurp/2.0, inktomi slurp, slurp.so/1.0" Is someone at Yahoo playing some sort of joke? |
2 | Pfui | 10:41 pm Jun 30, 2009 | ||||
| ShrinkTheWeb.com Crawler |
2 | Pfui | 8:44 pm Jun 30, 2009 | ||||
| Rexyobot |
10 | Pfui | 8:41 am Jun 30, 2009 | ||||
| WideCircles Analysis Agent |
2 | Pfui | 12:37 am Jun 30, 2009 | ||||
| Spider Hunting or Site Building? Goes in waves, do you trust your bot blocker? |
8 | incrediBILL | 9:26 pm Jun 28, 2009 | ||||
| smart.apnoti.com Robot/v1.34 |
3 | keyplyr | 5:53 pm Jun 28, 2009 | ||||
| DoubleVerify Crawler |
4 | keyplyr | 9:01 am Jun 26, 2009 | ||||
| Pinboard (pinboard.cantbedone.org) |
4 | Pfui | 3:34 am Jun 26, 2009 | ||||
| HolmesBot (http://holmes.ge) |
6 | Pfui | 4:20 pm Jun 23, 2009 | ||||
| Zanran ZanranCrawler/0.2 (helpdesk@zanran.com) |
3 | Frank_Rizzo | 8:22 am Jun 22, 2009 |
