Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| HeartRails Capture Yet another Screen Capture Website |
4 | Ocean10000 | 3:59 pm May 3, 2008 | ||||
| PhpDig/1.8.8 New Zealand |
2 | Bewenched | 8:12 pm Apr 30, 2008 | ||||
| Yahoo! Slurp/3.0 used for iffy purposes? 26 hits to "list.php" files -- on non-PHP site |
13 | Pfui | 11:49 am Apr 29, 2008 | ||||
| oegp v. 1.3.0 a confirmed scraper |
3 | Hobbs | 5:47 am Apr 29, 2008 | ||||
| Stealth GoogleBot, or Spoofed IP? Strange Hits in Stats, WHOIS Shows they come from Google |
4 | abhorrent12 | 3:31 am Apr 28, 2008 | ||||
| unknown |
23 | wilderness | 11:26 pm Apr 26, 2008 | ||||
| Ask Jeeves/Teoma oddities odd IPs and reverse DNS |
8 | incrediBILL | 1:45 pm Apr 26, 2008 | ||||
| Strange hits from internetserviceteam.com |
5 | rudyten | 6:39 pm Apr 25, 2008 | ||||
| SharedService.Crawler resolves to Microsoft? |
2 | Bewenched | 4:44 pm Apr 25, 2008 | ||||
| Blocking the bad boys keeping bots, breakers and bugs out |
4 | superclown2 | 11:55 am Apr 24, 2008 | ||||
| spider adds url? |
12 | smokeybarnable | 1:01 am Apr 24, 2008 | ||||
| Basalt |
4 | wilderness | 11:55 pm Apr 22, 2008 | ||||
| MLBot from Metadatalabs |
3 | incrediBILL | 11:41 pm Apr 22, 2008 | ||||
| TestBot yet another dodgy bot crawling from Amazon |
3 | Mokita | 8:06 pm Apr 22, 2008 | ||||
| My Latest Google, MSN, Yahoo Spider IP List IP list of GoogleBot, MSNBot and Yahoo Slurp spiders. CIDR formatted. |
3 | hdogan | 11:27 pm Apr 18, 2008 | ||||
| yangboz |
10 | wilderness | 6:26 pm Apr 17, 2008 | ||||
| darxi/1.1 New little creapy crawly |
3 | Ocean10000 | 2:01 pm Apr 17, 2008 | ||||
| Default User Agents Discussion |
7 | wilderness | 7:01 pm Apr 16, 2008 | ||||
| Yahoo! Slurp/3.0 released on new IPs Headaches for those Validating by IP instead of Reverse DNS |
12 | Receptional_Andy | 3:29 am Apr 16, 2008 | ||||
| HttpGetFile |
3 | Hobbs | 8:37 pm Apr 14, 2008 | ||||
| Exabot-Thumbnails Exalead |
5 | Bewenched | 7:48 pm Apr 13, 2008 | ||||
| Optioncarriere / Careerjet job search engines with generic ua's |
16 | incrediBILL | 6:43 pm Apr 13, 2008 | ||||
| Powerset who's adopting? |
10 | Hobbs | 8:38 am Apr 13, 2008 | ||||
| Scraper from Micro Synergy |
6 | keyplyr | 4:43 am Apr 13, 2008 | ||||
| bloglines crawler? |
5 | Bewenched | 4:25 am Apr 13, 2008 |