Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| User Agent: Ken |
2 | fusion5 | 8:34 pm Jun 20, 2006 | ||||
| FDSE robot |
5 | bull | 1:18 am Jun 20, 2006 | ||||
| Snappy/1.1 urltrends bot |
4 | Mokita | 6:04 am Jun 19, 2006 | ||||
| Nutch Sightings from 100+ IPs |
18 | incrediBILL | 3:51 am Jun 19, 2006 | ||||
| I'm a newbie Know what this is? |
5 | jcmoon | 5:03 pm Jun 15, 2006 | ||||
| what's wrong with my sitemap.xml sitemap |
4 | youriv | 10:56 pm Jun 14, 2006 | ||||
| WISEbot/1.0 (WISEbot@wisenut.co.kr; http://wisebot.wisenut.co.kr) No robots.txt |
12 | GaryK | 11:26 pm Jun 12, 2006 | ||||
| exactseek.com Last forum thread seems to be from 2002 |
5 | GaryK | 6:09 pm Jun 12, 2006 | ||||
| Teoma's new IPs a direct hit... |
14 | incrediBILL | 11:32 pm Jun 11, 2006 | ||||
| Worio don't worio, be happy |
5 | incrediBILL | 8:26 pm Jun 11, 2006 | ||||
| Yahoo? Overture? mozilla/4.0 |
10 | fiestagirl | 12:54 pm Jun 11, 2006 | ||||
| Looksmart crawling with Mozilla/4.0 User-agent Doesn't identify itself |
4 | jdMorgan | 7:30 pm Jun 9, 2006 | ||||
| Strange? Just a heads up |
4 | wilderness | 6:27 am Jun 8, 2006 | ||||
| mailer.hiphopcaucus.net Anyone see it? |
5 | youfoundjake | 6:21 am Jun 8, 2006 | ||||
| Surf Control ScSpider/0.2 |
5 | fiestagirl | 5:08 am Jun 8, 2006 | ||||
| GoogleBot/2.1 Has anyone else seen this UA? |
25 | GaryK | 1:36 am Jun 8, 2006 | ||||
| dp131.data.yahoo.com -- Mozilla/4.0 Yet Another Whatever from Yahoo that doesn't ask for robots.txt |
5 | Pfui | 12:06 am Jun 8, 2006 | ||||
| mp3Spider cn-search-devel at yahoo-inc dot com Yahoo China bot didn't read robots.txt |
3 | GaryK | 4:39 pm Jun 6, 2006 | ||||
| Kyluka |
10 | fusion5 | 10:16 pm Jun 5, 2006 | ||||
| Human or Spider? |
2 | besnette | 8:07 pm Jun 4, 2006 | ||||
| 404s show weird compound URLs This must be a bot but what are they looking for? |
9 | starhugger | 3:07 am May 31, 2006 | ||||
| Yeti |
3 | keyplyr | 5:21 pm May 30, 2006 | ||||
| FatBot+2.0;+www.FatLens.com Who is this guy? |
4 | hvacdirect | 5:07 am May 24, 2006 | ||||
| Favcollector/2.1 Qe? |
3 | Woz | 8:03 pm May 23, 2006 | ||||
| MSNPTC/1.0 on advertising.msn.com MSNPTC/1.0 spider of Microsoft AdCenter |
3 | woland71 | 2:35 pm May 23, 2006 |