Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| BOTW Spider |
2 | Mokita | 12:13 am Jan 10, 2007 | ||||
| WaybackMachine and Scrapers Do scrapers scrape archive.org? |
2 | Umbra | 7:32 pm Jan 9, 2007 | ||||
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;) Repeated hits by this UA, with thedomain.com as referer.[2] ( 1 2 ) |
33 | montclairguy | 2:03 pm Jan 4, 2007 | ||||
| Sogou spider? Good or Bad? |
2 | phi148 | 3:28 pm Jan 2, 2007 | ||||
| fake google bot? |
6 | pp46 | 11:19 am Jan 2, 2007 | ||||
| LinkWalker/2.0 Anyone knows what this bot does? |
3 | ibnuasad | 9:29 pm Jan 1, 2007 | ||||
| bot/1.0 (bot; http://; bot@bot.bot) |
7 | Mokita | 7:41 am Dec 29, 2006 | ||||
| Yodaobot 1.0 is coming Do you notice the Outfoxbot 0.x? Last weekend it released as Yodaobot 1.0 |
16 | chedong | 6:04 pm Dec 28, 2006 | ||||
| Exabot-Test/1.0 |
2 | GaryK | 6:35 am Dec 25, 2006 | ||||
| Yahoo invents browser that doesn't need an OS! Y! sitemap checker |
2 | jdMorgan | 5:05 pm Dec 22, 2006 | ||||
| IE7 Save Site UA? |
7 | keyplyr | 9:58 am Dec 21, 2006 | ||||
| User-Agent begins with "User Agent::" |
4 | Umbra | 12:51 am Dec 19, 2006 | ||||
| WorQmada/1.0 |
6 | bobothecat | 6:02 am Dec 18, 2006 | ||||
| Bigsearch.ca new Canadian search engine currently Beta |
3 | Mokita | 2:58 am Dec 16, 2006 | ||||
| MSNBot from APNIC? Is this a spoofed user-agent? |
5 | Kadence | 7:53 pm Dec 15, 2006 | ||||
| MSNPTC/1.0 Bot Gone Wild MSN's AdCenter Bot |
4 | speakSEM | 7:31 pm Dec 14, 2006 | ||||
| new disguise of rippers? |
8 | the_nerd | 3:37 am Dec 14, 2006 | ||||
| Analyzing requests on every pageview vs. Every 1 minute Pros & cons... |
8 | MichaelBluejay | 2:34 am Dec 14, 2006 | ||||
| HMSE Robot Not a clue what it is. |
3 | GaryK | 1:02 am Dec 14, 2006 | ||||
| New Search Engine Spider IPs |
4 | volatilegx | 3:31 pm Dec 13, 2006 | ||||
| Nokia6600 |
14 | wilderness | 10:37 pm Dec 10, 2006 | ||||
| Yahoo Bots |
8 | sem4u | 11:02 am Dec 6, 2006 | ||||
| Amazon |
3 | wilderness | 3:32 pm Dec 2, 2006 | ||||
| Is Ebay attempting wholesale site downloads? Regular attempts from the EBay Network to download my entire site. |
24 | AlexK | 12:32 am Nov 30, 2006 | ||||
| masidani bot v0.6(www.masidani.com) Holiday shopping bot |
3 | GaryK | 12:18 am Nov 27, 2006 |
