Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Why would ebay visit my site? |
7 | smokeybarnable | 6:28 am Sep 11, 2006 | ||||
| Microsoft with no UA Microsoft hitting site with no UA |
2 | ssgumby | 9:15 am Sep 7, 2006 | ||||
| ActiveTouristBot |
16 | Mokita | 12:08 am Sep 7, 2006 | ||||
| Fatlens |
3 | smokeybarnable | 5:37 pm Sep 6, 2006 | ||||
| MSN IP list |
3 | Brett_Tabke | 11:07 pm Sep 4, 2006 | ||||
| Googlebot visit pages that not exist googlebot is indexing pages that we dont have links |
3 | ivanff | 11:23 am Sep 2, 2006 | ||||
| What is Net::Trackback/1.01? -Good/Bad? |
5 | fusion5 | 1:57 am Sep 1, 2006 | ||||
| Filtering Internet noise: That is counting only human visitors I want to improve my own web logging tools |
10 | vite_rts | 12:58 pm Aug 28, 2006 | ||||
| Is Inktomisearch spider ADHD? Yahoo! Spider requests only 1 or 2 pages per visit. |
2 | geoffyp | 4:12 am Aug 28, 2006 | ||||
| problem with JSESSIONIDs and robot crawlers JSESSIONIDs combined with the use of Struts with Hibernate |
5 | KBee | 6:31 pm Aug 25, 2006 | ||||
| Getty Images spider? |
5 | Stu_Rogers | 7:02 am Aug 22, 2006 | ||||
| "libwww-perl/5.65" from an IBM address block scanning for blog feeds? |
2 | zCat | 9:42 am Aug 21, 2006 | ||||
| blogsearchbot-pumpkin No Robots.txt |
2 | Ocean10000 | 4:42 am Aug 21, 2006 | ||||
| New Russian Spider webalta.net |
9 | GaryK | 10:10 pm Aug 20, 2006 | ||||
| Found an IP range that I am banning Located it via Google Analytics |
10 | Mokita | 4:16 am Aug 19, 2006 | ||||
| china |
4 | topr8 | 3:24 pm Aug 18, 2006 | ||||
| Fast Meta Crawler |
4 | TheFlipguy | 4:02 am Aug 18, 2006 | ||||
| List of bad/spam bots out there? |
16 | Grinler | 8:36 pm Aug 16, 2006 | ||||
| Yahoo improving? . maybe |
4 | Mokita | 12:33 am Aug 15, 2006 | ||||
| QFKBot pagefetch didn't ask for robots.txt |
2 | Mokita | 12:08 am Aug 15, 2006 | ||||
| Looks like a browser but acts like a crawler AppleWebKit/521.25 |
2 | GaryK | 9:09 pm Aug 13, 2006 | ||||
| 192.comAgent violates robots.txt |
5 | Mokita | 8:40 pm Aug 12, 2006 | ||||
| Googlebot-Image/1.0 Off the deep-end |
3 | wilderness | 10:40 pm Aug 11, 2006 | ||||
| Layered Technologies |
2 | wilderness | 9:19 pm Aug 11, 2006 | ||||
| lynx/2.8.5dev.16 libwww-fm/2.14 ssl-mm/1.4.1 openssl/0.9.7a |
6 | jake66 | 3:52 pm Aug 11, 2006 |