Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| csci_b659/0.13 |
3 | bull | 7:07 pm Apr 20, 2006 | ||||
| noyona_0_1 |
3 | Staffa | 3:34 pm Apr 20, 2006 | ||||
| WebCopier v4.3? What is it? Found this in my logs |
4 | Essex_boy | 7:59 pm Apr 19, 2006 | ||||
| 66.151.103.144 & 66.151.103.181 |
2 | hfactor | 11:14 am Apr 19, 2006 | ||||
| Yahoo! Mindset Didn't ask for robots.txt |
8 | Pfui | 11:07 pm Apr 15, 2006 | ||||
| LG/U8120/v1.0 ignoring robots, nocache tags |
6 | keyplyr | 2:43 pm Apr 14, 2006 | ||||
| Strange Yahoo Bot I think |
3 | innocbystr | 7:31 pm Apr 13, 2006 | ||||
| What is the name the Yahoo image bot? or is it a seperate bot? |
5 | Iczer | 2:15 pm Apr 12, 2006 | ||||
| Interesting from Yahoo! |
3 | volatilegx | 12:29 pm Apr 12, 2006 | ||||
| 64.233.173.67 Google spider |
14 | Key_Master | 3:03 am Apr 11, 2006 | ||||
| schibstedsokbot |
4 | bobothecat | 9:23 pm Apr 10, 2006 | ||||
| Google Wireless Transcoder |
3 | Umbra | 6:22 pm Apr 7, 2006 | ||||
| InfoPath.1 How to block |
3 | JadedJane | 4:28 pm Apr 7, 2006 | ||||
| Strange spidering something is trying to find non existing files... |
5 | netchicken1 | 12:15 pm Apr 6, 2006 | ||||
| Comrite/0.7.1 comrot robot identification query |
5 | fusion5 | 11:33 pm Apr 3, 2006 | ||||
| rel="nofollow" confusion don't follow |
7 | Drumat5280 | 8:49 pm Apr 3, 2006 | ||||
| Spider Design walking the tightrope |
5 | BerndH | 7:27 pm Apr 3, 2006 | ||||
| TeragramCrawler how to block this search engine crawler |
3 | mswiryn | 11:06 pm Mar 31, 2006 | ||||
| FreshNotes crawler |
4 | keyplyr | 11:07 pm Mar 29, 2006 | ||||
| Can I ban Mediapartners-Google/2.1? |
4 | joker197cinque | 6:31 pm Mar 28, 2006 | ||||
| differences between real and fake googlebots how to recognize real googlebot |
8 | hurgada | 3:52 am Mar 28, 2006 | ||||
| Current Googlebot IP's For geo-targetting between UK and US sites |
4 | TinkyWinky | 3:48 am Mar 28, 2006 | ||||
| csci_b659/0.13 |
3 | wilderness | 11:03 pm Mar 22, 2006 | ||||
| Msn? |
2 | wilderness | 8:35 pm Mar 22, 2006 | ||||
| more Hurricane |
2 | wilderness | 8:16 pm Mar 22, 2006 |