Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| that Yahoo thingy |
2 | wilderness | 6:13 am Dec 28, 2012 | ||||
| Anyone know the name of Wayback Machine's robot? One site has not been crawled all year... wondering why. |
19 | JeffOstroff | 8:46 pm Dec 27, 2012 | ||||
| Human or robot? another one for the profilers |
4 | lucy24 | 9:09 am Dec 27, 2012 | ||||
| Ning/1.0 eat my 403 |
15 | Bewenched | 2:51 am Dec 27, 2012 | ||||
| TosCrawler |
14 | lucy24 | 6:55 pm Dec 25, 2012 | ||||
| WIndows NT 9.0 - Yeeeehaaa Back from the future! |
6 | blend27 | 10:54 pm Dec 23, 2012 | ||||
| Google scraping through China? |
7 | dstiles | 8:13 pm Dec 20, 2012 | ||||
| Data reseller bot shortlist? |
5 | Sgt_Kickaxe | 5:40 pm Dec 14, 2012 | ||||
| 'artviper(tm) 85.25.9.83 |
8 | Bewenched | 8:43 pm Dec 13, 2012 | ||||
| Microsoft- really? IP indicates microsoft, UA is strange |
4 | dupres01 | 8:23 pm Dec 13, 2012 | ||||
| Cloaked yahoo crawler |
6 | Bewenched | 5:41 am Dec 12, 2012 | ||||
| QippoBot/1.4+(+http://www.qippo.com/bot/) [2] ( 1 2 ) |
34 | Bewenched | 7:31 am Dec 11, 2012 | ||||
| DomainVader/1.0 |
3 | MxAngel | 9:42 pm Dec 10, 2012 | ||||
| CityGrid.com URL Validation 107.22.0.185 |
7 | Bewenched | 8:28 pm Dec 6, 2012 | ||||
| findlinks bot 130.83.167.153 |
4 | Bewenched | 8:00 am Dec 4, 2012 | ||||
| Yasni |
10 | incrediBILL | 3:03 pm Dec 3, 2012 | ||||
| Site Rating System |
3 | Ken_S | 2:22 am Dec 3, 2012 | ||||
| ChinaCache North America |
17 | incrediBILL | 8:57 pm Dec 1, 2012 | ||||
| Definitive Sources of IP Lists |
9 | incrediBILL | 8:42 pm Dec 1, 2012 | ||||
| More virtual servers Linode Cloud |
7 | Bewenched | 9:31 pm Nov 30, 2012 | ||||
| Interesting visit from Google Merchant Center? |
3 | Bewenched | 8:41 pm Nov 29, 2012 | ||||
| Rogue .pdf grabber 85.73.112.xx |
5 | Bewenched | 3:44 pm Nov 27, 2012 | ||||
| Big Fluffy Cloud cloud servers that is |
5 | Bewenched | 2:25 am Nov 27, 2012 | ||||
| More Chinanet 119.128.0.0 [2] ( 1 2 ) |
36 | not2easy | 7:29 pm Nov 26, 2012 | ||||
| GET Request |
6 | Ken_S | 10:03 pm Nov 25, 2012 |