Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Twisted PageGetter |
2 | Pfui | 3:58 pm Sep 17, 2009 | ||||
| findfiles.net/0.96 |
2 | keyplyr | 1:41 pm Sep 17, 2009 | ||||
| ScrapBook Site scraper bot; also discloses user's local file info |
27 | Pfui | 2:21 pm Sep 16, 2009 | ||||
| AutoPager Egads |
13 | incrediBILL | 10:03 am Sep 16, 2009 | ||||
| solelynetwork |
2 | dstiles | 9:44 pm Sep 14, 2009 | ||||
| www.travel-agent-finder.co.uk |
2 | GaryK | 8:42 am Sep 14, 2009 | ||||
| Firefox Ubuntu (jaunty) crawler or screen shots? From 09/08/24 to 09/13/24 and still going |
5 | incrediBILL | 9:33 pm Sep 13, 2009 | ||||
| open proxies major providers |
3 | wilderness | 10:47 pm Sep 10, 2009 | ||||
| Earthlink mobile |
3 | wilderness | 9:15 pm Sep 10, 2009 | ||||
| MJ12bot When you're a hammer, everything looks like a nail[2] ( 1 2 ) |
38 | Pfui | 7:18 pm Sep 7, 2009 | ||||
| searchevabot |
2 | Pfui | 8:04 am Sep 7, 2009 | ||||
| MSN fakes referrers [5] ( 1 2 3 4 5 ) |
131 | SEOPTI | 6:47 am Sep 7, 2009 | ||||
| Known Spider Information and References |
12 | incrediBILL | 9:15 am Sep 5, 2009 | ||||
| User-agents get longer and. |
3 | dstiles | 1:24 am Sep 4, 2009 | ||||
| Bot from IANA |
6 | nativenewyorker | 5:12 pm Sep 3, 2009 | ||||
| Viralheat Bot |
11 | Pfui | 11:50 pm Sep 1, 2009 | ||||
| SindiceFetcher/sig.ma |
2 | keyplyr | 9:20 am Aug 30, 2009 | ||||
| arste.info bot |
2 | Pfui | 8:35 am Aug 30, 2009 | ||||
| jc unknown var cloak.daily-2009-08-25-62-00.vars.referer |
5 | 1EightT | 10:22 pm Aug 26, 2009 | ||||
| GingerCrawler/1.0 |
12 | Mokita | 4:18 pm Aug 23, 2009 | ||||
| FunWebProducts, is it more than just a browser add-on? |
4 | JS_Harris | 3:50 am Aug 23, 2009 | ||||
| Is returning a blank page to known search crawlers acceptable? Filtering search crawler clicks to affiliate links |
3 | JS_Harris | 12:26 am Aug 23, 2009 | ||||
| Google Relative links |
6 | wilderness | 7:11 am Aug 21, 2009 | ||||
| Toata dragostea mea pentru diavola All my love is for the devil |
3 | Ocean10000 | 4:34 pm Aug 20, 2009 | ||||
| Windows 3. |
3 | wilderness | 4:28 pm Aug 20, 2009 |