Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Rash of strange requests for robots.txt today |
1 | SumGuy | 2:54 am Jan 4, 2025 | ||||
| Amazonbot again Amazonbot - block or allow? |
9 | jmccormac | 7:23 am Jan 2, 2025 | ||||
| Code200 UAB Corenet, Metacluster, Teso, Oxysales and Oxylabs |
3 | SumGuy | 10:48 pm Dec 18, 2024 | ||||
| Private prefetch proxy in Chrome These are the hits from fetch.tunnel.googlezip.net? |
6 | SumGuy | 3:22 pm Nov 18, 2024 | ||||
| Weird traffic from old model mobile phones |
11 | Dooku | 4:20 am Nov 6, 2024 | ||||
| Stange hits: Meta IPs, Google UA Spoofed UA? Scraper using Meta IP? |
1 | asterickx | 1:17 pm Nov 2, 2024 | ||||
| Increase in Facebook bot hits |
14 | dstiles | 8:53 pm Oct 23, 2024 | ||||
| GuzzleHttp/7 |
3 | not2easy | 12:53 am Oct 14, 2024 | ||||
| AwarioBot 37.27.129.xxx |
2 | Bewenched | 5:38 pm Oct 13, 2024 | ||||
| More on IPXO Getting their CIDR's from geofeed.txt |
1 | SumGuy | 1:15 am Sep 29, 2024 | ||||
| googlebot from Georgia ? |
5 | Wmff | 5:22 pm Sep 28, 2024 | ||||
| these ips are from google or pretending to be Googlebot |
16 | pramod | 4:25 pm Sep 28, 2024 | ||||
| go-resty/2.12.0 some new |
2 | SumGuy | 4:31 pm Sep 14, 2024 | ||||
| UA workona-favicon-service |
1 | Pfui | 7:45 am Sep 11, 2024 | ||||
| User-Agent: Custom-AsyncHttpClient (asked for a lot of crap) |
3 | SumGuy | 4:43 am Sep 9, 2024 | ||||
| Four-Minute Bytes Dive with me into the AWS weeds for a moment... |
3 | Pfui | 12:58 am Sep 6, 2024 | ||||
| meta-externalagent externalhotlink by another name? |
2 | lucy24 | 9:42 pm Sep 5, 2024 | ||||
| Redundant \ bot |
3 | not2easy | 9:13 pm Sep 1, 2024 | ||||
| Yet another "Cloud" Server with rogue bots 140.248.0.0/16 |
4 | Bewenched | 2:16 am Aug 30, 2024 | ||||
| Why is it that when I see GET /.env or POST /index.html It's always from a Microsoft Azure IP? |
1 | SumGuy | 12:46 am Aug 30, 2024 | ||||
| Nextcloud Server Crawler Seen today, a rare sighting |
1 | SumGuy | 11:45 pm Aug 18, 2024 | ||||
| mozilla What will they think of next? |
9 | lucy24 | 11:42 am Aug 13, 2024 | ||||
| PerplexityBot Perplexing |
5 | Pfui | 6:39 am Aug 12, 2024 | ||||
| PS Daily |
1 | lucy24 | 9:25 pm Aug 11, 2024 | ||||
| ChatGPT / openai bot First time I'm seeing them |
3 | SumGuy | 7:51 am Aug 10, 2024 |