Toggle navigation
Dashboard
Dashboard Tracker
Add / Edit My Forums
System Announcements
Discussion
Active Thread Feed
Top Threads Today
Top Threads This Week
Top Threads Library
Go!
View All
View All
Login
Logout
Home
Forums Index
General SEO Issues
/
Crawler, Spider, and User Agent ID
10:33 pm Sep 21, 2026
This page requires javascript.
Forum Moderators:
open
Message Too Old, No Replies
newspaper
keyplyr
Msg#:4799266
11:19 pm on Apr 5, 2016
(gmt 0)
UA: newspaper/0.0.9.8
Protocol: HTTP/1.1
Robots.txt: No
Host: Various
Simplified python article discovery & extraction. Coming from Google Cloud but could be run from anywhere.
keyplyr
Msg#:4813260
9:39 pm on Jul 23, 2016
(gmt 0)
UA: newspaper/0.1.7
Protocol: HTTP/1.1
Robots.txt: No
Host: AWS
52.28.0.0 - 52.28.255.255
52.28.0.0/16
Join The Conversation
Register
For Free! -
Become a
Pro Member
!
See forum categories
-
Enter the Forum
Moderators and Top Contributors
Moderator List
| Top Contributors:
This Week
,
This Month
,
Aug
,
Jul
,
Archive
,
Top 100 All Time
,
Top Voted Members
Hot Threads This Week
September 2026 Google Search Observations
AdSense Earnings and Observations - September 2026
An old fashioned desktop WYSIWIG HTML editor
Home
Forums Index
General SEO Issues
/
Crawler, Spider, and User Agent ID
10:33 pm Sep 21, 2026