View Profile: phred Personal Information: Member: phred Votes: 0 Member Since: May 11, 2008 Last Post: archived Total Posts: 55 Interests: Biography: Bio Page URL: Social Media Links: Location Information: Company Information: Recent Activity: Recent Messages xpath saveHTML - how?   in (deprecated) XML Development Robots.txt - Disallow: or Disallow: /   in Sitemaps, Meta Data, and robots.txt Hailoo Search   in Crawler, Spider, and User Agent ID block whole countries   in Crawler, Spider, and User Agent ID amazonaws.com plays host to wide variety of bad bots   in Crawler, Spider, and User Agent ID verify-Compliance Notice   in Crawler, Spider, and User Agent ID Kintiskton   in Crawler, Spider, and User Agent ID Iana   in Crawler, Spider, and User Agent ID Do Bot-Blocking Techniques Alter Bot Behavior?   in Crawler, Spider, and User Agent ID WordTracker Attempts Crawling My Site!   in Crawler, Spider, and User Agent ID Curious sequence of events   in Crawler, Spider, and User Agent ID Non-white list blocking   in Sitemaps, Meta Data, and robots.txt Whitevector Crawler   in Crawler, Spider, and User Agent ID fdfdkll.html   in Crawler, Spider, and User Agent ID UA - EmailSearch   in Crawler, Spider, and User Agent ID Bots, spiders, harvesters - IPV6   in Crawler, Spider, and User Agent ID myvzw.com aka vzw.msn.com   in Crawler, Spider, and User Agent ID Yahoo Slurp - no hostname   in Crawler, Spider, and User Agent ID Getting Google to index more than 1 page   in Google Search and SEO CatchBot   in Crawler, Spider, and User Agent ID MSN bot finds php robots.txt   in Crawler, Spider, and User Agent ID Rewrite robots.txt to php script   in Apache Web Server Googlebot / image bot causing probs?   in Crawler, Spider, and User Agent ID How does Google know?   in PHP Server Side Scripting AOL changing IPs   in Crawler, Spider, and User Agent ID Are all UAs ending in "SV1)" invalid?   in Crawler, Spider, and User Agent ID If-Modified-Since - Internal 500 error   in Apache Web Server