Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Is the robots.txt header imp? It is passing html header to plain text |
5 | AjiNIMC | 6:15 am Jun 15, 2006 | ||||
| robots.txt format for different SE need help to findout correct format |
2 | token128 | 12:38 am Jun 10, 2006 | ||||
| simple robots.txt clarification folders within folders |
5 | headown | 8:55 pm Jun 9, 2006 | ||||
| Recently added robots.txt, validated, but pages still showing up Validated in Google Sitemaps but pages still indexed |
2 | blueheelers | 8:45 am Jun 6, 2006 | ||||
| Alexa revealing too much information once again Alexa revealing too much information once again |
2 | Gomvents | 12:18 pm Jun 5, 2006 | ||||
| wildcard |
8 | stevelibby | 12:53 am May 31, 2006 | ||||
| AdsBot-Google's robots.txt specs (FYI from the "Google AdWords" Forum) |
1 | Pfui | 12:48 am May 28, 2006 | ||||
| can someone please check this. can someone confirm this robots.txt is correct... |
3 | keentolearn | 9:51 am May 27, 2006 | ||||
| Sub Domain Issue |
2 | is300 | 4:05 pm May 25, 2006 | ||||
| Cache renders excluded CSS? |
8 | glengara | 10:16 am May 24, 2006 | ||||
| Having 2 versions of a site, how to not get one spidered Non SE friendly and SEO site |
1 | epadgett | 4:28 pm May 23, 2006 | ||||
| Robots.txt Validation - Robots.txt |
3 | Suman | 1:09 am May 23, 2006 | ||||
| robots.txt and https |
2 | AzDude | 8:56 pm May 18, 2006 | ||||
| yahoo slurping for www.file1.html/robots.txt what's wrong that slurp can't figure out? |
3 | paybacksa | 9:08 am May 14, 2006 | ||||
| Do search Engines crawl and spider registered content? Spiders, crawlers, indexing, registered content |
6 | Oimachi2 | 9:17 am May 13, 2006 | ||||
| How do i add a robots.txt.? New to all this and havent got a clue |
3 | gooders2274 | 1:00 pm May 12, 2006 | ||||
| Cant see competions robots.txt |
4 | da95649 | 12:56 pm May 12, 2006 | ||||
| oodle crawler how to block it? |
2 | moneybagsuk | 12:52 am May 10, 2006 | ||||
| Robots and SMF PHP Forum Script |
1 | freewebsiteideas | 11:20 pm May 9, 2006 | ||||
| Subdomains, DNS, IIS How to handle robots.txt requests for multiple subdomains... |
1 | jtoddv | 9:23 pm May 8, 2006 | ||||
| Should I use this template? Very silly question of a beginner... |
4 | wolfadeus | 9:53 am May 4, 2006 | ||||
| Robots "follow command." Whether this command works or makes them leave. |
3 | 1simone | 4:38 pm May 2, 2006 | ||||
| how to set joomla robots.txt file Joomla Robots |
2 | cafedelmar | 4:37 pm Apr 24, 2006 | ||||
| Yahoo Robots Going Crazy |
3 | belfastboy | 7:39 pm Apr 23, 2006 | ||||
| Need Trailing Slash Clarification |
8 | Israel | 11:17 pm Apr 22, 2006 |
