frames an robots.txt

Forum Moderators: open

Message Too Old, No Replies

frames an robots.txt

are some frames invisible to search engines?

andrewsclothing

10:47 pm on Feb 27, 2008 (gmt 0)

I have inherited a page with frames, I know it is no good for search engines but it is all I have to work with.
I am wondering if crawlers can see any of the pages within the frames.
Googles' webmaster tools seems to tell me that it only follows links from my MENU frame, yet does not follow any links from my MAIN frame.

If this is the case how do I make the crawler see the links within these frames?
Do I use 'allow' in my robots.txt? If so will the crawler jump to those pages or do they still need a link to them for the crawler to follow?
Do I make a page with a sitemap, listing every page that loads in my main frame and link to this from my MENU frame?

(Please don't say redesign the whole page without frames as this is something I am currently unwilling to do.)

Thanks in advance for any ideas.

caveman

11:05 pm on Feb 27, 2008 (gmt 0)

Hey andrewsclothing, welcome to WebmasterWorld!

This was a pretty good old thread on the topic:

[webmasterworld.com...]

Be sure to also follow the link that tedster offered about halfway down the page on Pros and Cons.

Receptional Andy

11:05 pm on Feb 27, 2008 (gmt 0)

If you want the frames pages indexed (as opposed to performing well) then link to them from a page that is itself already indexed. Your frames page will then appear in search engine databases.

Please don't say redesign the whole page without frames as this is something I am currently unwilling to do

The objections to frames are IMO valid. I've come across few sites where frames conferred any usability advantage whatsoever, and this seems top be the only argument for their use. If you are unable or unwilling to change frames, then you'll have to accept compromises on performance.