Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Robots Meta Tag & nofollow

         

vetofunk

7:59 pm on Jan 11, 2008 (gmt 0)

10+ Year Member



I just found a piece code on each product page of a buddies site of mine.

The code is as follows:
<meta name = "robots" content="nofollow">

This pretty much tells the search engines not to pass and link juice through all links on the page, correct?

tedster

9:17 pm on Jan 11, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes - it even says "don't follow the links on this page", so the target urls for these links will only be indexed if there is another method to locate them. This would even apply to internal links on the site, so that particular robots meta-tag would stop the circulation of PR and other link influence throughout the site.

Lord Majestic

9:22 pm on Jan 11, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The design of nofollow in META tag was to prevent bots from following links on that page, whereas design of wrongly named rel="nofollow" link attribute was actually to tell search engines that passing rank value down this link might not be a good idea - there is nothing limiting passing rank in the former and actual following of links in the latter.

So if you don't want "juice" to be passed then you are better off making sure all links are marked with rel="nofollow" rather than using META.

vetofunk

9:35 pm on Jan 11, 2008 (gmt 0)

10+ Year Member



His Pagerank on all his pages went downhill the last 3 months and I am wondering this had something to do with it.

Robert Charlton

9:36 am on Jan 12, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Sometimes these metas are put on development pages or templates to keep them out of the index while they're being developed, but then accidentally left on when pages go live.

Could this be the situation here?

[edited by: Robert_Charlton at 9:36 am (utc) on Jan. 12, 2008]

vetofunk

4:09 pm on Jan 14, 2008 (gmt 0)

10+ Year Member



Thanks for the replies everyone. I am waiting to hear back from developer to see why this was added.

vetofunk

4:11 pm on Jan 16, 2008 (gmt 0)

10+ Year Member



You were right Robert, she said this was something added when the new site was developed. It has now been removed.

One more question for everyone. We have the following tag on all the "Print" pages (they have many stores and people bring in the printed products):

<meta name = "robots" content="noindex, nofollow, noarchive">

I wanted to make sure that the search engines didn't spider these page as they are almost identical to product pages. Is this enough and the best way to do it? I also have a no follow tag on the links pointing to the "Print" page.

Lord Majestic

4:13 pm on Jan 16, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Best to use robots.txt to disallow crawling of print pages explicitly.

phranque

12:06 am on Jan 17, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



vetofunk:

i think you should also add nofollow to the links to your print pages...

Robert Charlton

7:50 am on Jan 17, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



One more question for everyone. We have the following tag on all the "Print" pages (they have many stores and people bring in the printed products):

<meta name = "robots" content="noindex, nofollow, noarchive">

Best to use robots.txt to disallow crawling of print pages explicitly.

The most explicit way to disallow all references to the print pages is to use the robots noindex meta tag, but not also to use robots.txt. If you use robots.txt to block a print page, it is true that Google won't spider the content of the page, but that prevents Google from reading the robots meta nofollow on the page. In that event, Google will index the url of the print page if the link is publicly available, albeit it won't show any content from the page. This url may or may not end up appearing in a Google search.

By itself, the noindex attribute in the link to the page won't prevent the page from being indexed by Google, because it's possible for Google to follow another link to this page, or to spider it via urls it finds in publicly accessible log files.

And... if you had the noindex meta on the print page, but you had a noindex attribute in the link to the print page, and that was the only link to the print page, it's not clear how that would affect Google's indexing the url. ;) Google wouldn't spider the print page and therefore wouldn't see the noindex meta. Google would index the link anchor text as part of the source page content, but opinions and logic varies on whether Google would index that text as a link anchor.

See discussions about links with the noindex attribute showing up in some backlinks to a page....

Nofollow links showing up in backlinks
[webmasterworld.com...]

phranque

9:00 am on Jan 17, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



from the transcripts of stephan spencer's interview of matt cutts at pubcon 2007 [mattcutts.com]:
At least for Google, we have taken a very clear stance that those links are not even used for discovery; they are not used for PageRank; they are not used for anchor text in any way.

(the full text of that q&a is almost halfway in the transcript)

Robert Charlton

9:15 am on Jan 17, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



phranque - Thanks for the interview reference. I quote pretty much the same language from another interview in the post I cited above about nofollow links showing up in backlinks.

I said...

...then you've got to wonder how the links appear in the index at all. Not "even... for discovery" is a pretty strong statement... and, as I note in the previous discussion, "the nofollow meta tag is what you use when you want to prevent Google from indexing the link itself."

My dilemma is I can't reconcile this with the original post, and a similar observation on another thread I cite, that people are seeing nofollow links appearing in Google backlinks.

One thought that comes to mind is that the links may have been indexed before they were nofollowed, and perhaps hadn't been purged from the backlinks index at the time they were seen.

phranque

1:38 pm on Jan 17, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



he mentions in the interview that at one time there was a strange set of circumstances that could have caused it but they considered it a bug and therefore fixed the problem.

don't remember if he mentions the time frame...

g1smd

11:10 pm on Jan 17, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The robots disallow directive still allows the disallowed URL to accumulate PageRank - which is completely wasted.

Print-friendly pages are usually best done using CSS. There is no separate URL involved for those at all then.