Google indexing /feed URLs

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google indexing /feed URLs

Google is indexing my wordpress site as www.domain.com/post/feed

kektex

9:27 pm on Sep 15, 2006 (gmt 0)

Hello,
I´ve got a couple of wordpress blogs that I´ve been keeping for some time now (one is 1year+ and the other one about 8 months).Both sites have a lot of indexed pages,but for some reason google has been indexing them as www.domain.com/post/feed/ instead of just www.domain.com/post and they are all supplemental. I´m talking about 500 indexed /feed pages.
Most of the traffic coming in from google is now coming in to these /feed urls and that´s no good since that is just an xml feed and they just leave.
Both sites have updated sitemaps and I fixed the identical meta description on pages around 2 weeks ago (which I thought was causing the supplemental results problem).
Any ideas about what might be going on?
Thanks!

Bluesplinter

11:51 pm on Sep 15, 2006 (gmt 0)

I don't know, but I've run into it quite a bit on the searching side. I'll find a result that looks interesting, and click to find myself within an unintelligible jumble of XML.

As Nero Wolfe would say, Pfui.

lammert

11:56 pm on Sep 15, 2006 (gmt 0)

Google is indexing every readable content it finds, including XML files. The nice thing is that Google recognizes wildcards in robots.txt. You can add the following rule to your robots.txt:

User-agent: Googlebot
Disallow: /*/feed/$

This solved the problem for my WordPress based blog. You may have to wait one year before the supplemental results are gone. Google is slow with cleaning the supplemental index.

kektex

12:47 am on Sep 16, 2006 (gmt 0)

Thanks lammert I´m gonna give that a shot.
I have been searching for other blogs being affected by this and I haven´t found any, so I don´t know if it´s some setting in particular I set in wordpress or what is making this happen

Dead_Elvis

1:49 am on Sep 16, 2006 (gmt 0)

Yep, I stumbled onto this same problem about three months ago. It's not just particular to your site.

You might also check to see if Google is indexing your trackback URLs. I ultimately fixed it with a robots.txt file:

Disallow: /*/*/*/*/feed/
Disallow: /*/*/*/*/trackback/

Good luck!

kektex

3:51 am on Sep 16, 2006 (gmt 0)

Thanks Elvis!
Anyone have any idea how long it might take google to remove these results from the index?
On one of the sites I´m even seeing a bunch of those registrar parking pages! I regged the domain and I forgot to change the DNS while I got to using that domain and now there are like 20 results for www.example.com/?q=Hotel+Suites+Weddings and stuff like that.The worst part is that they actually go to the index page of the blog, is there actually any way to remove those results?

Ma2T

1:40 am on Sep 28, 2006 (gmt 0)

Easy answer for the future is simply to remove this feed link url from single.php. Then google will never find it.

I noticed the same thing, and this link is not important to me, so I simply deleted it.

Ma2T

1:45 am on Sep 28, 2006 (gmt 0)

"there are like 20 results for www.example.com/?q=Hotel+Suites+Weddings and stuff like that.The worst part is that they actually go to the index page of the blog, is there actually any way to remove those results? "

If someone knows the answer to this, let me know! I have almost the very same problem.

CainIV

5:54 am on Oct 6, 2006 (gmt 0)

Goto the Apache thread and do some reading on the information JD Morgan and friends have posted. I have found the answer there.

tedster

6:02 am on Oct 6, 2006 (gmt 0)

Is this the thread?

[webmasterworld.com...]