Robots.txt Disallow by Accident, SERP's Gone - HELP! - Webmaster General forum at WebmasterWorld

Forum Moderators: phranque

Message Too Old, No Replies

Robots.txt Disallow by Accident, SERP's Gone - HELP!

Robots.txt Error

tommyjon

1:51 pm on Apr 30, 2012 (gmt 0)

Hi All -

I made a crucial mistake in my Robots.txt file which told Google not to crawl site and take it out of the SERP's. Took me about a week or so to find the issue but at that time my site was taken out of the SERP's. It has been about 5 days since correcting the Robots.txt file and the Google bot is back crawling the site and the Cache has updated. In Webmaster Tools the blocked pages are going down.

I still have not seen any of my rankings come back and all traffic is gone, my site has about 2000 pages, PR3 and 1.5 year old. Can anyone tell me if my site will ever appear back in the SERP's? What is a good time frame to let Google sort this out? Could this be permanent?

g1smd

7:48 pm on Apr 30, 2012 (gmt 0)

It can take a few days or so. Be patient.

tommyjon

11:33 pm on Apr 30, 2012 (gmt 0)

Thanks for the reply, been almost a week and still my home page isn't even ranking? I will try not to panic... I just hope it comes back and this is not permanent :(

g1smd

11:56 pm on Apr 30, 2012 (gmt 0)

Run Xenu Linksleuth over your site and check what it finds.

sunnyujjawal

4:34 am on May 1, 2012 (gmt 0)

You can also try Google webmasters fetching feature .
Click on Diagnostics >> Fetch as Googlebot. Enter the URL's which you want to fetch . Click Fetch and the page will auto-refresh after 5 seconds and click "Submit to index".
I try this so many times and within 24 hours pages will be indexed.

Rest..Google never ignore Robots.txt...

lucy24

7:07 am on May 1, 2012 (gmt 0)

Google never ignore Robots.txt

Well, googlebot by that name never ignores...

tommyjon

1:36 pm on May 1, 2012 (gmt 0)

I used Google Fetch, still didn't work. My pages are indexed but not showing in the serps. What kind of scares me is that I got un-natural links warning but seemed to recover fine. Then once I changed the Robots.txt my site was out of the serps. Fixed the Robots.txt and here we are almost a week later and my site still isn't in the serps?

Did I get penalty? Google is crawling.... How long would you wait before I should start to worry?