In my WMT, when you do a "fetch as googlebot", after it reports "success", a new option "submit to index" appears. So clicking that option might help to speed up re-indexing, at least it's worth a try.

aristotle the pages are still indexed as when i run a site:www.mysite.com search they are all there. it is just as if google does not want to show them in the search. these are pages that have ranked 12345 for certain terms for years....but now magically disappeared since a robot.txt block for three days. i really hope its just a case of it needing to do a deep crawl again

Try this test:
Take a sentence of at least 12 words from one of your pages, enclose the whole sentence in quotation marks, and do a google search for it. It should be a sentence that isn't likely to be found on any other site.

If your page with that sentence has been scraped, and your site is penalized, scrapers may rank above you, but your page should also show up if it is really indexed. (But you might have to click "show all results" to see your page)

the page i chose would not have been scraped....as many are...but my page came up first in search...so the pages are still indexed.... but it seems G is not showing them as it was not able to crawl the site for three days and it is not sure what it might be rendering? am i right in thinking that?

Meelosh relax. Here is a story to cheer you up. Well, you know I had two naughty Tumblrs that brought in together about 1,000 visitors a day. I don't know if you followed the story about Tumblr that when it was bought by Yahoo it started to crack down on such blogs. Basically, what Tumblr did was to add noindex tag to the template. In about a week, G deindexed almost all the pages and images and traffic declined to zero in about 10 days.

After a big outcry, about three months, Tumblr backtracked and decided to take off the noindex tag. G started to reindex the pages and was able to crawl them back in about a week, though, the caches were slow to update. The best part: almost all the traffic came back as if nothing changed.

the page i chose would not have been scraped....as many are...but my page came up first in search...so the pages are still indexed.... but it seems G is not showing them as it was not able to crawl the site for three days and it is not sure what it might be rendering? am i right in thinking that?

What happens when you do the test on a page that has been scraped? If your page ranks first, it's a good sign. But if the scrapers rank above you, it usually indicates that your site is penalized, but in this case could mean a temporary demotion due to the recent server problems.

I've accidentally blocked googlebot on a site by uploading the wrong robots.txt file. I spotted the problem 3 days later when half my Google traffic was gone. I fixed the robots.txt file and the traffic kept falling for another day before rebounding to the original level 3 days after that. It happens, just let Googlebot do its thing when you've made sure it's no longer blocked.

Note: this was several years ago(2007?), google didn't hold it against me then, I don't know for certain that they still don't hold that type of mistake against webmasters now. I don't see why they would.

Go with JS_Harris' experience. This happens all the time, probably about one post per month on this board alone, and Google usually sorts it out within a matter of days. It's *normal*, no reason to panic, no reason to talk of penalties.

My two very recent experiences from this year:

1) Site A, robots.txt returning HTTP 500, site dropped within 2 weeks, fixing robots.txt resulted in traffic recovering within 4 days (most traffic was arriving to home page, which was the first to recover)

2) Site B, robots.txt uploaded with "Disallow: /" from pre-production domain. Site sunk in ranking five days later, which is when the problem was noticed. Once the correct robots.txt was uploaded, the site fully recovered traffic 2.5 weeks later. The reason for slower recovery of the traffic was because lots of traffic came from inner pages that took longer to recover (home page recovered ranking couple of days after the correct robots.txt was uploaded, but it took Google two and a half weeks to recover inner pages to previous ranking levels).

In both cases, after the fix, Fetch as Googlebot (via WMT) was done firstly for robots.txt to check Google got it ok, then Fetch as Googlebot was done again for the domain root and the "Submit URL and all linked pages to index" was used for the domain root.

Therefore, if most of your traffic is comming from home page or the smaller number of pages linked directly/closely to the home page, the recovery is faster. If there is a significant portion of traffic that comes from many internal pages that are deeper linked, the recovery is slower. In any case, you should expect the recovery within few weeks. However, large sites may experience slower recovery.

Thanks for all the "stories" and just as an update....the bot did a deep crawl last night and now atleast the main pages are being shown in search and hopefully full recovery over the next few days.