Forum Moderators: open
User-agent: *
Disallow: /
Does this mean that the site has blocked every attempt to spider it in the last few years, and that I have to resubmit it everywhere? I deleted the robots.txt file because I wasn't sure how I would change it to allow spidering from major search engines.
Does this mean that the site has blocked every attempt to spider it in the last few years, and that I have to resubmit it everywhere?
Yes, the site has been Disallowed from indexing.
No need to resubmit anywhere. Just remove the robots.txt file or remove the / from the Disallow line and you'll be fine. Although you may want to look at Disallowing other parts of the site.
If there are links pointing to that site, which there probably are after a couple of years, you'll start to see the various bots traversing the site over the next few weeks (typically).
robots.txt Forum [webmasterworld.com]
