I can't say I've been seeing any more of this nasty practice than in the past - at least not in the top SERPs. I do, however, find a growing number of sites that are doing it. Someone is clearly publicizing some bad advice - based on trying to squeeze every last drop of PR out of the site, I suppose.

The right way is a true 404 status, perhaps with a custom error message that might look like the home page. But when you do things like that, the URL returns a 404 status and will no longer rank.

Are you seeing a true 404 status in the http headers in these cases? If so, sounds like Google has a problem. I'd expect to see a 301 to a 200 for the URL to stay in the results.

Although those sites might be listed in the SERPs, you can't see whether they have a warning about "soft 404 errors" in their WebmasterTools data.

I tested few websites again.

They are some way 302 redirecting any "not available" URL to home page or another page(Sorry first I thought its a 301, seems like all using 302 after carefully checking).(some show a 404 then redirect, and some redirect directly).

So if you start guessing, trying random URL's, you wont see a 404, you will be redirected some way or another.

Some also will do no redirect, but will serve any url you ask for, like /root/whatever will show /root/ index page.

They are ranking for the home page, no one is ranking for an not-available/redirected page, those redirected pages are not indexed for sure.

They'll eventually be flagged as having low technical quality.

some show a 404 then redirect

What do you mean by that?
- Return a 404 status code in the HTTP header then perform a meta refresh to some other URL?
- Show the words "404 Not Found" on the page while returning a different status code in the HTTP header?

After returning a HTTP 404 status code, that's the end result. It's not possible to perform a HTTP redirect after that.

The only way to see multiple status codes is if the first one is a 30x redirect. There may be multiple redirects in a chain, and that's bad too. The final code will be a 2xx or 4xx code.

Yes

The website Return a 404 status code in the HTTP header then perform a meta refresh to some other URL?

The response body show:
<meta HTTP-EQUIV="REFRESH" content="0; url=http://www.example.com">

This is a website which is strictly capturing the most searched keyword in its language.

The website taking 1st result for English for same industry also use the 302 redirect!

As tedster said, I feel some one is adopting this school and seems like many using it.

The question remains, if this have something to do with their good ranking!

It'll have something to do with their eventual downfall I would hope.

Even we all know that redirecting 404 errors to home page (or another page) isn't the right thing to do.

No we don't know. Based on what? Rumors and spins? You need to give priority to the site's usability. A 404 page has it's uses but for non-existing requests, I would do 301 to the most appropriate place.

Having said that you need to ensure the 301s are done correctly. Incorrectly setup 301s can damage ranking so if you're uncertain leave the default error handlers in place.

As far as Google is concerned, I don't think they will care about a meta-refresh on the 404 page. Googlebot can still tell which pages are not found. If a meta-refresh is a better user experience than showing a generic 404 page, then there shouldn't be too much wrong with it.

That being said, I can't imagine that refreshing the page of users is as good as showing a truly helpful 404 page right off the bat. Why redirect to the home page, when you could put home page content right on a custom 404 page and save visitors the hassle of the refresh?

I also find that it is usually worth paying lots of attention to 404s on your website. I correct all incoming broken links using 301 redirects. I also try to run the url through site search and redirect based on the first result. So example.com/keyword will redirect to the actual url example.com/article?id=4.

I highly doubt that what these websites are doing with their 404 pages is contributing to their rankings much, but I would recommend paying attention to your 404s to keep your users happy.

I correct all incoming broken links using 301 redirects.

I used to so that, but with the huge rise in "broken links from bad places" I take a far more cautionary view these days.

Sorry to bring this one back, but I have a question about this. I have went through and redirected a client's top 200 landing pages to corresponding pages on the new site (same domain). They still have 1000+ product pages left, which would take forever to do the same.

Would it be better to...

1) Create a custom 404 page, that redirects within 10 seconds.
2) Create just a custom 404 page with content similiar to the homepage
3) Redirect these remaining pages to the homepage

Thanks,

Veto

1. no
2. maybe
3. no

Would the only definite 'yes' be to redirect them all to the correct pages on the 'new' site?...or do you have a better solution?

4) Return a 410 for these pages and make a custom 410 page with information targeted at people who know what they're looking for. Only feasible of course if those 1000+ URLs have some unfying element that lets you collect them all into a single 410. For example if you've changed from long messy URLs to short pretty ones: then a simple look at THE_REQUEST (or, ahem, IIS equivalent) should do it.

One site I visit regularly has displayed a "we're remodeling" page for several weeks now. Fortunately it's got a Search function that works-- at least for the specific column I read there.

The only definite 'yes' would be to redirect them all to the correct pages on the 'new' site.

It sounds like you are taking the correct approach by starting with the most important pages. But, only redirecting 17% of the urls correctly is an SEO disaster waiting to happen.

I would look for a way to automate the process. For example if there is site search available, I would write a script to plug the title of the page in to try to find the new url.