Am I correct that what you want is to prevent the bots from those search engines from accessing your site?

Why not just exclude the bots from the pages you don't want spidered?

One way to just add a META tag to the pages in question, like: <meta name="robots" content="noindex,nofollow">

If you don't have a robots.txt file for your site, you should add one of those too, and you can use that file to exclude the bots from any area(s) of the site you wish.

As there have been occasional reports of some bots from the major engines at times missing or disobeying these kinds of instructions, to be thorough, we usually employ both methods and it seems to work fine.

Why not just exclude the bots from the pages you don't want spidered?

I'm guessing that what muzuk786 is looking for is a way to allow good bots to spider his pages but a way to exclude the bad bots. If that's the case, I wouldn't use the robots meta tag.

I'm also guessing that while he's saying 301, he may be thinking about using the .htaccess file to ban bots. There have been a series of threads in WebmasterWorld over the years on this subject, with a list of the bad bots. Here's I believe the latest of these threads...

A Close to perfect .htaccess ban list - Part 3
More tips and tricks for banning those pesky "problem bots!"
[webmasterworld.com...]

Also, try further Google searching of WebmasterWorld, with this kind of search:

ban bots .htaccess site:webmasterworld.com

[edited by: Robert_Charlton at 3:52 am (utc) on Aug. 28, 2006]