You could always do some scripting to check for a microsoft live reference in the HTTP_REFERER header.

This could be done via mod_rewrite, and send such hits over to a script for further processing, such as adding their IP to a 403 ban list, etc.

I don't want all from Live banned, just the ones coming in on one word searches, because they're scrapers.

I use this (in .htaccess):

RewriteCond %{HTTP_REFERER} (milf) [NC,OR]
RewriteCond %{HTTP_REFERER} (x x x) [NC,OR]
RewriteRule .* - [F]

to block the referral spammers, which will work for you too if all of your people are sending referrers.

If I am not mistaken, I got this code a while back off webmasterworld. :)

>>if all of your people are sending referrers

They aren't all unwelcome and not all are a PITA. Most visitors are 100% legitimate and more than welcome. If someone comes in from search using a search phrase such as "discount yellow widgets" or "yellow widget stores," then it's a living, breathing human being and they'll find what they're looking for. But if they arrive with the single search word "widgets" then it's software running for either data mining (no biggie) or to scrape for MFA sites, and the latter are a nuisance and represent a hazard for some sites if they go far enough, which many do.

I do know there's a way to sniff for s/e referral query strings and then process accordingly, but it was a long time ago, so my memory on it is very dim.

>>If I am not mistaken, I got this code a while back off webmasterworld.

Jake, I've been spending a lot of time "reading" in the code forums at WebmasterWorld (which I heartily recommend, rather than just copying and pasting), and unless I can "read" what something is doing, I won't use it until I know what it'll do.

RewriteCond %{HTTP_REFERER} (milf) [NC,OR]
RewriteCond %{HTTP_REFERER} (x x x) [NC,OR]
RewriteRule .* - [F]

Do (milf) and (x x x) represent what actual query words could be? If so, then that could do it.

[edited by: Marcia at 10:49 am (utc) on Feb. 18, 2008]

milf and x x x are actual words I filter, the x's of course need to remove the spaces because WebmasterWorld filter the "word"..

And I do not copy/paste anything until I believe it won't break my site either. Only posted the reference as a working, live example that already works. :)

I use it myself, which is why I mentioned "I got this code a while back off webmasterworld" -- not to claim this as my work.

Believe me, I do not spend my time looking for ways to make myself look smart by answering questions I don't understand.

By the way you described your problem.. the remedy sounded as simple as the htaccess filter I use to block spam bots, that is the only reason I posted it.

By the way you described your problem.. the remedy sounded as simple as the htaccess filter I use to block spam bots, that is the only reason I posted it.

If those are the actual query words, then that might just do the trick. I've got a fairly new domain not doing much of anything on other engines yet that gets those requests constantly, it sure couldn't hurt to try.

Thanks, will do!