i've been getting this on and off for the last week - any ideas as to what is going on or what can we do to stop it?
bingdude
9:04 pm on Nov 11, 2011 (gmt 0)
Responded to both via IM. ;)
SEOPTI
6:55 am on Nov 13, 2011 (gmt 0)
gethan:
You are member since 2001, I believe you should have your own server and even under biggest crawler load you will stay at 1% CPU and 1% RAM usage. Or are you with 1000+ other people on a VPS machine which consts $ 3.99/month, if this is the case I believe website creation is your hobby.
I see no problem at all, Bing is hitting my sites with 100k+ bot requests/day.
You should feel lucky and thankful they try to crawl and index your site, send you traffic, send you conversions, people are crying because they don't see bingbot for months.
Sending mails to their abuse department because they crawl you hard is pointless and complete nonsense.
If you don't want their traffic just set your crawl rate in webmaster tools as low as possible or block them in robots.txt.
I love Bing traffic and their awesome traffic conversion levels so I go for the highest possible crawl rate day and night.
If you are completely unable to handle few bingbot requests how do you treat GBot and scraper requests, they are hitting your site 10 times harder.
Look at your code, probably your php/mysql code/queries were poorly written or you have not optimized your DB (indexes ...).
