Oh, I also did a couple of other things:

1. I used the no translate tag for Google Translator:

<meta name="google" content="notranslate">


2. I am trying a robots.txt whitelist (shown below). Yes, I know not all bots honor it but it's worth a try if keeps a few off before they even get started visiting my site.


User-agent: *
Crawl-delay: 10

User-agent: googlebot
Disallow:

User-agent: Mediapartners-Google
Disallow:

User-agent: AdsBot-Google
Disallow:

User-agent: msnbot
Disallow:

User-agent: Mediapartners
Disallow:

User-agent: Slurp
Disallow:

User-agent: bingbot
Disallow:

User-agent: jeeves
Disallow:

User-agent: ia_archiver
Disallow: /

User-agent: *
Disallow: /

Thanks for the kind words!

It's nice to know when others are helped by such simple advice.

You can block counties for free in the .htaccess file and there are lots of sites out there that give you cut & paste code to drop in so you don't have to know much except how to edit a file to make it work.

My experience with blocking some abusive countries was a huge initial spike in AdSense but it then dropped again and leveled off, but it leveled off higher than previously because I no longer had all the junk traffic.

Google tells you not to block people from accessing your website but they also penalize you for doing it and don't penalize those that rip you off and often reward them better than your own site!

Makes no sense whatsoever.

Thanks, Bill. That's good to know.

We'll see what happens with Google. My blog is linked to from prominent publications in its niche, so maybe that will help with google. I don't know.

But I am going to run this as is for a month or two, then I'll see what's what. I do NOT miss the foreign traffic server headaches at all! This is the first time I've been able to relax about it for a while. The revenue wasn't worth the headaches.

Google tells you not to block people from accessing your website but they also penalize you for doing it and don't penalize those that rip you off and often reward them better than your own site!

Makes no sense to any of us I think. Not that I block other countries because that is polar opposite to the primary purpose of my sites. Educational for all.

Not that I block other countries because that is polar opposite to the primary purpose of my sites.

I have the same problem with my biggest traffic site in that it's global but I finally couldn't take it anymore and China got blocked in the firewall, not just port 80, because of the hackers, spam and scrape attacks. No matter how much technology you put out there to stop the scraping, the speed they would hit the server, asking for hundreds of pages a minute, because of it's database back end,could render it useless and a server than can't respond doesn't do anyone any good so I took it to the firewall but that's just me.

Since then I've also got way more horsepower and now cache pages so the site can sustain a much heaver load these days but the scrapers are also more powerful with bigger bandwidth than ever before so it's a wash.

If I were going to do business with Asia I think I'd set up a second server and route all the Asian traffic to that box and let the rest of the world use one less abused server and Asia would just get whatever they get and I wouldn't worry about it.

There are downsides to widespread blocking based on geolocation.

- First what happens when a decision maker goes on holiday outside of your service area. That could be a huge OUCH if he can't reach it in order to see what's being bought. These things can break deals.

- (most) search engines are global, they might eventually like you less if you block them too much

- those in it for the bad things (scrapers, spammers, etc.) know dozens of techniques to work around it.
e.g.: they use TOR, they use a shared webhosting platform, they use a VPN tunnel, they use a botnet, ...
While it buys you some peace, they will come back with a vengeance.

Swa66,

Nothing is foolproof forever. But many of these things aren't particularly "determined" bots. They are just hitting the server because they can. So blocking cuts down on the vast, vast majority of them. And that's really the important point.

- (most) search engines are global, they might eventually like you less if you block them too much

- those in it for the bad things (scrapers, spammers, etc.) know dozens of techniques to work around it.
e.g.: they use TOR, they use a shared webhosting platform, they use a VPN tunnel, they use a botnet,

There are products on the market that block TORs, and they won't tell you how they do it either. I could tell you how all that could be blocked, it's not that hard, just a little more challenging, but then you would know to much and be a threat to national security.

If you think I'm kidding... <cue the black helicopters>

However, back to AdSense, I think the sudden jump in earnings is smart pricing suddenly being confused and it has to dump out all the higher paying ads first. Since all the low rent traffic has gone away it probably gets initially lost and thinks your traffic is worth top dollar briefly before it recalculates the value of the remaining traffic which might take a few days/weeks.

I my experience you'll come out with higher earnings somewhere in the middle after smart pricing figures it all out.

Keep us posted on how it prices in a few weeks.

In a past life I managed a service dept that delivered services around the globe (new york, london, singapore, ...). I had staff working for/with me in those locations and in order to deliver the service we bought high end ICT gear.
I had the choice where I bought it, either we could buy it centrally and ship it, get it through customs etc, or we could buy it locally and be done with it. Now for certain equipment we had deals in place with the manufacturer and for other stuff, we usually used a local supplier for speed and flexibility.
I've had a number of suppliers over the years removed from the supplier list due to them blocking me based on geolocation. Sure I could easily tunnel though our local internet connection, but it's a pain to have to do so.
If I can't see their stock levels, their pricing, etc and compare it to my other options - well then they will not make a sale.

I've ran into a supplier near Washington DC at a point that had a huge warehouse full to the brink of equipment but they for some silly reason assumed that not telling me their stock depending on where I was would be beneficial to their business. I did check it: if you had an apparent US geolocation they did show their stock levels, with an EU based one they did not show it. In the end they lost a lot of business just by refusing to show me their stock levels. They delivered fast, had a huge stock, where competitive on price, had good terms etc. But the extra trouble to be able to see their stock levels meant I usually went elsewhere. For one reason: it was simpler if I wasn't on a desk in the US.. .

Preaching to the choir.

Still doesn't change the reality of the situation for me, fumanchu, and many others.

You decide which way you have to go, draw a line in the sand and that's how it goes.

FWIW, most sites wouldn't tank under a heavy load unless it wasn't using a database in the first place, so for instance static sites wouldn't notice unless it was an all out DDoS. Plus many sites, like little specialized niche suppliers like you mention would be less likely to be targets of such activities in the first place.

However, some suppliers are required to do such things because of their vendors not wanting pricing to be seen in other parts of the globe due to the differences in currency exchange, cost of doing business there, etc. That's why many websites, including Amazon, have things like "add to cart to see price" so it's not indexed and available to surfers around the country or globe without you going to each individual site and when you get there they may show you a geolocation specific price.

Lots of reasons for doing those things, or just blocking it to avoid the hassle in the first place.

BTW, Bill, I forgot to mention this. I'm not exactly blocking the foreign traffic. I'm simply redirecting it to...google.com! Heh. :D

So perhaps google will appreciate the extra traffic. I'm sure they can handle it. ;)

You get that option when you decide to use country blocking. Block it outright, or redirect it. I hope all the bots enjoy googling for whatever. :)

Interesting... I would seriously consider blocking Russia & China for sure due to past spam/abuse/bots. Some of my site content is of interest outside North America, although NA is the main market.

Is there an easy way for a non-server guy to block two countries? Is it as simple as using the .htaccess file? Right now I run a forum and block individual IPs after they spam. And I've set a crawl delay in the robots.txt for the bots that never send me traffic, to at least slow them down a lot. I'm going to rethink that as I will probably just block all the known useless bots.

Is there an easy way for a non-server guy to block two countries? Is it as simple as using the .htaccess file?

You can block as many as you want, no limits.

And I've set a crawl delay in the robots.txt for the bots that never send me traffic, to at least slow them down a lot.

Waste of time as only a couple of legit bots even implement crawl delay.

Easy way for non-server guy is to ask your server folks to install mod_geoip lite if it's an Apache based web server. You can block by country. Downside to that is the "lite" version runs about 1-2 months out of date, so you may occasionally block legit users.

I'm pretty sure that blocking countries for the purpose of increasing your AdSense earnings is a big nono. It doesn't matter how you try to validate it, bots, poor nations, they'll probably can you. It's not a good idea to post shady things like that publicly.

I'm pretty sure that blocking countries for the purpose of increasing your AdSense earnings is a big nono. It doesn't matter how you try to validate it, bots, poor nations, they'll probably can you. It's not a good idea to post shady things like that publicly.

You jest ..you seriously think that G want you to be showing ads for medical supplies and scooters for senior citizens in Los Angeles on your site about Seniors in LA ..to people from Karachi..? ..You think that G and the advertisers would be happy about them paying for the clicks..?

"Risk to advertisers" ( costing them money for no perceivable benefit ) is usually in the "banned messages" that folks get from G..

I block countries based on thousands of web page probes for exploits, forum comment spam attempts, dictionary attacks, malicious crawling, etc. on a daily basis. Google is not one of the reasons.

Blocking certain IP ranges has not impacted real user traffic to my sites in the least, so Google isn't impacted. For instance, my sites had half a dozen real users from China per year (if that many) which might see Adsense, and prior to blocking China thousands of non-user page requests.

There is a legitimate need for blocking which has nothing to do with Google, and it's an admin's decision what works best for their business model and user base. :)

Leosghost, that advertiser should not be targeting Karachi, should they? Google has spent a lot of time developing targeted ads and giving advertisers accurate geo targeting tools. If they don't know how to use them then that's their fault. If Google wanted you to filter countries in AdSense they would give you the tools to do so. I'm sure there are people that have valid reasons for blocking whole countries, but that does little against sophisticated attacks. Just don't go around telling people you're making more money because you blocked a bunch of countries, Google is very interested in that type of behavior.

I think I'll tell people what I wish to ( incidentally I did not say anything about making more money by blocking countries ) ..( you imagined that I did ?..)( it says so nowhere in my post :) ..without any "input" from you..concerning what you think Google are interested in..:) or what you think advertisers should be doing..

You don't appear to understand "who is driving the bus"..and who's "input" Google care about..

Doubtless you think that those of us who run more than one site and in different languages ( such as French or German ) are going to not block countries such as Indonesia or China from them..appropriate nick..the thing is..it is actually more complex than you think..as motorhaven illustrated..and as many hundreds of threads which you must have missed demonstrate and attest to..

But one thing that you can be sure of.."the banned" emails that people say they get..invariably mention" "risk to advertisers"..and many have had those emails after being hit by swarms of clicks from countries that advertisers did not target..

One can even hire human click swarms in such countries to put competing adsense sites out of business..same way as one can pay for link blasts to a competing site..

There's no Google AdSense policy against banning countries. And traffic quality matters very much to Google.

OK, instead of blocking, I'm going to detect & serve alternative affiliate ads. Here is the code I'm going to use.

//visitor pings the service - free and uses the same database as mod_geoip
$.getJSON('http://freegeoip.net/json/', function(geoip) {
//if the country code is ...
if (geoip.country_code&&(geoip.country_code=='CN'||geoip.country_code=='IN')) {
//prepare cookie date
var expdate=new Date();
expdate.setTime((expdate.getTime()+86400000));
//set a cookie, to detect later and replace ads server-side
document.cookie='blockadscookiename=1; expires='+expdate+'; path=/; domain=www.example.com';
}
});

pros - no installation, client side, no effect on page speed (you can call it after page load event)
cons - visitor sees the first ad, you can only detect cookie & replace ads for consequent requests

Leosghost, you're not making any sense. How can someone from a country which has not been targeted click on the ads? They should not be appearing for that country. No matter what you think, it's not your business, it's Google's system and they can interpret their policies any way they want and close accounts without notice or explanation for lesser offences. Or have you missed the all the threads on that? If you think you have any control over the bus, you're dreaming.

How can someone from a country which has not been targeted click on the ads? They should not be appearing for that country.

Where'd you hear that?

You are the one who does not understand "who is controlling the bus"..

I don't think for a single moment that publishers are..

Nor do I think you have the slightest idea or experience about what Google wants from publishers or advertisers..

Btw.. very very many advertisers do not target by country ( as even you remarked above ) on the "content network", ( they should, but they do not ) ..Google is not going to make them "suffer" for it if a publisher shows their pages to a country and someone runs a click swarm at it..

But, as I suspect that your miss-understanding of both Google and myself, is based on English not being your first language..

That is the only explanation that I can find for you now accusing me of believing all the totally inaccurate things which you have been posting so far in this thread..things that I have been telling you are incorrect since I joined the thread..

Whatever your reasons..I'm done with you trying to put your words in my mouth and then trying to tell me that I'm wrong for saying them..

My point and experience is exactly the same as netmeg's

There's no Google AdSense policy against banning countries. And traffic quality matters very much to Google.

and many thousands of other adsense publishers agree with us..you were the one arguing the opposite..right up until your post #4578247 where you imagined that it was me that has been espousing the views which actually came from you in your posts..

I don't have time to waste debating with someone who when they realise they were wrong..imagines that it was actually other posters typing the words ..

Carry on..

I don't have time for your nonsense either. I have never dealt with anyone that does not use geo-targeting, it's their fault for allowing everyone in the universe to click their ads. Publishers have very little say when it comes to adsense, advertisers on the other hand...

netmeg, are you suggesting people are circumventing google's geotargeting and clicking ads within regions that are not targeted and therefore should never have been shown the ads? From my testing google does a very good job keeping ads targeted, even with cities close together.

Either way, it's up to you guys if you want to block countries for the purpose of increasing your earnings. That's what this thread is about. I'm just saying that it may not be the best idea to tell everyone (including Google) about it.

netmeg, are you suggesting people are circumventing google's geotargeting and clicking ads within regions that are not targeted and therefore should never have been shown the ads? From my testing google does a very good job keeping ads targeted, even with cities close together.

Happens all the time. Geo-targeting is never 100% on the display network. Lots of people use proxies. And many advertisers (specially the less-than-expert ones) don't bother to geo target at all, to their vast detriment.

(I have a lot of experience with AdWords, moreso even than with AdSense. There's some as would call me a bit of an expert)

It's perfectly fine to block countries from seeing your site or your ads. Whether it's wise depends on your situation, but Google isn't going to slap you down because of it.

[edited by: netmeg at 6:48 pm (utc) on May 27, 2013]

Btw .for other readers of the thread..most CPC publisher networks have some basic things in common..rules and advice for advertisers ..and rules and advice for publishers..

All the major search engines have different "tiers" of "publisher"..Yahoo and Bing refer to the higher level ( those with the most traffic ) as "partners" in their "partner network"...Google refers to them as "premium publishers"..

Which ever search engine's ad network they are in..these" partner" publishers have more "leeway" in the way they design their sites and the way they "blend" their ads into their sites etc..

Yahoo's partner publishers, have amongst the rules that they must enforce ( and the rules have remained the same since the Yahoo Bing deal )..that a partner publisher must ensure via whatever method they wish ( redirects via scripts or htaccess or whatever ) ..that no more than 5% of the traffic that clicks on the ads shown on their sites , is generated from IP addresses outside their designated Geo- graphical area (s)..any publisher whose site consistently sends more than 5% of non Geo-targeted traffic will have their quality score reduced..( similar effect to being "smart priced" ) and may have their contract with Yahoo Bing search rescinded..

The list of countries from which "traffic" is considered "acceptable" for Yahoo Bing search partners is not a large list..traffic quality is extremely important to them..and they do not expect the advertisers to be responsible for the filtering or targeting of the traffic that clicks on their ads from publishers within the system..it is the publishers responsibility to control their traffic quality..and the Geo-targeting..especially as the publisher is required to be logging at least 2 million distinct separate visitor IPs per month to be able to participate..

Google requires even higher separate distinct visitor IPs figures monthly to be in their "premium partner" system..

My contacts with Google premium partners ( whilst all parties are extremely discrete, and do not go into details, as the TOS of these contracts prohibit anyway )..tell me that Google is equally vigilant as regards the "quality of the traffic" that clicks on the ads displayed on premium publishers sites.. and that if the site is in French, that Google would be highly suspicious of large numbers of clicks from India or the Philippines*, and that the site owners are expected to make sure ( by similar methods to those advised by Yahoo ) that advertisers are not exposed to such risks..

All search networks Geo-target..none of their Geo-targeting is perfect.and sometimes it can go radically wrong..if that causes problems for the advertisers when their ads are displayed on publishers sites..it will not be the search network(s) , nor the advertisers, who will be thrown under the bus..

It is perfectly reasonable to expect that smaller publishers in Google's are also expected to not put the advertisers "at risk"..one of the fastest ways to get banned..is to have one's site which is aimed to appeal to a specific group of visitors, and to attract traffic from a particular area ( say a tourism site dealing with Florida, wishing to attract English speaking tourists to the Southern USA ) ..that suddenly gets thousands of clicks on the adsense from India or Pakistan or China,or Indonesia..

Geo-Ip targeting of sites is very easy, and very widespread, despite what some might say.. You can even "play safe" and redirect them to Google.com :)

or..you could allow the entire world to access and click on the adsense on your tourism in Florida site..

Question is ..do you feel lucky..

*Similar scenarii would apply to other languages, and their expected traffic and ads..

If they are using vpns/proxies, does that not defeat the whole purpose of blocking the suspecting countries? It's easy enough to detect fraudulent clicks and click swarms with patterns, Google just needs to step up the defense.

mod_geoip detects many anonymous proxies (and doesn't require inefficient calls to external sites to lookup IPs unlike the code example Levo posted)

There are also easy methods to block Tor exit nodes, examples on the web are plentiful.