It would be difficult to test them for a few hours but then it seems like an effective test would take at least a week, or possibly a month. The thing is that if you test an ad type and it works well for a day, that could simply be due to a single advertisers campaign which uses that ad type.

yes, but even the monthly stats seem to be off, so what then?

Ah yes, I see what you mean now. AdSense is going through some mood swings so it's difficult to plot them against past performance, But it seems like the ad types could be compared against each other during the same time frame.

My sites are in a constant state of testing. I test ad spots, I test colors in the link units, I test ad block sizes. I test against time periods - a month ago, three months ago, and a year ago. (My sites are mostly seasonal so that makes sense for me) In fact, I recently did one of the comparison charts in Google Analytics for traffic and sources, and it's amazing how close the ups and downs of traffic correspond from one year to the next (I've got a lot more traffic this year, but it rises and falls exactly the same)

When I pull my earnings stats into Excel and compare week to week or month to month, the decline seems pretty alarming, but when I compare by the year, I'm almost caught up as of this month, and will probably do better next month.

It's *very* difficult but I have finally disciplined myself not to go off half cocked and change things until my tests have run for at LEAST a week, and preferably two. I generally see enough normalization over that period to be able to draw some conclusions and make changes accordingly. You just can't get any idea of what's going on in a day or even three days. If you make a change based on that data, it's like driving with only one eye open. You *might* be okay - but you might get hit in your blind spot.

And I try to revisit some tests when I have the time; sometimes weird things happen. For example, last year I rotated link units in red, blue & green, and the green were far and away the worst performers - I ended up removing them for my busy season. Tried them again this year, and for some reason, they're beating out red and blue two to one. I have no explanation for that.

You can't just do this day to day. You have to be able to look long, or you're going to go stark raving mad.

I've been finding it impossible to test positions for the last 12 months.

Perhaps I've been unlucky or have been subject to co-incidence, but I got fed up and moved a massive great ugly ad right at the top of long pages across my best site to the same massive ugly thing at the bottom of the same. No difference in any of the stats so far.

Maybe I'm smart priced (pages take 2 days each to create so we're not looking at MFA, but they are not aimed at being useful to advertisers so it could well be that I'm smart priced). Previously I had been able to test positions on the same pages very quickly.

Smart reporting is my guess - the stats seem to change more slowly these days. I'd give changes 3 months to flow to their new level these days.