Not *quite* the same thing as an ad rotator, but still, an improvement in my book. I'll be interested to see if Google's conclusions turn out to be the same as mine.

It's very quick to set up a test. I put 2 in place in a matter of moments after getting the email. Looking forward to seeing the results.

It's a good addition. Thank you for reporting it!

Well right off the bat, it doesn't look like there's a way to tell it which site (if you have multiple sites) and it doesn't take into account channels. So I'm not entirely sure how that's going to work.

I have channels set up by site and by placement on each page on each site, and I see those listed. Testing several channels now.

Wonder if they'll extend the experiments into DFP.

I'm about 26 hours into mine and so far no Confidence Score to report even though the results are definitely favoring one ad style. Hope it's not stuck. I'll give it till morning before ending it if there's still no Confidence Score result.

I dunno how much traffic you get, but I'd probably leave it at least a few days to a week.

is there a limit as to how many experiments you can run at once? it seems like you can only do 15.

it would be good if you could do split testing on other stuff too, like flicking the switch off for interest ads, and using 3rd party ads.

I dunno how much traffic you get, but I'd probably leave it at least a few days to a week.

Enough traffic (80,000 plus page views)during the 40 hours that the test was running to reinforce what I was pretty sure all my other testing over the years had already proved. On my desktop test there was a 18% difference in RPM and on mobile it was worse...70% lower RPM. With numbers like that I'm not waiting around for G's Confidence Score to appear...which they said would occur around the 24 hour mark.

I'm not waiting around for G's Confidence Score to appear

Same here. I could see almost immediately that the changes weren't working and canceled the testing.

I actually forgot to include the first day of stats in my previous response so it was over 100,000 views on Desktop and 23% worse RPM.

This tool is useless for me - and probably yourselves as well. I've already tested that text ads perform the best, but Google will penalize in terms of RPM earnings you if you select to run just text ads only. At this time, to be achieve the highest earnings 99/100 times you have to be opted into all ads types.

I've given up on all experiments. No matter what I've tried, nothing can be improved. I think AdSense is rigged and you cannot outsmart it in any shape or form.

Really, only just, we have been making all comparison for years, but I personally think that our suggestions have finally started getting through and addsense team began paying attention to the most valuable advise that they can get from their publishers.

but Google will penalize in terms of RPM earnings you if you select to run just text ads only. At this time, to be achieve the highest earnings 99/100 times you have to be opted into all ads types.

No sir, that's not 100% true for all units in all positions. For example I have a text only ad unit in a higher CTR location who's earnings are decimated if I turn on images because the CTR on images is very low in comparison. I do agree however for ads in low CTR areas of your site, if you're getting a very low CTR on a unit then having image/text enabled is probably best.

In general image ad units don't get as a high a CTR as text only units so there is a CTR tipping point between the two.

As for the A/B testing tool I find it useful so long as you only use a particular unit in one location of one site. If you re-used the same unit in multiple locations or on multiple sites then you won't get actionable data with the tool.

And it depends on the site. For my sites, image ads get a much higher CTR but a lower EPC.

i had a few experiments recommend text-only instead of text/image. so i changed them over but then my scorecard rating dropped — it says i should change all the ads back to text/image. you can't win!
...i think i'll go with the experiment.

OK, so after few days of experimenting on two ad units my originals show better results. BUT, google still bets on variations. I wonder if they push their beloved "allow text and image ads for better performance" suggestion through the confidence score, because in both cases poorly performing variations are "text and images", while my originals are image only, and google still gives 90-something confidence score to variations.

Why would I switch to something that brings less money? :-/

@matrix_jan

You should not, this is the whole point of running experiment to discover appropriate add type that brings in more revenue. I personally would not worry much about scores, those are just scores and nothing more than that.

those are just scores and nothing more than that.

Yep. They are really kind of useless.

How do you decide what "wins" -- e.g. what to choose? Is it based on

- CTR?
- RPM?
- Earnings?

Is it not the point of choosing the best color or design of the text ad is to find out which elicits the most clicks?

My dilemma is with one test I am currently doing.

CTR of the original is 37% higher than the variant; but the variant's RPM is 46% higher and earnings 47% higher than the original. This test has been running since Experiments was launched, and both ad units being tested have gotten solid sample sizes

So the variant has poor performing CTR, but better RPM and earnings. Should I select the style that gives the better RPM even though it can't generate a much clicks as the other version?

Experiments is already saying that the variant is winning. But I guess I'll wait until it gives me 95% confidence level.

@alika

I assume, you were referring to identical positioning of the adds, but 2 different add types, text only on one and image only on another? If my assumptions are correct than it is simple, there are far too many image adds and very little number of publishers who had hoped in to text and image adds or image adds only, therefore all image advertisers are paying more than triple than they will pay for regular text adds, this is why your RPM and therefore revenue is high.

Color is the only variation for this test -- not image vs text only; or different placement.

Same ad unit shown in the same section of the site in the same placement (my Adsense channels are broken down by topics and by ad units so I know what ad unit in what section of the site in what placement are working), but different colors

The thing being tested is the color scheme alone. One is for the current color scheme I have and the other is for a different color scheme

I'm running a few experiments myself, and not mush to report yet. What I do find frustrating is 2 of my sites get a lot of traffic, yes ads for only one site can have experiments. Has G decided the other doesn't need it?

yes ads for only one site can have experiments. Has G decided the other doesn't need it?

Is there an existing ad unit for that second site? For example

"SITE B Leaderboard"

I have sites with really little traffic, but since I have ad units for these sites, I am able to run Experiments on them.

If the sites don't have ad units, try creating one for those sites. Then choose it for your experiment

@alika, I have specific ad channels set up for these sites. They are showing now, but hadn't when the feature first appeared. Thanks!

i had a bit of a disaster trying these experiments. i let them run for about ten days and my earnings plummeted. the experiments which ran to conclusion just told me what i already knew... text ads are better than image ones.

i have stopped all of the experiments now and my earnings have returned to normal.

i don't mind these occasional hiccups though, because it gives you a chance to look at things afresh... moving a few ads around, changing some of the sizes... there's nothing like a big earnings drop to remind you that you can always do better!

If you never experiment with AdSense (using that feature or not), you're not using AdSense correctly. :)