Tough Decisions When Ad Copy Testing

The more people that come to your site, the more people will buy your products, or fill out a lead form, or sign up for a newsletter. That’s pretty intuitive, if each visitor has the same chance of converting. So getting new visitors is a pretty good goal, right?

Maybe, I guess.

The problem is that not everybody has the same chance to convert, and for PPC, new traffic is only desirable if the cost is justified. Too often, marketing folks are looking to increase the click through rates of their search ads- which is a good goal, as long as conversion rates don’t go down. If I’m paying for more traffic, but getting the same number of leads, my campaign is not cost-effective.

The success of an ad cannot be measured without looking at the conversion rate. I’ve been asked how the ad copy can possibly influence conversion rate once the visitor gets to the site, because isn’t that the job of the landing page? Doesn’t the ad get the click, and the landing page get the conversion?

No!

The ad and the landing page work together to shape the visitor experience. The ad sets up expectations, the landing page has to deliver on them. Only by looking at both conversion rate and click through rate can the success of an ad be determined. And when testing, knowing what you are trying to accomplish before the test is started will make finding a winner much easier.

Know what you are testing

I looked at the results of an ad copy test the other day. One ad had a higher CTR, the other had a higher conversion rate. The difference between both metrics was statistically significant at a 99% confidence level. Which ad do I choose to keep?

First of all, this does happen. And if a marketer doubts the relevance of the ad to the conversion rate (isn’t that what the landing page is for?), the ad can have more impact than they may think, because that’s what sets up the expectation for the page. If a visitor comes into the site expecting a free coffee maker, and there is no free coffeemaker, the quality of the landing page isn’t very important. Setting up the proper expectation is one of the most important roles of the ad.

So what to do? Things to consider:

1. What was the goal of the test?

2. How do the CPA’s compare?

3. How many impressions are required for each ad to get a conversion?

Higher CTR means more traffic, but more cost. Higher conversion rate means a lower CPA. Looking at my two ads, I had to choose between low CPA, and high conversion volume.

So the two metrics used to decide the better performer are the total conversion rate (conv/imp) and the CPA. If the difference in conversion rate is enough to make up for the difference in CTR, and the high conversion rate ad gets more conversions per impression, that’s the winner. If not, you still don’t have a winner.

In other words,

If (cr1 x CTR1 > cr2 x CTR2), and (cr1 > cr2), the value of the CTR doesn’t matter, because ad 1 is getting more conversions at a lower cost. The ultimate goal.

If the high conversion rate ad is getting less conversions per impression than the high CTR ad, then the CPA is important, as well. Is the marginal cost of the additional conversions greater than the average revenue per conversion? Here’s an example (assuming identical CPC, and these are not the stats from my ad test.):

impressions

Clicks

cost

CTR

conv

Conv rate

CPA

TCR

Ad1

10000

400

$ 300.00

4.00%

100

25.00%

$ 3.00

1.00%

Ad2

15000

250

$ 187.50

1.67%

130

52.00%

$ 1.44

0.87%

If I’m expecting an average revenue of $4 per lead, it looks like Ad1 is better, because there are more conversions, and under the revenue cut off of $4.

Looking closely, though, if each ad gets 10,000 impressions, Ad1 gets 100 conversions, and Ad2 gets 87 conversions. The marginal cost of those additional conversions from Ad1 is $13.46… way over the revenue expectation of $4 each.

So know why you’re testing. If it’s to maximize conversions subject to a total CPA constraint, then Ad1 is the winner. If it’s to maximize profit, then Ad2 is the winner. Also keep in mind that each measurement has its own standard error, so a margin of error for the total conversion rate has to start at the low end of both CTR and conversion rate, and go up to the high end of both.

Because this client is consistently hitting daily budgets, I went with Ad2 to maximize profit, after making sure that 90% confidence intervals for the total conversion rate of both ads did not overlap.

Point It About the author
Comments:
  • Lisa
    Reply

    Great post Shad. I agree, it’s not always about CPA and it’s not always about CTR when evaluating ad copy tests, which is often difficult to explain.

    Not to complicate things, but perhaps another factor here, as it is always, is quality score. If a better CTR, the keyword-ad variation pair should have a better quality score, and therefore a lower CPC or higher position, which would affect CTR and CPA.

    February 4, 2009 at 7:37 am
  • Hi, great post.

    One question though: I didn’t understand the marginal cost per lead calculation (13.46$).

    Plus, do you know where can I get statistics regarding the marginal cost per lead? Can AdWords help me with this?

    Thanks a lot,

    eran

    June 23, 2009 at 10:23 am
  • shadracks
    Reply

    For marginal cost per lead, I divided the difference in cost over the difference in leads. The difference in cost I found to be $174.75 (using constant CPC and CTR for ad2, and using 10000 impressions for ad2 instead of 15000), and the difference in leads I found to be 13 (100-87).

    AdWords won’t have this, so if it’s something you are looking at, it will take some time to calculate it. Especially if the impressions aren’t the same for both ads.

    July 14, 2009 at 1:10 pm

Leave a Comment: