Facebook Advertising Feature Update: Split Testing

facebook advertising split testing

As digital marketers, we all know the importance of testing – whether that be keywords or ad copy in SEM, banners and placements on social or different DSPs for programmatic display. Testing is crucial to understanding what works and doesn’t work in digital advertising.

I would argue that of all the major ad buying platforms available to us today (ie. AdWords, Bing Ads, Facebook, Twitter, GDN, etc.) the greatest benefits and most powerful insights, from continuous testing are going to come from social channels. If we exclude remarketing and focus on prospecting, there simply are no other channels (with the possible exception being some of the more advance programmatic display platforms) that offer the level of targeting that, for example, Facebook does.

Not all is grand in the world of Facebook though. As any long-time Facebook advertiser knows, running and managing ads on Facebook can be a very tedious process. Thankfully, Facebook is aware of this and  makes changes to their ad platform on an almost weekly basis (no joke). A recent update to their platform, second in importance only to (proper implementation of) the Facebook pixel: Split testing.

Split testing, seriously??? Hard to believe this is actually a new feature but it is (unless you’ve been using a 3rd party ad tool) so if you aren’t using it, start, now.

What is Facebook Split Testing?

Facebook’s help page explains it quite clearly: “Split Testing helps advertisers understand how different aspects of their ads affect campaign performance. Similar to A/B testing, Split Testing lets you test different versions of your ads so you can see what works best and improve future campaigns.

facebook advertising split testing

Warning: the above explanation isn’t exactly true. The biggest limitation (and it’s big) with Facebook Split Testing is that you can’t set up a test with the sole purpose of testing different creative variations. What you say?!?! Yes, it’s true. Don’t worry, I’ve informed Facebook of the oversight 🙂

Facebook’s explanation about split testing continues:

“For example, Split Testing allows advertisers to easily test the same ad on two different audiences to see which audience performed better. Or, to test two delivery optimizations, such as conversions or link clicks, to determine which yield the best results. Advertisers use Split Testing to interpret how changes in different variables such as these might impact ad success.”

As you can see, there’s nothing here about testing different ads against the same audience or delivery options.

What do you need to know:

  • Campaigns have to be set up in the main Ads interface or Power Editor

facebook advertising split testing set up

  • You can only test three different variables at the moment:

facebook advertising split testing features

  • Budgets & Length of campaign:
    • Based on the size of your audiences or splits, Facebook will provide a minimum required budget that it feels will provide enough data to be statistically significant (This will typically require running ads for a minimum of 3 days with a budget of $800+.)
  • Once a split test campaign is active, you can’t edit any of the variables, otherwise it ends the test

For additional information check out Facebook’s help page on the topic but for now, let’s look at some real results.

We are constantly running creative, audience & placement tests for our social advertising clients, but with this feature I’m far more confident in the results (and it’s WAY easier to manage). For one client in particular though, the data we’re gathering from split testing is going to have an immediate impact on our strategy moving forward.

The client is a major insurance provider in the US and our goal is to determine what delivery optimization (aka bidding tactic) will result in the lowest cost per lead.  Note on measurement: while leads are being tracked via the Facebook pixel, we also look at both last-click and a custom attribution model in Google Analytics to gauge performance.

Testing parameters: Compare CPC, CPA (1-day) & CPA (7-day) bidding tactics

Who wins: it depends of course!

The first two screenshots show results from two campaigns with the same audiences but targeting different states. The third screenshot is an entirely different prospecting audience targeting a single state.

facebook advertising split testing results

What have we learned? For this particular client, we’ve seen over the last year that while CPA bidding tends to result in lower avg CPL, that wasn’t always the case. CPC bidding almost always resulted in significantly lower avg CPCs. which in turn, despite lower conversion rates, returned CPLs that were at times competitive with CPA bidding. What we’re seeing (or confirming) with these split tests (the 3rd test hasn’t spent enough yet to be statistically significant) is that it’s not all about bidding on Facebook to drive website conversions … we can’t assume that CPA bidding is going to automatically result in the lowest CPL.

What sort of results have you been seeing from split testing on Facebook?

Ian Mackie About the author
  • Hey Ian,

    Awesome tips! You hit the nail on the head when you said, “As digital marketers, we all know the importance of testing – whether that be keywords or ad copy in SEM, banners and placements on social or different DSPs for programmatic display.”

    Will definitely share this on my Facebook page.

    Keep the wisdom bombs flowing. You are A-W-E-S-O-M-E!


    Jimmy R.

    February 20, 2017 at 3:50 pm

Leave a Comment: