Thinking · Newsletter
How do I set up a creative testing framework for Meta and TikTok ads?
Test one creative idea at a time, hold everything else constant, size budget to conversions, run at least 7 days and judge on cost per purchase or install.
TL;DR. Set up creative testing so that only the creative changes: one test per creative idea, with the same audience, placements, optimization event and landing page as your scaling campaigns, a budget sized to conversions rather than to a round number, and one success metric picked before launch. Run each test for at least 7 days (TikTok's help center minimum; its business blog says 14) and until enough conversions have arrived, judge it on cost per purchase or install at the event you scale on, move the winners into scaling, and write what the test taught you into the next brief. On Meta, use the creative testing option in Ads Manager or a dedicated testing campaign; on TikTok, use Split Test with the Creative or Creative Assets variable.
Why test ideas, not tweaks
Introducing Andromeda, its ad retrieval system, Meta's engineering team wrote that with generative AI tools "the number of ads creatives in Meta's recommendation systems is expected to grow significantly." Jon Loomer's advice to advertisers on creative diversification is blunt: "Unless they're distinctly different enough, any variation is unlikely to make a meaningful impact on performance." So test motivations, hook families and formats, not button colours.
How to set up a creative testing framework, step by step
- Write the question before the brief. Each test should answer one thing, such as "does a problem-first hook beat a demonstration-first hook for this audience?", so that a null result still teaches you something. It is the first item on our pre-flight checklist for a batch: what the batch exists to answer, in one sentence.
- Pick the tool. On Meta, the creative testing option inside an existing ad lets the test ads run where your campaign already runs; Loomer reports that "Meta should distribute similar ad dollars to each ad in the test" and that "each person will only see one of the ads." On TikTok, Split Test lets you "test two different versions of your ads" and "split your audience into two equal groups, with each group seeing only one ad group."
- Hold everything else constant. Both platforms say one variable per test: Meta's developer documentation says "Select only one variable per test," and TikTok's says "You can only select one variable for each split test." Keep audience, placements, bid strategy, optimization event and destination identical to your scaling campaigns, so a winner in test is a winner in the place you will spend.
- Size the budget to conversions. TikTok's business blog says to "set a budget of at least 20x CPA based on your campaign history," and its help center recommends a budget that gives an Estimated Testing Power "of at least 80%." RocketShip HQ's working rule is that "daily test budget equals expected cost per conversion times the conversions a day you want," aiming for "ideally 10 conversions a day per test, with 2 or 3 as the bare minimum when cost per conversion is very high." On Meta's in-campaign tool, Loomer reports that "Meta recommends dedicating no more than 20% of your budget to this test."
- Run it long enough, and do not touch it. TikTok's help center says "We recommend testing for a minimum of 7 days to obtain the most reliable results," and that you "don't change your Split Test settings after it has started running." Its business blog says "Run the test for at least 14 days in order to gather reliable results." For Meta's tool, Loomer writes the length "seems to be 7 days by default." RocketShip's rule, which we share: "Judge after enough results have arrived, not after a fixed number of days."
- Decide on one metric, the one you scale on. TikTok uses the Key Metric you set "to compare the two ad groups and determine the highest performing ad"; Meta's tool asks you for "the metric that will be used to determine success." Pick cost per purchase (or per install, for apps) at the event your scaling campaigns optimize for. RocketShip's definition of a winner is the same: "an ad whose cost per conversion, on the event your scaling campaigns optimize for, holds up at test spend." Hook rate and hold rate are diagnostics; we explain why in measure creative on purchases.
- Graduate winners carefully. Move a winning ad into scaling as a new ad next to the current winners. On TikTok, the business blog advises "don't increase budget by more than 50% for each adjustment" and to "Wait for 2 days, and then repeat this process until you reach your target budget."
- Write the result into the next brief. Read results at the level of the idea, not the single ad. RocketShip puts it simply: "If several ads in a direction win, the idea works; make more of it." Then "Keep a log of every test, what it tried and what it taught you." That log is where the next brief starts.
Meta and TikTok side by side
| Meta (creative testing in Ads Manager) | TikTok (Split Test) | |
|---|---|---|
| What it splits | Test ads within an existing ad, similar spend each, no audience overlap (per Loomer) | Two ad groups, audience split into two equal groups |
| Creative variables | The test ads you edit | "Creative" or "Creative Assets", which includes the opening 2 to 3 second hooks |
| Duration | 7 days appears to be the default (per Loomer) | Minimum 7 days (help center), at least 14 days (business blog), maximum 60 days |
| Budget | No more than 20% of the budget (Meta's recommendation, per Loomer) | At least 20x CPA (blog); power of at least 80% (help center) |
| How a winner is called | The success metric you choose; results in the Experiments section | Key Metric; a winner only if results are statistically significant, at a 90% confidence rate |
The two sources TikTok publishes disagree on duration. Our read: treat 7 days as the floor and conversion volume as the real gate.
The test most teams cannot read
The variable is the creative idea. In our view, the commonest unreadable test is a new creative pointed at a new landing page in the same test: you learn nothing about either. And an A/B winner is only the better of two ads; whether it adds demand is a separate question, covered in incrementality testing for creative.
Where testing sits in the loop
In creative operations, defined we call the stages read, brief, make, launch, and read and log. Testing turns the last stage into the first: every result goes into the log, and the next brief starts from it. That is how Flixr runs accounts, a loop every week of brief, make, launch, read, log, judged on the client's own numbers.
Questions people ask
Should I test creative in a separate campaign or inside my scaling campaign? Either works if only the creative changes. Loomer prefers Meta's in-campaign tool because "you can keep your ads in the existing ad set"; RocketShip starts from "a dedicated testing campaign, with a separate ad set for each creative direction." We would choose by budget: if a separate campaign cannot buy enough conversions, test inside.
How many creatives should I test at once? TikTok's Split Test compares two versions at a time. RocketShip suggests "typically at least about 8 per ad set" across angles, hooks and formats in a dedicated testing campaign. Our rule of thumb is fewer ideas, each with enough budget to reach a verdict.
How long should a creative test run on TikTok? At least 7 days per TikTok's help center, which also says split tests can run for a maximum of 60 days. TikTok's business blog recommends at least 14 days, so leave longer when conversions are slow.
What metric should decide a creative test? Cost per purchase, install or other conversion at the event your scaling campaigns optimize for. Click-through, hook rate and hold rate explain why an ad worked; they should not decide whether it did.
Does a test winner mean the creative is incremental? No. A split test tells you which of two ads did better under the same conditions. To know whether it brought sales that would not have happened anyway, run a holdout.
Sources: TikTok, "Split Test Best Practices"; TikTok, "About Split Testing"; TikTok, "About Split Testing Variables"; TikTok, "How to create a split test"; TikTok For Business, "How to test, learn, and scale your performance campaigns", 8 May 2024; Meta for Developers, split testing guide; Meta Engineering, "Meta Andromeda", 2 Dec 2024; Jon Loomer, "Meta's Creative Testing Tool: Setup, Strategy, and Results"; Jon Loomer, "Make creative testing part of your process", 6 Jan 2026; Jon Loomer, on Andromeda and creative diversification; RocketShip HQ, "How to structure Meta campaigns for creative testing", 13 Apr 2026.