Truth tests: set one up and read the result
On this page
A test shows what one ad platform really adds. It pauses that platform's ads in some of your regions for a few weeks, then compares your store's orders there with the regions where the ads kept running. Open it from the sidebar: Truth > Tests.
Before you start
- Your store is connected, so Truth can count your orders by shipping region.
- The platform you want to test has a connected ad account. Without one the page shows Connect an ad account.
- You are an Owner or an Admin. Only an Owner or an Admin can set up a test.
- A platform runs one test at a time. Another can be set up when the first one ends.
Set up a test
Click Set up a test. The set-up has four steps, and the card Your test so far keeps your choices in view.
Step 1: Platform
- Choose the platform whose ads the test pauses in some regions. Each platform shows how many regions have orders. A platform that is not connected is listed under Not connected, with Connect an ad account.
- Check the Name. It is how the test is listed, and Tapper fills it in from the platform and the month.
- Click Continue.
Step 2: Regions and length
- Regions: tick the regions where the test can pause the platform's ads. Truth pauses some of them and keeps the rest running. Tapper ticks a suggestion from your own orders.
- Share of orders to pause: a bigger share gives an answer sooner, and pauses the platform for more of your sales while the test runs.
- Smallest change to find: how small an effect on your orders the test must be able to see. A smaller one takes a longer test.
- Read What this test takes: Length, Ends, Pauses, Finds and Paused spend, worked out from your own orders for your picks.
- Click See the plan.
When your picks cannot make a test, the page says so. A bigger share to pause, more regions or a bigger change to find can make a test possible.
Step 3: What you do
Truth shows the plan: how many days it runs and the change it can find.
- The regions to pause the platform's ads in, with the steps for that platform's own ads manager. Use the copy button to copy the list of regions.
- Keep running in: the other regions. Change nothing there.
- Who pauses the ads, where it is offered: choose Tapper does it and pick the ad account and campaigns, and Tapper takes the regions out of those campaigns and puts them back when the test ends. Choose I do it to pause the ads in those regions yourself.
- Confirm: tick the box once the ads are paused. The result means something only while those ads stay paused.
- Click Continue.
Step 4: Start
- Check the summary: Platform, Name, Runs, Paused and Result. The test runs from tomorrow, in days of your ad account's time zone.
- Click Start the test.
Starting the test changes nothing in your ad account, unless you chose Tapper does it in step 3. Truth reads your orders each morning and gives the result when the test's days are over.
A set-up you leave is kept as a draft. Open it again with Continue, or click Drop this draft to discard it.
The Tests list
Each test is one row. Click a row to open the test's own page. Search by name, or filter by Status and Platform.
- Test: the name and the platform.
- State: where the test stands.
- Paused share: the share of your orders in the paused regions.
- Days read: the days Truth has read so far, out of the test's length.
- Result: the result in a few words, or when it comes.
The three dots at the end of a row hold Continue and Drop for a draft, and End for a running test.
What each state means
- Draft: the set-up is not finished. The result cell reads Ready to start. Click Continue to finish it.
- Agreed: a test arranged with Tapper's team, with its start day. Tapper's team starts it, so there is nothing for you to do.
- Running: the test has started. The result cell reads Result due, with the day.
- Ended: the test's days are over, or the test was ended early.
A test's own page
The page is headed with the platform's name, for example What Meta really adds. Its facts row shows Platform, State, Paused share, Days read, Start and Planned end.
While it runs
- Still running shows the days left and the day the result comes.
- An early read of your orders so far. An early read moves; the result counts at the end.
- The regions to keep the ads paused in, with the same steps as in the set-up.
- Readings so far lists each read: Read on, Days read, Orders where ads ran, Orders where paused and Change.
When it ends
The Result card sets two figures side by side:
- What Truth measured: the result from your own orders, with the days it covers, the range it could be and how sure the test is.
- What the platform claimed for the same days: the platform's own count for your whole ad account, in every region.
The result reads one of three ways:
- The platform added a number of orders: its ads bring orders you would not have had without them. Judge its budget on the orders it adds, not on the ones it counts.
- The platform added no orders: your orders were higher in the regions where its ads were paused. A second test would confirm it.
- No clear effect from the platform: the test found no difference in orders between the regions with the ads and the regions without. A longer test, or more regions paused, can find a smaller effect.
What this means says the same in one plain sentence, with what the result lets you decide.
End a test early
Open the test and click End the test, then confirm. Its result stops there and the platform is free for another test. If Tapper paused the regions, Tapper puts them back in your ad account. If you paused them, turn the paused regions back on in your campaigns. A test ended before its last day has no result.
Related
Was this helpful?
Still need help?
Chat with our team and mention this article.