Testing and Measuring Audience Performance

Recommended First Tests

There are three specific Audience and ad set tests to start your Disco Audiences journey with:

• A Disco Small Direct
• A Disco 5k Lookalike Seed
• A combination of Disco Small Direct and 5k Lookalike

These specific starting points help jumpstart your testing and optimization by getting reasonable representation of each type and combination that Disco can provide you, allowing you to more quickly land on the most valuable Disco Audiences for your brand.

Good Testing Baselines

For a clean test, run a “Business as Usual” campaign leveraging your usual set of creatives and the Disco Audience, one ad set at a time.

Launch an ad campaign on the same day your Disco Audience is readily available in your Ad Account. To ensure a good comparison between Disco Audience performance an non-Disco audience performance, verify the following criteria are true:

• The competitive campaign’s audience only targets shoppers in the United States.
• The competitive campaign’s audience excludes your existing customers.
• The creative assets for each campaign are the same.
• The ad budget for each campaign is proportional to the size of the campaign’s audience.

With paid advertising, it’s an industry standard to let a campaign run for 1-2 weeks. While results may be seen within 3 days for a given Disco Audience, it is still recommended to test for 1-2 weeks.

When testing out audiences it’s good to start both small and focused, similar to the settings recommended for your first test, then add in additional sizes and categories. Throughout the process you’ll want to have good control groups to compare to as well, which would also be similar to the parameters outlined for your first test.

Once you have a baseline you’ll want to take some measurements for both your control and your new audience and begin the process of iteration to dial in the best audience size and category selections for your needs.

Measuring Performance

Picking Key Performance Indicators

As with any good testing strategy, keep tabs on comparative marketing key performance indicators (KPIs), such as:

• Cost per acquisition (CPA)
• Cost per mille (CPM)
• Return on ad spend (RoAS)
• Cost per click (CPC)
• Click-through rate (CTR)
• Conversion rate (CVR)

Add any other key metrics important to your marketing team to the list of things to track, the above metrics are simply common options to get you started. You'll know best which of those KPIs are the most meaningful to your business, and from those you want to hone in on for your testing.

Making Decisions Based on Results

Measuring Audience performance is considering its efficiency compared to to your control groups. Based on the KPIs above, your Disco Audience can fall into one of three efficiency ranges compared to a control group

1. Less efficient: A Disco Audience can be considered less efficient if it is performing worse than your control audience by 15% in your selected KPIs
2. Similarly efficient: A Disco Audience is in the similar efficiency category if it is within 15% of your control audience's efficiency for the same KPIs
3. More efficient: A Disco Audience is considered more efficient if it is beating your control audience by 15% or more in the selected KPIs

If the Disco Audience is less efficient...

Consider abandoning the audience to focus on variations of your more performant audiences, which could be the previous step in your testing journey (say, for instance, a 10k Lookalike Audience with no category after testing a category), or simply starting with a fresh audience. 

If the Disco Audience is performing similarly...

We recommend two tests. First, testing that size/type audience against a category, for example if a 5k lookalike seed is similar to your control audience, install a 5k lookalike category audience to test with. This step can (and should for many) be repeated with different categories to find which are more or less performant for you. When you find a category specification that works, scale up the audience size with that same configuration.

If the Disco Audience is overperforming...

Continue the current audience activation to prove out efficiency over time, refreshing the audience as needed. Take what you've learned from this test and begin a new set of tests from the top.

Parameters for Additional Tests

Based on performance, you may want to scale up size or pivot to a category version of the performant audiences. As long as you’re keeping good testing methods in mind you will be able to find increasingly efficient and useful audience configurations for your needs.