A/B Testing with Ad Management

Compare the efficiency of two setups with our A/B testing capabilities

Start an A/B test

To get started on an A/B test on a stack, you will need that stack to be live

eg here: the "For user guide” stack

You can do A/B tests on any stack (default stack included). And you can have multiple A/B tests running at the same time on different stacks.

Edit your stack with the changes you want to A/B test, then validate i.

This should create a new, ”ready to deploy”, version of your stack.

Clicking on the "Deploy” arrow will bring this menu up.

Here you can modify the "Deploy this stack on X% of the traffic” variable to choose the percentage of the population that will use the new stack.

Once the partition chosen, click Deploy and you will be brought back to the site’s page.

Here you can see that we set up a 50-50 divide in between our two stacks versions (v1 and v2).

Note that this A/B test only applies to one stack for the website, the default stack is unaffected. However, if you wish to implement the test on all calls on your website, you can simply implement a similar A/B test on every stack on the site at the same time.

Interpret and utilize the results

After some time has passed and your stacks have run through many auctions, you can go to one of the dashboards and select the stacks used in your A/B test with the add stacks option in the filters on the right to see how each version is performing.

We recommend you compare with KPIs that are independent of the number of auctions such as RPM, fill-rate, or eCPM.

Indeed, since you may not have deployed your stacks 50-50, comparing your versions by revenue would require taking into account the ratio of deployment of each version.

With those KPIs you be able to see how each version did separately, allowing you to easily compare and deduce which is most effective.

If you want to see how how each stack version looks like on your site you can use: Force ad stack version (for preview before deploying at 100%)

Remove A/B test

To remove an A/B test simply find the stack on which the test is running, choose the version you want to keep, click deploy and set the percentage to 100%.