SkrymDocs
A/B Testing

A/B Testing

Test the performance of alternative strategies

Last updated 14 Oct 2024

A/B testing allows you to compare different Strategies in real-time by directing a portion of user traffic to each variant.

For example, you could assess how your conversion rate differs after adding a new Tactic/Condition before using the new Strategy in production.

How It Works

A/B testing is straightforward:

  1. Create multiple strategies that you want to test against each other.
  2. Assign a percentage of traffic to each strategy.
  3. Schedule the test to start automatically distributing traffic based on your settings.

Feel free to go through the step-by-step guides below to learn how to work with A/B Testing on Scope:

Follow these steps to create an A/B Test:

  1. Navigate to the Strategies page.
  2. Click on Add New under the A/B-tests section.
  3. Configure your test settings.
  4. Save the test configuration.

Note: The test remains in a DRAFT state and will not run until scheduled.

Important

The traffic allocation is randomized based on user sessions, not each individual checkout page load. If a user reloads the checkout within the same session, they will see the same options.

A/B Testing Statistics

You can view statistics from your A/B tests by navigating to the individual A/B test page from the Strategies page on Scope. For each A/B test, you can see the following statistics:

  • Total number of presentations
  • Total number of conversions and conversion rate
  • Number of confirmed orders for each transport method
  • Average displayed shipping price for confirmed orders
  • Average shipping cost you incurred for confirmed orders
  • Average profit and loss you incurred on shipping

Checkout Statistics

If you have any questions or concerns, please feel free to reach out to info@skrym.se!