Skrym LogoA/B Testing

A/B Testing

Test the performance of alternative strategies

A/B testing allows you to compare different Strategies in real-time by directing a portion of user traffic to each variant.

For example, you could assess how your conversion rate differs after adding a new Tactic/Condition before using the new Strategy in production.

How It Works

A/B testing is straightforward:

  1. Create multiple strategies that you want to test against each other.
  2. Assign a percentage of traffic to each strategy.
  3. Schedule the test to start automatically distributing traffic based on your settings.

Feel free to go through the step-by-step guides below to learn how to work with A/B Testing on Scope:

Create

Follow these steps to create an A/B Test:

  1. Navigate to the Strategies page.
  2. Click on Add New under the A/B-tests section.
  3. Configure your test settings.
  4. Save the test configuration.

Note: The test remains in a DRAFT state and will not run until scheduled.

Important

The traffic allocation is randomized based on user sessions, not each individual checkout page load. If a user reloads the checkout within the same session, they will see the same options.

Schedule

To schedule an A/B Test:

  1. Go to the Strategies page.
  2. Select the test you want to schedule.
  3. Click on Schedule.

The system will automatically start the test at the specified start date.

Extend

To extend an A/B Test:

  1. Visit the Strategies page.
  2. Select the test you wish to extend.
  3. Update the End Date.
  4. Click Save.

The test will continue until the new end date.

Abort

To abort an active A/B Test:

  1. Navigate to the Strategies page.
  2. Click the Abort button on the desired test.

This will immediately stop the test, reverting traffic distribution to the default production strategy.

A/B Testing Statistics

You can view statistics from your A/B tests by navigating to the individual A/B test page from the Skrym Icon With BackgroundStrategies page on Scope. For each A/B test, you can see the following statistics:

  • Total number of presentations
  • Total number of conversions and conversion rate
  • Number of confirmed orders for each transport method
  • Average displayed shipping price for confirmed orders
  • Average shipping cost you incurred for confirmed orders
  • Average profit and loss you incurred on shipping

Checkout Statistics

If you have any questions or concerns, please feel free to reach out to info@skrym.se!