Skip to content

A/B Tests

A/B testing is a way for developers to run a controlled experiment between two or more versions of something in their game to determine which is more effective. Your audience is segmented into two groups: the control group (current performance – no changes) and your test group.

A/B testing is a great way to find the best pricing, game difficulty or test any of your hypotheses.

Screenshot

Before using all AB Tests must be added to the table.

Name Description
Name The name of the Test.
Variants Each user gets one random variant.
Status Active - the Test is running. Inactive - the test was canceled or wasn't yet started. Finished - the test is finished, and all new users will automatically be assigned to the Best Variant, meanwhile old user won't change their variant. FinishedWithChange - the test is finished and all users, including the old ones, will change their variant to the Best Variant.
Target Users The portion of your total users, who will participate in the test.
Users Type Defines the kind of users to target the test: New, Old, or All. If the app is opened for the first time, the user is considered a new one until he restarts the app.
Condition Conditions that are required to start the A/B Test.
Best Variant Is used when AB Test is over and you want to specify the variant which performed the best.

Balancy ensures that each user is not simultaneously participating in two A/B tests. If a user has a running A/B test, he is skipping all other A/B tests, even if they target 100% of the audience. Once the A/B test is finished, the user will be able to join a new one.

A/B tests don't need priority. When the game starts and a user doesn't have any running A/B tests, the user collects all Active A/B tests and picks just one of them, according to the chances.

There are currently 3 ways to work with A/B testing:

  1. Conditions
  2. Use the AB Test Node in Visual Scripting
  3. Make your logic

Section for programmers

After the initialization of the SmartObjects, you can get all current A/B tests and their values for the user. Call this method only after OnSmartObjectsInitialized is invoked.

var allTests = AbTestsManager.GetAllTests();
Debug.LogWarning("All Tests " + allTests.Count);
foreach (var test in allTests)
    Debug.Log("Name = " + test.AbTest.Name + " ; Variant = " +test.Variant.Name + " ; isFinished = " + test.Finished);

Next: Segmentation