Skip to content

A/B Tests

A/B testing allows developers to run a controlled experiment between two or more versions of something in their game to determine which is more effective. Your audience is segmented into the control group (current performance – no changes) and your test group.

A/B testing is a great way to find the best pricing and game difficulty or test any of your hypotheses.

Before using, all AB Tests must be added to the table.

Name Description
Name The name of the Test.
Groups Each user gets one random group.
Target Audience The portion of your total users participating in the test.
Users Type Defines the kind of users to target the test: New, Old, or All. If the app is opened for the first time, the user is considered a new one until he restarts the app.
Conditions Conditions that are required to start the A/B Test. After A/B Test starts for a player, it won't be stopped for him even if the conditions turn to False.

Screenshot

Balancy ensures that each user is not simultaneously participating in two A/B tests. If a user has a running A/B test, he is skipping all other A/B tests, even if they target 100% of the audience. Once the A/B test is finished, the user can join a new one.

A/B tests don't need priority. When the game starts, and a user doesn't have any running A/B tests, the user collects all Active A/B tests and picks just one of them, according to the chance.

There are currently 3 ways to work with A/B testing:

  1. Conditions
  2. Use the AB Test Node in Visual Scripting
  3. Make your logic

Section for programmers

After the initialization of the LiveOps package, you can get all current A/B tests and their variants for the user. Call this method only after OnSmartObjectsInitialized is invoked.

var allTests = Balancy.LiveOps.ABTests.GetAllTests();
Debug.LogWarning("All Tests: " + allTests.Count);
foreach (var test in allTests)
    Debug.Log("Name = " + test.AbTest.Name + "; Variant = " + test.Variant.Name + "; isFinished = " + test.Finished);

//You can manually launch a test and place the user in the specified variant:
LiveOps.ABTests.StartABTestManually(test, variant);

External Tests

If you want to manually launch AB Tests and define the variants assigned to each player using your algorithms or any 3rd party solution, follow the next steps:

  1. When creating AB Test add an Empty condition with false value, automatically making AB Test unavailable for all players.

    Screenshot 2. Invoke the following code to start AB Test and assign a variant manually:

    var allABTest = DataManager.SmartObjects.Analytics.ABTests;
    var myTest = allABTest[0];//find the AB Test you want to start for this player
    var myVariant = myTest.Variants[0];//find the variant you need
    
    LiveOps.ABTests.StartABTestManually(myTest, myVariant);