Skip to content

A/B Tests

A/B testing is a way for developers to run a controlled experiment between two or more versions of something in their game to determine which is more effective. Your audience is segmented into two groups: the control group (current performance – no changes) and your test group.

A/B testing is a great way to find the best pricing, game difficulty or test any of your hypotheses.

Before using all AB Tests must be added to the table.

Name Description
Name The name of the Test.
Groups Each user gets one random group.
Status - Active - the test is running.
- Inactive - the test was canceled or wasn't yet started.
- Updated - the test was changed, but the changes are not deployed.
- Finished - the test is finished, and all new users will automatically be assigned to the Best Group.
Target Audience The portion of your total users, who will participate in the test.
Users Type Defines the kind of users to target the test: New, Old, or All. If the app is opened for the first time, the user is considered a new one until he restarts the app.
Conditions Conditions that are required to start the A/B Test.

Screenshot

Balancy ensures that each user is not simultaneously participating in two A/B tests. If a user has a running A/B test, he is skipping all other A/B tests, even if they target 100% of the audience. Once the A/B test is finished, the user will be able to join a new one.

A/B tests don't need priority. When the game starts and a user doesn't have any running A/B tests, the user collects all Active A/B tests and picks just one of them, according to the chances.

There are currently 3 ways to work with A/B testing:

  1. Conditions
  2. Use the AB Test Node in Visual Scripting
  3. Make your logic

Section for programmers

After the initialization of the LiveOps package, you can get all current A/B tests and their values for the user. Call this method only after OnSmartObjectsInitialized is invoked. 

var allTests = Balancy.LiveOps.ABTests.GetAllTests();
Debug.LogWarning("All Tests: " + allTests.Count);
foreach (var test in allTests)
    Debug.Log("Name = " + test.AbTest.Name + "; Variant = " + test.Variant.Name + "; isFinished = " + test.Finished);

Next: Segmentation