A/B Tests¶
A/B testing allows developers to run a controlled experiment between two or more versions of something in their game to determine which is more effective. Your audience is segmented into the control group (current performance – no changes) and your test group.
A/B testing is a great way to find the best pricing and game difficulty or test any of your hypotheses.
Before using, all AB Tests must be added to the table.
Name | Description |
---|---|
Name | The name of the Test. |
Groups | Each user gets one random group. |
Target Audience | The portion of your total users participating in the test. |
Users Type | Defines the kind of users to target the test: New, Old, or All. If the app is opened for the first time, the user is considered a new one until he restarts the app. |
Conditions | Conditions that are required to start the A/B Test. After A/B Test starts for a player, it won't be stopped for him even if the conditions turn to False. |
Balancy ensures that each user is not simultaneously participating in two A/B tests. If a user has a running A/B test, he is skipping all other A/B tests, even if they target 100% of the audience. Once the A/B test is finished, the user can join a new one.
A/B tests don't need priority. When the game starts, and a user doesn't have any running A/B tests, the user collects all Active A/B tests and picks just one of them, according to the chance.
There are currently 3 ways to work with A/B testing:
- Conditions
- Use the AB Test Node in Visual Scripting
- Make your logic
Section for programmers¶
After the initialization of the LiveOps package, you can get all current A/B tests and their variants for the user. Call this method only after OnSmartObjectsInitialized is invoked.
var allTests = Balancy.LiveOps.ABTests.GetAllTests();
Debug.LogWarning("All Tests: " + allTests.Count);
foreach (var test in allTests)
Debug.Log("Name = " + test.AbTest.Name + "; Variant = " + test.Variant.Name + "; isFinished = " + test.Finished);
//You can manually launch a test and place the user in the specified variant:
LiveOps.ABTests.StartABTestManually(test, variant);
External Tests¶
If you want to manually launch AB Tests and define the variants assigned to each player using your algorithms or any 3rd party solution, follow the next steps:
-
When creating AB Test add an Empty condition with false value, automatically making AB Test unavailable for all players.
2. Invoke the following code to start AB Test and assign a variant manually:
var allABTest = DataManager.SmartObjects.Analytics.ABTests; var myTest = allABTest[0];//find the AB Test you want to start for this player var myVariant = myTest.Variants[0];//find the variant you need LiveOps.ABTests.StartABTestManually(myTest, myVariant);