Control Groups

On this page, we will explain what Control Groups are, why you should be using them, and how to set them up for your activities.

​🤷‍ What are they?

A Control Group can be added to any Activity. Adding a Control Group to your Activity will make sure that randomly selected players within the targeted Segment won't receive any player engagement from the Activity.

​🤔‍ Why are they used?

The simple reason why you would use a Control Group is to test and evaluate if a campaign was successful. Comparing the numbers from players that received the offer and didn't receive the offer, will give you an idea of how successful the campaign was.
With this information, you can then evaluate if it’s potentially worth running with a similar campaign in the future or maybe even stop an ongoing campaign. Or else, it can point toward some amendments being needed.
Follow up on the numbers for your Activity in the Activity Conversion Dashboard. Read more about Analysing the Results further down on this page. Still not convinced? Get inspired: Reasons Why You Should Be Using Control Groups​

​⚙‍ How does it work?

Inside the Activity or Lifecycle, you can set the percentage of your Control Group. Players inside your Segment will be picked at random to be included in the Control Group - all according to the percentage you've set.

​🔧‍ How to set it up

When you build your Activity, beneath your Action Group(s) you'll find the option to add a Control Group.
With a simple click, you activate the Control Group - setting the delivery percentage to a default 10%. This can easily be changed to any percentage of your liking, however, we believe that 10% is an ideal number.
How to add a control group
How to add a control group

​👩‍🔬‍ Analysing the Results

There's no point implementing Control Groups unless you invest time in analysing the results.
Once your Activity has run, it's time to compare the conversion numbers for the players who received the player engagements and the ones who received nothing at all.

📝‍ Tracking conversion

It's important that your Activities are tracking conversion in the first place. The option to track conversion is set inside your Activity when you build it. By default, your Activities are tracking conversion for 24 hours, but you can also choose to track the conversion for anywhere between 48h - 10 days:
Make sure to track conversion
Make sure to track conversion
If this is not set up in your Activities, the conversion numbers will not be stored for you, which means you can not follow up on any conversion data.

​🔢‍ Understanding the numbers

You can access all the collected conversion data inside of the Data Studio -> System Dashboard -> Activity Conversion.
There’s also a shortcut to the conversion numbers (taken from the Activity Conversion) for the specific Activity if you hover over the three-dot menu from the Activity overview:
Activity Conversion
Activity Conversion
Checking the Activity Conversion of your Activity you can see the top section of numbers, displayed in the picture below.
Example 1 of conversion numbers for an activity
Example 1 of conversion numbers for an activity
This is where you can compare the numbers for your Action Group(s) (containing players who received the player engagements) with your Control Group (containing the players who didn't receive anything).
Example 1: Looking at the numbers, from the Activity displayed in the above example, you can see that Action Group B is displaying better and more successful numbers in all areas in comparison to the Control Group.

​📊‍ How the Sample Size can affect the Summary outcome

Let’s take another example of an Action Group vs. a Control Group:
Example 3:
  1. Action Group A has 1500 fires and 250 players made a deposit, which makes the conversion rate 16.67%
  2. The Control Group had just 10 fires and 1 player made a deposit, which makes the conversion rate 10%
Does this mean that the Action Group A is better because it has a higher conversion rate?
In statistical terms, even though Action Group A is doing better than the Control Group, the fact that it only fired to 10 players means that the sample size is not big enough to justify that it’s better. Since the Control Group is very small, this is taken into consideration. Because there's only 10 people the logic is taking into account that this might just as well be a coincidence and not a trend. If the Control Group had 100 fires instead and 10 of the players made a deposit the conversion rate would still be 10%, but now we are more confident that the Action Group is actually doing better because we have a much larger sample to base the conclusion on. So, to sum it up: The statistical test checks if the difference between the conversion rates, when taking the sample size into account, is big enough to be significant or if it’s just potentially just a coincidence.