Contents
Should test and control groups be the same size?
The size of the control group, or any test group for that matter, depends on the size of the total population. If the desired confidence level for the test is 95% and the minimum acceptable margin of error is 5%, the control group will need to be larger, about 20% for the 100 participant example above.
Which methodology is a control group?
See also scientific method. A typical use of a control group is in an experiment in which the effect of a treatment is unknown and comparisons between the control group and the experimental group are used to measure the effect of the treatment.
How big should the sample size be for a control group?
You should not allocate less than 20% of the sample to the control condition, save for situations when you are looking for large effects (e.g., 8 point lifts) and/or using large samples (e.g., 15,000 participants).
When to reduce the size of the control group?
Minimal losses in power occur when we shrink the control size to 40% [of the sample]. A 25% to 30% range is a good compromise, as this exposes 70% of the sample to the treatment, yet still does not harm power terribly.
How to test the hypothesis of sample size?
The hypothesis you are trying to test is that the mean of the experiment group is the same mean as the mean of the control . Essentially you are testing if . Suppose that the true variance (not sample variance) of the experiment group is and that you have a sample size . Likewise the control group variance and sample size is and .
What is the true variance of the experiment group?
Suppose that the true variance (not sample variance) of the experiment group is σ E 2 and that you have a sample size n E. Likewise the control group variance and sample size is σ C 2 and n C. From your samples you will be examining X E ¯ − X C ¯ to test the hypothesis μ E − μ C = 0.