Contents
Which test is relevant for unequal variances?
Welch’s Test for Unequal Variances (also called Welch’s t-test, Welch’s adjusted T or unequal variances t-test) is a modification of Student’s t-test to see if two sample means are significantly different.
Can you do an unpaired t-test with different sample sizes?
A paired t-test when you have unequal sample sizes does not make any sense, conceptually or mathematically. Conceptually, a paired t-test is good for when your “before” values have a lot of variance, relative to the difference between your before and after values.
How to compare a sample with a distribution?
When we compare a sample with a theoretical distribution, we can use a Monte Carlo simulation to create a test statistics distribution. For instance, if we want to test whether a p-value distribution is uniformly distributed (i.e. p-value uniformity test) or not, we can simulate uniform random variables and compute the KS test statistic.
How to compare two distributions using discrete KS?
The following is a procedure to conduct the discrete KS test for two samples: Find the min and max of the combined sample to define our range. e.g. for a sample size of 500, we can expect 25 samples per bin by choosing 20 buckets.
How to compare two income distributions in practice?
The red vertical line is the KS test statistic value of the two original samples. As expected, the KS test statistic for the actual income samples is far away from the distribution. This suggests we can reject the null hypothesis that states the income samples are identical (i.e. p-value is zero).
How to compare two p-value distributions in practice?
For instance, if we want to test whether a p-value distribution is uniformly distributed (i.e. p-value uniformity test) or not, we can simulate uniform random variables and compute the KS test statistic. By repeating this process 1000 times, we will have 1000 KS test statistics, which gives us the KS test statistic distribution below.