“The Student’s t Test is used to compare the mean of two normally distributed samples, preferably of equal size and variance. More specifically, the Student’s t Test gives you a probabilistic estimate of the likelihood that your samples were randomly selected from the same population, i.e. that the Null Hypothesis is true. If the Student’s t Test suggests that your samples were probably not taken from the same population, you can conclude, with some certainty (how certain depends on your p value), that your samples were taken from different populations.
Thinking of the Student’s t Test in terms of “signal to noise” ratios, the “signal” is the difference between the mean of two samples (similar to the Z test). All other things being equal, large differences in means are more likely to be statistically significant. However, if the distribution of values in the population (or, in this case, the samples) is large, (i.e. there is a lot of noise), the difference in means is less likely to be significant, and more likely to be due to random variation. For the Student’s t Test, the square root of the sum of the sample variances squared, divided by the sample sizes, is used as an estimate of noise.
Absolute value of (sample mean – population mean)
sqrt(s1^2/n1 + s2^2/n2) where s = variance and n = number in each sample
Formual for t
t = [abs(population mean – sample mean)] / [sqrt(s1^2/n1 + s2^2/n2)]
Getting a p value
Once you have t, you probably want to know if it is “statistically significant.” In order to do this, you have to look it up in a table, or use a computer program. Remember, the t Test assumes normally distributed data. Because the probability of normally distributed occurrences obeys known mathematical formulas, we (or statisticians) can theoretically calculate the likelihood that a given z value occurs simply out of chance. In order to look up the p value that corresponds to your t statistic, you need to know the degree of freedom, which in this case is (n1 + n2 – 2). You also have to decide whether or not you are conducting a one-tailed (ex. x > X) or two tailed (ex. x != X) test. Note that as your degrees of freedom increase (i.e. the sample sizes get larger), the t test approaches the Z test. Whether or not the P Value you find is statistically significant depends on what P Value you find acceptable.
How to Calculate
Comparison to the z Test
The major difference between the t Test and the z Test is that with the z Test, one of the groups (the population) is much larger than the other (the sample). So, when thinking about signal:noise ratios, it is easy to calculate noise (which in this case is the Standard Error of the Mean). With the t Test, there is no reference “population,” so you don’t have the luxury of accurately calculating noise. To compensate for that, the t Test approximates noise by using the variances. As the sample population sizes get larger (and the degrees of freedom increase), the t statistic approaches the z statistic, i.e. the tests are essentially the same.”