Math · Introductory statistics · Concept
Hypothesis testing: t-tests and p-values
A hypothesis test asks whether data are consistent with a claim. The null hypothesis states the claim, such as μ = 500; the alternative states what you suspect instead. The t statistic measures how far the sample mean falls from the claim in standard errors, and the p-value is the probability of a result at least that extreme if the null hypothesis were true.
Two hypotheses
H₀ is the claim being tested and is written with =. Hₐ is the alternative: μ ≠ μ₀ for a two-sided test, or μ < μ₀ or μ > μ₀ for a one-sided test chosen before seeing the data.
The t statistic
The distance from the claim, measured in standard errors. When H₀ is true and the population is roughly normal, it follows a t distribution with n − 1 degrees of freedom.
The p-value
The p-value is the probability, assuming H₀ is true, of a statistic at least as extreme as the one observed, in the direction of Hₐ. A two-sided test counts both tails.
The decision
Choose a significance level α, often 0.05, before testing. If p ≤ α, reject H₀: the data would be unusual if it were true. If p > α, fail to reject H₀, which is not proof that H₀ is true.
Wrong decisions
Rejecting a true H₀ happens with probability α. Failing to reject a false H₀ is also possible, especially with small samples; a larger sample makes the test more sensitive.
Significance is not size
A tiny difference can be statistically significant in a huge sample, and a large one can fail to be in a small sample. Report the estimate and a confidence interval, not just the decision.
Tests and intervals agree
A two-sided test at α = 0.05 rejects μ = μ₀ exactly when μ₀ lies outside the 95% t interval.
Common mistakes
- Reading the p-value as the probability that H₀ is true.
- Accepting H₀ when p > α: the data only fail to reject it.
- Choosing a one-sided alternative after seeing which way the data point.
- Reporting one tail for a two-sided test: a two-sided p-value counts both tails.
Key terms
- Hypothesis test
- A way to check whether data are consistent with a null hypothesis: compute a test statistic and ask how surprising it would be if the null were true. A test can give evidence against the null, but it never proves the alternative.
- Null hypothesis
- H₀, the claim a test assumes so it can compute probabilities, usually “no effect” or a specific value such as μ = 50. Failing to reject it doesn’t prove it true.
- Alternative hypothesis
- Hₐ, the claim you are looking for evidence of, such as μ > 50 (one-sided) or μ ≠ 50 (two-sided). Choose it from the question before looking at the results.
- p-value
- The probability, assuming the null hypothesis is true, of a result at least as extreme as the one observed. A small p-value is evidence against the null; it is not the probability that the null is true.
- Significance level
- α, the cutoff chosen before a test, often 0.05: reject the null if the p-value is below it. It is the chance of rejecting a null hypothesis that is actually true.
- t-test
- A test about a mean, or a difference in means, that uses the t distribution. A paired test works on the differences within pairs; Welch’s two-sample test doesn’t assume equal variances.
- Student t distribution
- A bell-shaped distribution like the standard normal but with heavier tails, used when the population standard deviation is estimated from the sample. With more degrees of freedom it gets closer to the normal.
- Degrees of freedom
- The number of values free to vary once estimates have been fixed, such as n − 1 for one sample’s standard deviation. It sets the shape of the t, chi-square and F distributions.
Work through an example
A filling machine should put 500 mL in each bottle. A random sample of 25 bottles has mean 497.2 mL and standard deviation 6 mL. Test H₀: μ = 500 against Hₐ: μ ≠ 500 at α = 0.05.
Run a one-sample t-test →Sources and scope
Authored study material. Tool results depend on the stated inputs and model assumptions.
Try in the workspace
Open the example inputs, change a value and keep a useful result on your board.
Find the p-value in Statistics Check t in Math Open worked example on a board Confidence interval in Math ReferenceYour existing work stays on this device. Examples open as editable copies.