Setting up a hypothesis test

The language and structure of a binomial hypothesis test: the population proportion p, the null hypothesis H0 : p = p0 and the alternative hypothesis H1, deciding between a one-tailed (p < p0 or p > p0) and a two-tailed (p ≠ p0) test from the wording, the test statistic X ~ B(n, p0) under H0, and what the significance level means.

Statistics · Hypothesis testing

Setting up a hypothesis test

Why it works

A hypothesis test uses the result of one sample to decide between two competing claims about a population parameter — here a population proportion

p

(the probability of "success": a defective item, a red bead, a late customer). We can't see the whole population, so we reason from how surprising the sample would be if a particular value of

p

were true.

The two hypotheses.

The null hypothesis $H_0$ is the "no change / status quo" claim, and is always an

equality:

H_0 : p = p_0

, where

p_0

is the value being assumed (often a previously known or claimed proportion).

The alternative hypothesis $H_1$ is what you are testing for. It takes one of

three forms, chosen from the wording:

$H_1 : p > p_0$ — an upper one-tailed test ("increased", "more than", "better");
$H_1 : p < p_0$ — a lower one-tailed test ("decreased", "fewer", "worse");
$H_1 : p \ne p_0$ — a two-tailed test ("changed", "different", "not equal to").

The single most useful reading skill is spotting whether a problem claims a direction (one-tailed) or merely a change (two-tailed).

The test statistic. We assume $H_0$ is true and model the number of successes in the sample as

X \sim B(n, p_0),

where

n

is the sample size. We then ask: under this model, how likely is a result as extreme as the one observed? If that probability is small enough,

H_0

looks implausible.

The significance level. The significance level

\alpha

(commonly

5\%

1\%

) is the cut-off for "small enough". Crucially,

\alpha

is the probability of rejecting $H_0$ when it is actually true — the risk of a wrong rejection you are willing to accept. A smaller

\alpha

demands stronger evidence before rejecting

H_0

.

The decision. Compute the probability, in the relevant tail, of a result at least as extreme as the observed one (assuming

H_0

). If it is less than $\alpha$ (for a two-tailed test, less than

\tfrac{\alpha}{2}

in that tail), the result lands in the unlikely region, so you reject $H_0$ ; otherwise there is insufficient evidence and you do not reject $H_0$ . Every conclusion must be stated in context.