Critical regions and actual significance level

The critical-region method for a one-tailed test: the critical value is the boundary and the critical region is the set of outcomes that would lead to rejecting H0. How to find it (the smallest c with P(X ≥ c) ≤ α for an upper test, the largest c with P(X ≤ c) ≤ α for a lower test), why the discreteness of the binomial means the actual significance level is usually below the nominal α, and how to use the region to conclude.

Statistics · Hypothesis testing

Critical regions and actual significance level

Why it works

Instead of computing a probability for each new observation, we can work out in advance which outcomes would cause us to reject

H_0

. That set is the critical region (CR), and its boundary value is the critical value. Then the test is just: is the observed value in the critical region?

Finding the critical value. Assume

H_0 : p = p_0

, so

X \sim B(n, p_0)

Upper test ( $H_1 : p > p_0$ ): the CR is $X \ge c$ , where $c$ is the smallest

value with

P(X \ge c) \le \alpha

Lower test ( $H_1 : p < p_0$ ): the CR is $X \le c$ , where $c$ is the largest

value with

P(X \le c) \le \alpha

.

You find

c

by checking cumulative probabilities either side of the boundary — the chosen

c

keeps the tail

\le \alpha

, while the next value out would push it over.

The actual significance level. Because

X

is discrete, the tail probability almost never equals

\alpha

exactly. The actual significance level is the real probability of landing in the critical region when

H_0

is true —

P(X \ge c)

for an upper test or

P(X \le c)

for a lower test. It is the genuine chance of a wrong rejection, and is usually a bit less than the nominal

\alpha

(e.g. an actual

4.68\%

for a "

5\%

" test).

Using the region. Reject

H_0

if and only if the observed value falls inside the critical region. This gives exactly the same decision as the probability method — it is just organised differently, which is handy when several results are tested against the same

p_0