Cumulative frequency, box plots and outliers

Building a cumulative frequency diagram and reading the median, quartiles and percentiles off it by plotting against the UPPER class boundary; testing for outliers with the 1.5×IQR rule or the mean ± 2 standard deviations rule; drawing a box plot whose whiskers stop at the most extreme value that is NOT an outlier; and comparing two distributions with a measure of location and a measure of spread.

Statistics · Data presentation

Cumulative frequency, box plots and outliers

Why it works

A grouped frequency table hides where individual values sit, so we can't read the median straight off it. A cumulative frequency diagram fixes that by recording, at each class, the running total of "how many values so far".

Plot against the upper class boundary. The cumulative frequency for a class is the total of that class and all earlier ones — and you only know you have reached that total once you pass the top of the class. So each point is

(\text{upper class boundary}, \text{cumulative frequency})

, starting from the lower end of the first class with cumulative frequency

0

. Join the points (a smooth curve or straight segments) to get a graph you can read both ways.

Here is the cumulative frequency diagram for the times (minutes) of

60

people, with class frequencies

6, 14, 20, 14, 6

:Reading a value. To estimate the median, go up the cumulative-frequency axis to the position, across to the curve, then down to the data axis. For grouped continuous data the positions are

\text{median at } \tfrac{n}{2}, \qquad Q_1 \text{ at } \tfrac{n}{4}, \qquad Q_3 \text{ at } \tfrac{3n}{4}, \qquad \text{the } k\text{th percentile at } \tfrac{k}{100}n.

Use

\tfrac{n}{2}

, not

\tfrac{n+1}{2}

— the

+1

version is for a discrete list of values, but a cumulative frequency graph is a continuous estimate. Here

n = 60

, so the median is read at cumulative frequency

30

: it falls between

(20, 20)

and

(30, 40)

, and interpolating,

20 + \tfrac{30-20}{40-20}\times 10 = 25

minutes.

Outliers. An outlier is a value far enough from the middle to be treated as unusual. A question always tells you which rule to use; the two standard ones are:

$1.5 \times \text{IQR}$ rule — a value is an outlier if it is below

Q_1 - 1.5(\text{IQR})

or above

Q_3 + 1.5(\text{IQR})

$2$ standard deviations rule — a value is an outlier if it is more than $2$

standard deviations from the mean, i.e. outside

\bar{x} \pm 2\sigma

.

The boundary values

Q_1 - 1.5(\text{IQR})

and

Q_3 + 1.5(\text{IQR})

are the fences; measure from the quartiles (or the mean), never from the median.

Box plots. A box-and-whisker plot draws the five-number summary on one axis: a box from

Q_1

Q_3

with the median marked inside, and whiskers out to the lowest and highest values. The catch: a whisker stops at the most extreme value that is not an outlier, and any outliers are plotted separately as crosses beyond it.Here the whisker reaches

48

(the largest in-range value) and the outlier at

60

is a separate cross — the whisker does not stretch out to it.

Skewness from the quartiles. If

Q_3 - Q_2 > Q_2 - Q_1

the longer tail is on the right, so the data is positively skewed; if

Q_2 - Q_1 > Q_3 - Q_2

it is negatively skewed.

Comparing two distributions. Always compare two things, in context: a measure of location (usually the median) — which set is higher on average — and a measure of spread (usually the IQR) — which set is more consistent. "The median mark in class B (28) is higher than in class A (22), so B scored higher on average; the IQR in B (6) is smaller than in A (10), so B's marks were more consistent."