Leave lesson

Statistics · Statistical distributions

1 / 11

Linear combinations of random variables

E(aX + b) and Var(aX + b), expectation and variance of aX ± bY for independent variables — variances ADD even when the variables subtract — the difference between nX and the sum of n independent copies, and combining Normal (and Poisson) variables.

Statistics · Statistical distributions

Linear combinations of random variables

E(aX + b) and Var(aX + b), expectation and variance of aX ± bY for independent variables — variances ADD even when the variables subtract — the difference between nX and the sum of n independent copies, and combining Normal (and Poisson) variables.

Why it works

Scaling and shifting one variable. If every outcome of XX is transformed to aX+baX + b: E(aX+b)=a E(X)+b,Var(aX+b)=a2 Var(X).\mathrm{E}(aX + b) = a\,\mathrm{E}(X) + b, \qquad \mathrm{Var}(aX + b) = a^2\,\mathrm{Var}(X). The mean is dragged along by both the stretch and the shift. The variance ignores bb completely — shifting every value by the same amount moves the distribution without changing its spread — and picks up a2a^2, not aa, because variance is measured in squared units.

Combining two independent variables. For independent XX and YY: E(aX+bY)=a E(X)+b E(Y),\mathrm{E}(aX + bY) = a\,\mathrm{E}(X) + b\,\mathrm{E}(Y), Var(aX±bY)=a2 Var(X)+b2 Var(Y).\mathrm{Var}(aX \pm bY) = a^2\,\mathrm{Var}(X) + b^2\,\mathrm{Var}(Y). The second line contains the most-examined fact on this topic: variances add even for a difference. Subtracting an unpredictable quantity adds just as much uncertainty as adding one — X−YX - Y is more spread out than either variable, never less. If you ever compute a negative variance, this is the line that went wrong.

2X2X is not X1+X2X_1 + X_2. Doubling one measurement gives Var(2X)=4σ2\mathrm{Var}(2X) = 4\sigma^2; adding two independent copies gives Var(X1+X2)=2σ2\mathrm{Var}(X_1 + X_2) = 2\sigma^2. One error repeated is worse than two independent errors — independent fluctuations partly cancel. Read the context: "five times the mass of one bottle" is 5X5X; "the total mass of five bottles" is X1+⋯+X5X_1 + \cdots + X_5.

Normal in, Normal out. Any linear combination of independent Normal variables is itself Normal, with the mean and variance given by the rules above. That means questions about totals, differences and comparisons of Normal quantities all reduce to a single Normal probability calculation. (Similarly, a sum of independent Poisson variables is Poisson — the rates add.)

Comparisons become differences. "Find the probability that YY exceeds XX" is P(Y>X)=P(Y−X>0),\mathrm{P}(Y > X) = \mathrm{P}(Y - X > 0), computed from the distribution of D=Y−XD = Y - X. This turns a two-variable question into a standard one-variable Normal calculation.