PROOF 199

Prove.

The sample variance divides by n−1

On a probability space.

A way to read this proof

First separate what is assumed from what is to be proved. At each step, ask which definition or earlier result permits the move. If it is unclear, return to that premise rather than skipping to the conclusion.

Definition D1

X₁, …, Xₙ are mutually independent and identically distributed: E(Xᵢ) = μ, Var(Xᵢ) = E[(Xᵢ − μ)²] = σ². Define the sample mean X̄ = (X₁ + ⋯ + Xₙ)/n.

Proof.

(1)Define the variance by Var(X) = E[(X − E X)²] and the covariance by Cov(X, Y) = E[(X − E X)(Y − E Y)].convention

(2)We work on a finite probability space. Expectation is determined as a sum over sample points.convention

∎

Lemma L1

E(X̄) = μ

Proof.

(1)E(X̄) = (1/n)(E(X₁) + ⋯ + E(Xₙ))D1, A1

(2)　　 = (1/n) · nμ = μ(1), D1, N

∎

Lemma L2

Var(X̄) = σ²/n

Proof.

(1)X̄ − μ = (1/n) Σ(Xᵢ − μ)D1, N

(2)(X̄ − μ)² = (1/n²) ΣᵢΣⱼ (Xᵢ − μ)(Xⱼ − μ)(1), A3, N

(3)E[(X̄ − μ)²] = (1/n²) ΣᵢΣⱼ E[(Xᵢ − μ)(Xⱼ − μ)](2), A1

(4)For i ≠ j, E[(Xᵢ−μ)(Xⱼ−μ)] = Cov(Xᵢ, Xⱼ) = 0. All n(n−1) such terms vanish.D1, A2

(5)For i = j, E[(Xᵢ−μ)²] = σ². All n such terms remain.D1

(6)E X̄ = μ, so Var(X̄) = E[(X̄−μ)²] = (1/n²) · nσ² = σ²/n(3), (4), (5), L1, D1, N

∎

Lemma L3

Σ(Xᵢ − X̄)² = Σ(Xᵢ − μ)² − n(X̄ − μ)²

Proof.

(1)Xᵢ − X̄ = (Xᵢ − μ) − (X̄ − μ)N

(2)(Xᵢ − X̄)² = (Xᵢ−μ)² − 2(Xᵢ−μ)(X̄−μ) + (X̄−μ)²(1), A4

(3)Summing over i gives Σ(Xᵢ−X̄)² = Σ(Xᵢ−μ)² − 2(X̄−μ) Σ(Xᵢ−μ) + n(X̄−μ)²(2), N

(4)Σ(Xᵢ − μ) = ΣXᵢ − nμ = n(X̄ − μ)D1, N

(5)The middle term is −2(X̄−μ) · n(X̄−μ) = −2n(X̄−μ)²(3), (4), N

(6)Σ(Xᵢ−X̄)² = Σ(Xᵢ−μ)² − 2n(X̄−μ)² + n(X̄−μ)² = Σ(Xᵢ−μ)² − n(X̄−μ)²(3), (5), N

∎

Theorem T1

E[Σ(Xᵢ − X̄)²] = (n − 1)σ²

Proof.

(1)E[Σ(Xᵢ−X̄)²] = Σ E[(Xᵢ−μ)²] − n · E[(X̄−μ)²]L3, A1

(2)Σ E[(Xᵢ−μ)²] = nσ²D1, N

(3)E[(X̄−μ)²] = Var(X̄) = σ²/nL1, L2, D1

(4)E[Σ(Xᵢ−X̄)²] = nσ² − n · (σ²/n) = nσ² − σ² = (n−1)σ²(1), (2), (3), N

∎

Corollary C

Defining s² = Σ(Xᵢ − X̄)²/(n − 1) gives E(s²) = σ². Dividing by n leaves the expectation short of σ².

Proof.

(1)E(s²) = E[Σ(Xᵢ−X̄)²]/(n−1) = (n−1)σ²/(n−1) = σ²T1, A1, N

(2)Dividing by n gives E[Σ(Xᵢ−X̄)²/n] = (n−1)σ²/n = σ² − σ²/nT1, A1, N

(3)If σ² > 0, this value is always smaller than σ². The moment deviations are measured from X̄ rather than μ, the sum of squares shrinks systematically by n(X̄−μ)² — in expectation, by exactly one σ².(2), L3, N

(4)Therefore the divisor whose expectation matches σ² is n − 1.(1), (2)

∎

Remark

Can A2 be removed from this proof?

Find the line where the number n − 1 is born. When independence falls, that number falls with it.