Definition D1
The mean x̄ of data x₁, …, xₙ is the value satisfying n x̄ = Σxᵢ. With the data arranged in increasing order and n = 2m + 1, the middle value x₍ₘ₊₁₎ is called the median x̃.
Proof.
(1)Σaᵢ means a₁ + ⋯ + aₙ; u − v means u + (−v); u ≤ v means u < v or u = v.convention
(2)That the mean exists and is unique was obtained in Proof 64. Below we treat the case of odd n.convention
∎
Definition D2
If 0 ≤ t, set |t| = t; if t < 0, set |t| = −t. For a reference value c, set L(c) = Σ|xᵢ − c|, S(c) = Σ(xᵢ − c)².
Proof.
Both expressions are quantities with the signs of the deviations removed. Only the method of removal differs.convention
∎
Lemma L1
If a ≤ b, then for every c, |c − a| + |c − b| ≥ b − a, with equality exactly when a ≤ c ≤ b.
Proof.
(1)By trichotomy there are only three cases: c < a, a ≤ c ≤ b, b < c.assumption, A10
(2)If a ≤ c ≤ b, then |c − a| = c − a, |c − b| = b − c, so the sum is b − a.(1), D2, A1, A2, A3, A4
(3)Suppose c < a. Since a ≤ b, c < b, and |c − a| = a − c, |c − b| = b − c(1), A9, D2
(4)(a − c) + (b − c) = (b − a) + 2(a − c)(3), A1, A2, A3, A4, E
(5)Since c < a, 0 < a − c, and hence 0 < 2(a − c)(3), A7, A4, A8
(6)b − a < (b − a) + 2(a − c)(5), A7, A2, A3
(7)The case b < c gives, by the same method, b − a < (b − a) + 2(c − b)(1), D2, A1, A2, A3, A4, A7, A8, A9, E
(8)Combining the three cases, the inequality holds, with equality only in case (2).(2), (6), (7)
∎
Theorem T1
The c minimizing S(c) is the mean, and the c minimizing L(c) is the median. The choice of loss determines the choice of representative value.
Proof.
(1)The square side was obtained in Proof 65. The only c minimizing S(c) is x̄.A14, D2
(2)Now consider the absolute-value side. Suppose n = 2m + 1 and x₁ ≤ x₂ ≤ ⋯ ≤ xₙ.assumption, D1
(3)For k = 1, …, m, applying L1 with a = x_k, b = x_{n+1−k} gives |c − x_k| + |c − x_{n+1−k}| ≥ x_{n+1−k} − x_k, with equality exactly when x_k ≤ c ≤ x_{n+1−k}.(2), L1
(4)The remaining term is the single |c − x₍ₘ₊₁₎|. It is at least 0 and is 0 only when c = x₍ₘ₊₁₎.(2), D2, A10
(5)Setting K = Σ_{k≤m}(x_{n+1−k} − x_k), L(c) ≥ K + |c − x₍ₘ₊₁₎|(3), (4), D2, A7
(6)For every k ≤ m, x_k ≤ x₍ₘ₊₁₎ ≤ x_{n+1−k}, so at c = x₍ₘ₊₁₎ all the equalities in (3) hold and the remaining term is 0. That is, L(x₍ₘ₊₁₎) = K(2), (3), (4)
(7)If c ≠ x₍ₘ₊₁₎, then 0 < |c − x₍ₘ₊₁₎|, so K < K + |c − x₍ₘ₊₁₎| ≤ L(c)(4), (5), A7, A3, A2
(8)Therefore L(x̃) < L(c), and the only c minimizing L is the median x̃.(6), (7), D1
(9)If n is even, there is no remaining term and the equalities in (3) hold for every c between the two middle values, so the minimum is not attained at a single point.(3), (5)
∎
Theorem T2
For paired data (xᵢ, yᵢ), let dᵢ = xᵢ − x̄, eᵢ = yᵢ − ȳ. If Σdᵢeᵢ = 0, then the sum of squares of x + y is the sum of the sums of squares of x and of y.
Proof.
(1)Σ(xᵢ + yᵢ) = Σxᵢ + Σyᵢ = n x̄ + n ȳ = n(x̄ + ȳ), so the mean of x + y is x̄ + ȳ.A11, D1, A5, E
(2)(xᵢ + yᵢ) − (x̄ + ȳ) = dᵢ + eᵢ(1), A1, A2, A3, A4
(3)(dᵢ + eᵢ)² = dᵢ² + 2dᵢeᵢ + eᵢ²(2), A13
(4)Σ((xᵢ+yᵢ) − (x̄+ȳ))² = Σdᵢ² + 2Σdᵢeᵢ + Σeᵢ²(3), D2, A11, A12
(5)Suppose Σdᵢeᵢ = 0. Then the middle term is 0, so the sums of squares add exactly.assumption, (4), A3
∎
Theorem T3
Under the same condition, the sums of absolute values do not add.
Proof.
(1)Consider the data x = (1, 1, −1, −1), y = (1, −1, 1, −1). Since x̄ = ȳ = 0, dᵢ = xᵢ, eᵢ = yᵢ.D1, N
(2)Σdᵢeᵢ = 1 + (−1) + (−1) + 1 = 0, so this data satisfies the condition of T2.(1), N, T2
(3)x + y = (2, 0, 0, −2) and its mean is 0, so L_{x+y} = 2 + 0 + 0 + 2 = 4(1), D1, D2, N
(4)L_x = 1 + 1 + 1 + 1 = 4, L_y = 4, so L_x + L_y = 8(1), D2, N
(5)Since 4 ≠ 8, the sums of absolute values do not add.(3), (4), N
(6)For the same data, S_x = 4, S_y = 4, S_{x+y} = 4 + 0 + 0 + 4 = 8, so the sums of squares add.(1), D2, N, T2
∎
Theorem T4
f(t) = t² is differentiable at every t, but g(t) = |t| is not differentiable at t = 0.
Proof.
(1)(t²)′ = 2t, so f is differentiable at every t.A17
(2)g(0) = 0.D2
(3)If 0 < h, then (g(h) − g(0))·h⁻¹ = h·h⁻¹ = 1(2), D2, A19, A3
(4)If h < 0, then (g(h) − g(0))·h⁻¹ = (−h)·h⁻¹ = −(h·h⁻¹) = −1(2), D2, A6, A19, A3
(5)Adding −1 to both sides of 0 < 1 gives −1 < 0, and by transitivity −1 < 1, so by trichotomy 1 ≠ −1A18, A7, A2, A3, A4, A9, A10
(6)If the difference quotient approached a single value as h → 0, that value would have to be both 1 and −1.(3), (4), A16
(7)This contradicts (5), so the limit does not exist, and g is not differentiable at 0.(5), (6), A15
(8)Each term |c − xᵢ| of L(c) is likewise not differentiable at c = xᵢ, for the same reason.(7), D2
∎
Corollary C
Choosing between squares and absolute values is not a matter of right and wrong, but of what comes along with the choice.
Proof.
(1)Choosing squares makes the representative value the mean; choosing absolute values makes it the median.T1
(2)Choosing squares makes the spreads of two uncorrelated data sets add; with absolute values they do not add even under the same condition.T2, T3
(3)Choosing squares makes the loss differentiable at every point; choosing absolute values makes it non-differentiable at the data values.T4
(4)The data 1, 2, 3, 4, 100 have mean 22 and median 3.D1, N
(5)Changing 100 to 1000 makes the mean 202 while the median stays 3.D1, N
(6)Therefore, choosing absolute values keeps the representative value from moving even when one data point moves greatly.(4), (5), T1
(7)The two choices confer different properties. Neither replaces the other.(1), (2), (6)
∎