14 统计学基础--线性回归模型

下载 1

王木木

发布于

2232

人观看

#信息技术

首先介绍了一元线性回归模型、多元线性回归模型，然后讨论了可化为线性模型的情形，介绍了一些基本概念，如均值向量、协方差阵、随机向量的二次型、多元正态分布、正态变量二次型的分布，进一步介绍了回归系数的LS估计与性质、约束最小二乘估计、广义最小二乘估计。讨论了一般线性假设的检验、回归方程和回归系数的显著性检验，最后介绍了问题的预测。

展开查看详情

1 . 1ÔÙ ‚5£8 . §7.1 Ú ó ‚5£8 .´y“ÚOÆ¥A^••2• .ƒ˜§§•´Ù§ÚO .ïÄ½A^ Ä:"ùÌ‡ke A‡ Ïµ 1. 3¢S¯K¥§Cþƒm 'X~äk‚5½Cq‚5 •6'X" 2. 3y¢-.¥§•,NõCþm 'X´š‚5 § ²L· C†§ò¬¤•‚5 'X" 3. ‚5'X´Cþƒm•{ü 'X§N´?n§nØÚ•{' õ§ù •¢SA^ Jø k Ž{" !òÏL¢~`²‚5ÚO . µÚ©a" ˜!˜ ‚5£8 . Cþƒm 'XŒ—Œ©•(½5'XÚš(½5'XüŒa§ênÚO´?nš(½5 CþÚO5Æ5 Æ‰"‚5£8 .´š(½5(äk‘Å5)Cþƒm'X •Ä . ƒ˜§X< N-(Y ) † p(X) ƒmk˜½ ƒ•'Xµ X Œž§Y •–•uŒ§ XØ Uî‚û½ Y " ð þ(Y ) † ð¬«(X1 )!–•þ(X2 ) Ú Yþ(X3 ) k˜½ 'X§ „ ØUî‚|^êÆ¼ê'XLˆ§‚ƒm 'X" ±þ~f¥§Ï~¡ Y •ÏCþ½•ACþ§¡ X •gCþ"Y ŠküÜ©|¤µ˜Ü ©´U d X û½ Ü©§§´ X ¼ê§P• f (X)¶,˜Ü©´dÙ§¯õ™\•Ä Ïƒ ) K•§¡•‘ÅØ § kµ Y = f (X) + e, (7.1.1) ùp e Š•‘ÅØ §b½ E(e) = 0"AO§ f (X) ´‚5¼êž§f (X) = β0 + β1 X §Kk Y = β0 + β1 X + e. (7.1.2) (7.1.2) ª¡•‚5£8 .½‚5£8•§§Ù¥ β0 Ú β1 ™•§~ê‘ β0 ´£8†‚ y = β0 + β1 X å§β1 ´ Ç" k˜| (xi , yi ), i = 1, 2 · · · n§òþã .^ L«• yi = β0 + β1 xi + ei , i = 1, 2 · · · n. (7.1.3) ei •‘ÅØ "e^· O•{¦ β0 , β1 O• β0 Ú β1 §“\ (7.1.2)¥òØ ‘ ei ^ ÙþŠ 0 “O§ Y = β0 + β1 X, (7.1.4) ¡•² £8•§§§´d n | * Š¼ "XJ²u §´Ü· £8•§§K(7.1.4) •x Y † X ƒm ƒ''X" 1

2 . ~7.1.1 p(X) †N-(Y ) ƒmkCq£8'X(7.1.2)§e L«Ø p X §¤kK•N -(Y ) Ù¦Ïƒ(X¢D!Ø !âõ )§b½N n ‡< pÚN- (xi , yi ), i = 1, 2, · · · , n§ O β0 Ú β1 β0 = −40, β1 = 0.6§K² £8•§• Y = −40 + 0.6X. (7.1.5) XJ` p160 cm§Ž N- y0 = 56 kg§¡ y0 = 56 • p160cm N- ýÿŠ" !õ ‚5£8 . ¢S¯K¥K•ÏCþ gCþ ØŽ˜‡§Xk X1 , X2 , · · · , Xp−1 §K§‚kXe‚5 'Xµ Y = β0 + β1 X1 + · · · + βp−1 Xp−1 + e, (7.1.6) ek (xi1 , xi2 , · · · , xip−1 , yi ), i = 1, 2, · · · , n§Kk yi = β0 + β1 xi1 + · · · + βp−1 xip−1 + ei , (7.1.7) ei •‘ÅØ §òþã•§|^Ý Lµ        y1 1 x11 x12 · · · x1,p−1 β0 e1         y2   1 x21 x22 · · · x2,p−1  β1   e2          . = . .. .. .. ..  .. + .. ,  ..   .. . . . .  .   .         yn 1 xn,1 xn,2 · · · xn,p−1 βp−1 en =µ yn×1 = Xn×p βp×1 + en×1 , (7.1.8) Ù¥ y •*ÿ•þ§X ¡• O (S.¡{)§β •™•£8ëê•þ§e ´‘ÅØ •þ§' u e Ï~kü«b½µ (1) Gauss)Markovb½({¡G-Mb½)µE(e) = 0, Cov(e) = σ 2 I, =µ (a) E(ei ) = 0, i = 1, 2, · · · , n; (b) V ar(ei ) = σ 2 , i = 1, 2, · · · , n; (c) Cov(ei , ej ) = 0, i, j = 1, 2, · · · , n, …i = j. (2) b½µe ∼ Nn (0, σ 2 I), =e1 , · · · , en ƒpÕá§äkƒÓ©ÙN (0, σ 2 ). e|^ é β0 , β1 , · · · , βp−1 ŠÑ O§ Oþ• β0 , β1 , · · · , βp−1 §K Y = β0 + β1 X1 + · · · + βp−1 Xp−1 (7.1.9) ¡•² £8•§§§´Äý £ã Y Ú X1 , X2 , · · · , Xp−1 ƒm 'X§„I‡?1u " n!Œz•‚5 . œ/ k .L¡þ´š‚5 § ´²L· C†§Œ±z•‚5 .§žwe~µ 2

3 . ~7.1.2 3Í¶ ²LÆ Cobb-Duglas ) ¼ê•µ Qt = aLbt Ktc , (7.1.10) Ù¥ Qt !Lt Ú Kt ©OL«• t c Š!NåÝ\Ú]7Ý\§a§b§c •ëê"L¡þ ´(7.1.10)´š‚5'X§eòü> éê ln Qt = ln a + b ln Lt + c ln Kt , - ln Lt = Xt1 , ln Kt = Xt2 , yt = ln Qt , β0 = ln a, β1 = b, β2 = c§Kk yt = β0 + β1 Xt1 + β2 Xt2 + et , t = 1, 2, · · · , T. (7.1.11) ùÒ=z¤‚5 . /ª" ~7.1.3 õ‘ª£8§ÏCþ Y ÚgCþ X ƒmäke 'X Y = β0 + β1 X + β2 X 2 + · · · + βk X k + e, ù´˜‡ k gõ‘ª§e-X1 = X, X2 = X 2 , · · · , Xk = X k §K Y = β0 + β1 X1 + β2 X2 + · · · + βk Xk + e, ÒC•˜‡‚5 . /ª" 5µ/£80˜c d5µ=I)ÔÚOÆ[Galton3ïÄ<a¢D¯KžJÑ/Regression0 ˜c§¦Â81078éIf pêâ§^ X )IŠ p§Y ) f p§ü µ=€"r(xi , yi )I3 † ‹I’þ§Œ—¤˜†‚§Ù5ÆŒ—µ(1) IŠ p X O\ž§ f p Y •O\§ù† ~£˜—¶(2) áup‡f @aIŠ f ²þ p‡'IŠ ²þ p$§‡ƒáuL ‡f@aIŠ f ²þ p‡'IŠ p"=‡N ˜‡y–µ p‡L²þpÝ(1078‡ IŠ²þ p) x = 68 =€ §¦‚ f ²þ pò$uIŠ p¶‡ƒ p$u²þ pÝ x = 68 =€ f ²þ p‡puIŠ ²þ p"Golton)ºµŒg,k˜« åå§ < p•¥mŠ/£80§Ø¬ü4©z"ùÒ´¤¢ £8 A" o!A^ é£8 .¤?1 ÚO©Û§Ï~¡•£8©Û"£8©Û ¢SA^8Bå5Ì‡k ±eA‡•¡µ 1. £ãCþƒm 'XµéÑé Y k-‡ƒ''X ÏCþ§ïá£8•§(CþÀJ) u ) ä¤¶ 2. ©ÛCþƒm'XµÏLé£8Xê O§ïá² £8•§ Y = β0 + β1 X1 + · · · + βp−1 Xp−1 . 3

4 .£8Xê βi Oþβˆi (i = 0, 1, · · · , p − 1) Œ 3˜½§Ýþ‡N Xi éYi K• Œ " ,˜•¡§A^˜ ÚO©Û•{§„Œ±©ÛgCþƒm•3 ƒ''X" 3. ýÿµ:ýÿ!«mýÿ" §7.2 eZý •£ ˜!þŠ•þ† • ½Â7.2.1 X = (X1 , X2 , · · · , Xn ) •‘Å•þ§K¡ E(X) = E(X1 ), E(X2 ), · · · , E(Xn ) •‘Å•þ X þŠ•þ§¡ n × n é¡ Cov(X) = E[(X − EX)(X − EX) ] = Cov(Xi , Xj ) n×n •‘Å•þ X • §Ù¥ Cov(Xi , Xj ) = E(Xi − EXi )(Xj − EXj ). i = j ž§Cov(Xi , Xi ) = V ar(Xi ). ½n7.2.1 X Ú b ©O• n × 1 ‘Ú m × 1 ‘ ‘Å•þ§A ´ m × n š‘ÅÝ § P Y = AX + b§K E(Y ) = E(AX + b) = AE(X) + E(b). y²µ A = (aij )m×n , b = (b1 , b2 , · · · , bm ) , X = (X1 , X2 , · · · , Xn ) , Y = (Y1 , Y2 , · · · , Ym ) , Kd Y = AX + b Œ• n Yi = aij Xj + bi j=1 n E(Yi ) = aij E(Xi ) + E(bi ), i = 1, 2, · · · , m. j=1 n íØ7.2.1 tr[Cov(X)] = V ar(Xi ), d?trA I£• A ," i=1 ½n7.2.2 Xn×1 •‘Å•þ§Kk Cov(X) = Cov(Xi , Xj ) n×n ≥ 0. y²µ c •?˜š‘Å•þ§U½Â•‡y² c Cov(X)c ≥ 0. P Y = c X §K V ar(Y ) = V ar(c X) = E[c X − c E(X)]2 = E[c X − E(X) X − EX c] = c E[(X − EX)(X − EX) ]c = c Cov(X)c ≥ 0, 4

5 . • Cov(X) ≥ 0. ½n7.2.3 A•m × n §Xn×1 •‘Å•þ§Y = AX §K Cov(Y ) = ACov(X)A . y²µ Cov(Y ) = E[(AX − AEX)(AX − AEX) ] = AE[(X − EX)(X − EX) ]A = ACov(X)A . ½n7.2.4 X Ú Y ©O• n × 1 ‘Ú m × 1 ‘ ‘Å•þ§Ap×n Ú Bq×m •~ê § K Cov(AX, BY ) = ACov(X, Y )B . y²µ l½ÂÑu" !‘Å•þ g. ½Â7.2.2 Xn×1 = (X1 , X2 , · · · , Xn ) • n × 1 ‘‘Å•þ§A = (aij ) • n × n é¡ §K n n X AX = aij Xi Xj i=1 j=1 ¡•‘Å•þ X g." XÛ¦ g. þŠ!• , ·‚keã½nµ ½n7.2.5 ‘Å•þ Xn×1 k E(X) = µn×1 , Cov(X) = Σn×n §K E(X AX) = µ Aµ + tr(AΣ). y²µ X AX = [(X − µ) + µ] A[(X − µ) + µ] = (X − µ) A(X − µ) + 2µ A(X − µ) + µ Aµ, du E[µ A(X − µ)] = µ AE(X − µ) = 0, k E(X AX) = E[(X − µ) A(X − µ)] + µ Aµ = E tr A(X − µ)(X − µ) + µ Aµ = trAE[(X − µ)(X − µ) ] + µ Aµ = tr[ACov(X)] + µ Aµ = tr(AΣ) + µ Aµ. AOµ (1) µ = 0 ž§E(X AX) = trAΣ¶ (2) Σ = σ 2 I ž§E(X AX) = µ Aµ + σ 2 trA¶ (3) µ = 0, Σ = I ž§E(X AX) = trA. ~7.2.1 ‘ÅCþ X •˜‘oN§E(X) = µ, V ar(X) = D(X) = σ 2 , X1 , X2 , · · · , Xn • n 1 ldoN¥Ä §¦ E(S )§Ù¥ S = 2 2 (Xi − X)2 . n − 1 i=1 5

6 . n )µ ò Q = (Xi − X)2 = (n − 1)S 2 L«¤ X = (X1 , X2 , · · · , Xn ) g.§P✶n •¤ i=1 k ƒ •1 n‘•þ§KE(X) = µ✶n , Cov(X) = σ 2 In . d n 1 1 X= Xi = ✶X n i=1 n 1 1 X − ✶n X = X − ✶n ✶n X = In − ✶n ✶n X = CX, n n Ù¥C = In − n1 ✶n ✶n , ù´˜‡é¡˜ ,=C 2 = C, C = C. l k n Q= (Xi − X)2 = (X − X ✶n ) (X − X ✶n ) i=1 = (CX) CX = X C 2 X = X CX, d½n7.2.1Œ E(Q) = E(X CX) 1 = µ2 ✶ C ✶ + σ 2 tr In − ✶✶ n = 0 + σ 2 (n − 1) = σ 2 (n − 1), l 1 E(S 2 ) = E(Q) = σ 2 . n−1 n!õ ©Ù 1. ½Âµ d˜ Ú ©Ù ½ÂN´í2 ˜„ œ/§ e õ ©Ù ½ Â" ½Â7.2.3 n ‘Å•þ X = (X1 , X2 , · · · , Xn ) äk—Ý¼ê n 1 1 f (x) = (2π)− 2 |Σ|− 2 exp − (x − µ) Σ−1 (x − µ) , (7.2.1) 2 Ù¥ x = (x1 , · · · , xn ) , −∞ < xi < ∞, µ = (µ1 , · · · , µn ) , Σn×n > 0 • ½ §K¡‘Å• þX ©Ù• n ©Ù§P• X ∼ Nn (µ, Σ). N´ yµ ∞ ∞ (1) f (x) ´—Ý§= f (x) > 0§… ··· f (x1 , · · · , xn ) dx1 · · · dxn = 1¶ −∞ −∞ (2) E(X) = µ, Cov(X) = Σ. 1 1 ∂(x1 ,··· ,xn ) 1 y² (1)ŠC† Y = Σ− 2 (X − µ)§K X = Σ 2 Y + µ§|J| = ∂(y1 ,··· ,yn = |Σ| 2 §l n n 1 −n 1 yi g(y) = f (Σ y + µ) · |J| = (2π) 2 2 −y y/2 e = √ e− 2 = f (yi ). i=1 2π i=1 6

7 .d?f (yi )´IO —Ý¼ê§ÏdY n‡©þ éÜ—Ý uz‡©þ —Ý ¦È"u ´Y n‡©þƒpÕá§…Yi ∼ N (0, 1), i = 1, · · · , n. Ï 1 1 1 1 E(Y) = 0, Cov(Y) = Σ− 2 Cov(X) Σ− 2 = Σ− 2 ΣΣ− 2 = I, 1 l d X = Σ 2 y + µ Œ• 1 E(X) = µ, Cov(X) = Cov(Σ 2 Y) = Σ. 2. õ ©Ù 5Ÿ ò•þXn×1 Úµn×1 ‰ƒA ©¬ X(1) µ(1) Xn×1 = , µn×1 = (7.2.2) X(2) µ(2) Ù¥X(1) , µ(1) •p × 1•þ¶X(2) , µ(2) þ• q × 1 •þ, p + q = n. òX • Σ kXe ©¬é /ª Σ11 Σ12 Σ= , (7.2.3) Σ21 Σ22 ùpΣ11 •p × 1 f• " ½n7.2.6 (1) X ∼ Nn (µ, Σ), …X, µÚΣ ©¬©Od(7.2.2)Ú(7.2.3)‰Ñ§Ù¥Σ12 = 0, Σ21 = 0, K X(1) Ú X(2) ƒpÕá§… X(i) ∼ N (µ(i) , Σii ), i = 1, 2. £2¤ AOe Σ = σ 2 I, X = (X1 , · · · , Xn ) , µ = (µ1 , · · · , µn ) , KXi ∼ N (µi , σ 2 ), i = 1, 2, · · · , n. y² (1) duX ∼ Nn (µ, Σ),…Ù • Σ k©¬é /ª(7.2.3), N´òX —Ý¼ ê©)•Xe/ª f (x) = f (x(1) )f (x(2) ) Ù¥f (x(1) )Úf (x(2) )©O•X(1) ∼ N (µ(1) , Σ11 )ÚX(2) ∼ N (µ(2) , Σ22 ). —Ý¼ê" (2) X ∼ Nn (µ, Σ)§KÙA ¼ê(c.f.)• n 1 1 2 2 ϕ(t) = ϕ(t1 , · · · , tn ) = E(eit X ) = eit µ− 2 t Σt = eitj µj − 2 tj σ . j=1 'u ‘Å•þ ‚5C† 5ke (Jµ ½n7.2.7 X ∼ Nn (µ, Σ), A • n × n Œ_~ê §b • n × 1 ~•þ§P Y = AX + b§K Y ∼ Nn (Aµ + b, AΣA ). 7

8 . y² ^A ¼êy²" E(eit Y )) = E(eit (AX+b) ) = eit b · E(eit AX ) (-t A = t ) 1 = eit b · E(eit X ) = eit b · eit µ− 2 t Σt 1 1 = eit b · eit Aµ− 2 t AΣA t = eit (Aµ+b)− 2 t AΣA t , = Y ∼ N (Aµ + b, AΣA ). 1 1 íØ7.2.2 X ∼ Nn (µ, Σ)§K Y = Σ− 2 X ∼ Np (Σ− 2 µ, I). 'u ‘Å•þ > ©Ùke (Jµ ½n7.2.8 X ∼ Nn (µ, Σ)§X, µ, Σ ©¬/ªX(7.2.2) Ú(7.2.3) ¤«§K X(1) ∼ Np (µ(1) , Σ11 ). Ón X(2) ∼ Nq (µ(2) , Σ22 ). d?p + q = n. y² 3½n7.2.7¥ Ip 0 A= , b = 0, −Σ21 Σ−1 11 Iq K Y = AX ∼ N (Aµ, AΣA )§dž X(1) µ(1) Y = AX = , Aµ = , X(2) − Σ21 Σ−1 11 X(1) µ(2) − Σ21 Σ−1 11 µ(1) Σ11 0 AΣA = , 0 Σ22·1 Ù¥ Σ22·1 = Σ22 − Σ21 Σ−1 11 Σ12 §l Y(1) µ(1) Σ11 0 Y= = AX ∼ Nn , , Y(2) ∗ 0 Σ22·1 d½n7.2.6Œ•µY(1) = X(1) ∼ Np (µ(1) , Σ11 ). 5 3þãy²¥§e Ip −Σ−1 11 Σ12 A= 0 Iq ^aq•{Œy X(2) ∼ Nq (µ(2) , Σ22 ). ½n7.2.7„Œ±?˜Úí2§¼ Xe(Jµ ½n7.2.9 Am×n ~ê §R(A) = m < n, K Ym×1 = AX ∼ Nm (Aµ, AΣA ). y²µ Ï Am×n •• m§3 n ‘‚5˜m•3 n − m ‡•þ† Am×n 1•þ©å5 A ¤ Rn ˜|Ä•þ§Pù n − m ‡1•þÝ • B(n−m)×n §P Cn×n = , C •n×nŒ B _ §Z = CX ∼ N (Cµ, CΣC ), l A AX Z1 Aµ AΣA AΣB Z = CX = X= ∼ Nn , , B BX Z2 Bµ BΣA BΣB 8

9 íØ7.2.3 3½n7.2.9¥§e Cþ ‚5|ÜE• 'u Cþ ½n7.2.10 y² Cov(AX, þ§ Øƒ'†Õá o! Cþ g. ©Ù 1. χ2n ©Ù ½Â95Ÿ Ñ§„5ênÚO6§2.4. 2. õ Cþ ½n7.2.11 (1) −1 K (X − µ) Σ (X − µ) ∼ χ2n . (2) X ∼ . X AX ∼ χ2r . y² (1) 1, 2, · · · , n§l (2) é¡˜  λ1   ..    Q AQ =       d= Ïd Ù¥ Y = Q X ∼ Nn (0, I)§l íØ7.2.4 (1) (2) e X ∼ .d½n7.2.8Œ•µZ1 = AX ∼ Nm (Aµ, AΣA ). C •˜‡1•þ= C = c§K cn×1 X ∼ N (c µ, c Σc)§=˜ " ü‡‚5. Õá5ke (J X ∼ Nn (µ, Σ)§K AΣB = 0 ž§AX Ú BX Õá" BX) = ACov(X)B = AΣB = 0§ AX Ú BX Øƒ'§du§‚´ C d§Ïd AX Ú BX Õá" g.Ñl χ2 ©Ù O•{ X ∼ Nn (0, Σ), Σ > 0 ( ½)§K X Σ−1 X ∼ χ2n . X ∼ N (µ, Σ)§ Nn (0, I)§An •é¡ §R(A) = r > 0§K A •˜ , =A2 = A)ž§ g 1 P Y = Σ− 2 X §K Y = (Y1 , Y2 , · · · , Yn ) ∼ Nn (0, I) ⇐⇒ Yi ∼ N (0, 1), i = n X Σ−1 X = Y Y = Yi2 ∼ χ2n . i=1 A Šš 0 = 1§=•3 Qn×n ¦ µ    1      ..   .   .     λr   1     =  0   0     ..   ..  .   .     0 0 Ir 0 A=Q Q QΛQ , 0 0 r X AX = X QΛQ X = Y ΛY = Yi2 , i=1 r X AX = Yi2 ∼ χ2r . i=1 eX ∼ Nn (µ, I), A2 = A, A = A, R(A) = r§K (X − µ) A(X − µ) ∼ χ2r . Nn (µ, Σ)§A é¡§R(A) = r§… AΣA = A§K (X − µ) A(X − µ) ∼ χ2r . 9

10 .3. Cþ ü‡ g.! g.†‚5. Õá5 'u˜‡ g.˜‡‚5. Õá5ke (Jµ ½n7.2.12 X ∼ Nn (0, I) §eB •m × n §A•n × né¡ §eBA = 0 §K BX †X AX Õá" y² du A = A §- R(A) = r, A é¡§ •3 Qn×n ¦   λ1    ..   .     λr    Q AQ =   = Λr , d= A = QΛr Q ,  0     ..   .    0 P Y = Q X, KY ∼ Nn (0, I)§Ïdk r X AX = X QΛr Q X = Y ΛY = λi Yi2 = Y1 Λr Y1 , Yi ∼ N (0, 1), i = 1, 2, · · · , r, i=1 ¯K=z• BA = 0 ž§BX = BQQ X = DY §¯ DY † Y Λr Y ´ÄÕáº du X ∼ Nn (0, I) =⇒ QX = Y ∼ N (0, In )§= Y1 , . . . , Yn iid ∼ N (0, 1)§K 0 = BA = BQQ AQQ = DΛr Q =⇒ DΛr = 0. ò D ©¬• D = ( D1 | D2 )§Ù¥ D1 ê• m × r§ D2 ê• m × (n − r)§K Λr 0 0= D1 D2 = D1 Λr 0 =⇒ D1 Λr = 0, 0 0 q Λr Œ_ =⇒ D1 = 0. ÏdD = ( 0 | D2 )§l     Y1 Yr+1 Y(1)  .   .  BX = DY = 0 D2 = D2 Y2 , Ù¥Y(1) = .   .  .  , Y(2) =  . ,  Y(2) Yr Yn du Y1 , · · · , Yr † Yr+1 , · · · , Yn Õá§BX = DY = D2 Y2 •† Yr+1 , · · · , Yn k'§X AX = r Y Λr Y = λi Yi2 •† Y1 , · · · , Yr k'§ öÕá" i=1 íØ7.2.5 X ∼ N (0, Σ) , K BΣA = 0 ž, BX † X AX Õá" − 21 y²µ P Y = Σ X ∼ Nn (0, I)§K 1 1 1 BX = BΣ 2 Y = BY, X AX = Y Σ 2 AΣ 2 Y = Y AY, 1 1 1 d?A = Σ 2 AΣ 2 , B = BΣ 2 . d½n7.2.12 (JŒ•µ B A = 0 ž, öÕá" 1 1 1 1 B A = BΣ 2 · Σ 2 AΣ 2 = BΣAΣ 2 = 0 ⇐⇒ BΣA = 0 10

11 .=ù˜^‡¤áž, BX † X AX Õá" 'uü‡ Cþ g. Õá5§ke (Jµ ½n7.2.13 X ∼ Nn (0, I), A Ú B •é¡ ,… AB = 0 ,K g. X AX Ú X BX Õá" y²µdAB = 0 Œ• B A = BA = 0, = AB = BA = 0 Œ †§Ïd•3ú Q§ ¦ A, B Óžé z§=     λ1 µ1  ..   ..  Q AQ = Λ =   .  , Q BQ = ∆ =    . ,  λn µn Ù¥ λ1 , · · · , λn Ú µ1 , · · · , µn ©O• A Ú B A Š" - Y = Q X ,K Y ∼ Nn (0, In )§Ïdk X AX = X QΛQ X = Y ΛY ; X BX = X Q∆Q X = Y ∆Y, du 0 = AB = QΛQ Q∆Q = QΛ∆Q ⇐⇒ Λ∆ = 0 •Λ Ú ∆ ¥ é ‚þ öš"A  Š´†m §=  λ1 0      ..   ..   .   .       λr   0      eΛ= §K ∆ =  §… s ≥ r,  0   µs+1       .   ..   ..   .      0 µn k r n X AX = Y ΛY = λi Yi2 , X BX = Y ∆Y = µj Yj2 . i=1 j=s+1 d Y1 , · · · , Yr † Yr+1 , · · · , Yn Õá, Ïdk X AX † X BX Õá" íØ7.2.6 X ∼ N (0, Σ) , e AΣB = 0 , K X AX † X BX Õá" 1 y²µ P Y = Σ− 2 X ∼ N (0, In )§K 1 1 1 1 X AX = Y Σ 2 AΣ 2 Y = Y AY, X BX = Y Σ 2 BΣ 2 Y = Y BY, 1 1 1 1 d?A = Σ 2 AΣ 2 , B = Σ 2 BΣ 2 . l AB = 0 ⇐⇒ AΣB = 0. 11

12 . §7.3 £8Xê LS O95Ÿ ˜! . Y •ÏCþ§é Y kK• gCþk p − 1 ‡§X1 , · · · , Xp−1 §§‚ƒmk‚5'X Y = β0 + β1 X1 + · · · + βp−1 Xp−1 + e, (7.3.1) e •‘ÅØ §β0 , β1 , · · · , βp−1 •™•£8ëê§β0 ¡•~ê‘§β1 , · · · , βp−1 ¡•£8Xê" (X1 , · · · , Xp−1 , Y ) n |* Š (xi1 , · · · , xip−1 , yi ), i = 1, 2, · · · , n§Kk yi = β0 + β1 xi1 + · · · + βp−1 xip−1 + ei , i = 1, 2, · · · , n. (7.3.2) Ø e1 , · · · , en ÷vGauss-Markov (G-M) b½µ    (a) E(ei ) = 0;  (b) V ar(ei ) = σ 2 ; (7.3.3)    (c) Cov(e , e ) = 0, i = j. i j ò•§|(7.3.2)^Ý L«µ        y1 1 x11 x12 · · · x1,p−1 β0 e1         y2   1 x21 x22 · · · x2,p−1  β1   e2          . = . .. .. .. ..  .. + .. ,  ..   .. . . . .  .   .         yn 1 xn,1 xn,2 · · · xn,p−1 βp−1 en =µ yn×1 = Xn×p βp×1 + en×1 , (7.3.4) d?e÷vG-M b½µ E(e) = 0, Cov(e) = σ 2 I. (7.3.5) d? y • n × 1 * •þ§X • n × p O §β •p × 1£8ëê•þ§e •n × 1 ‘ÅØ • þ§β Ú σ 2 ™•§·‚8 ´¦ β Ú σ 2 O" !LS O 1. β LS O ëê•þ β OkeZØÓ•{§Xk MLE •{Ú• ¦ O(Least Square estima- tion,{¡LS O) "e¡0 LS O§QãXeµ3(7.3.4)¥§P e = y − Xβ §¦ 2 2 e = e e = y − Xβ = min 12

13 .=ˆ • Šž β Š β ¡• LS O§P 2 Q(β) = y − Xβ = (y − Xβ) (y − Xβ) = y y − 2y xβ + β X Xβ, K ∂Q(β) ∂Q(β) = 0 ⇐⇒ = −2X y + 2X Xβ = 0 ∂β ∂β ⇐⇒ X Xβ = X y. (7.3.6) (7.3.6) ¡• K•§§ X X Œ_žk•˜) β = (X X)−1 X y, (7.3.7) ¡β•β LS O" ¯K´)(7.3.7)´Ä¦ Q(β) = y − Xβ 2 = min ºe¡ yƒ" 2 2 y − Xβ = (y − X β) + X(β − β) = (y − X β) + X(β − β) (y − X β) + X(β − β) 2 = y − Xβ +(β − β) X X(β − β) + 2(β − β) X (y − X β) I1 + I2 + 2I3 , d(7.3.6)ª´„ I3 = 2(β − β)(X y − X X β) = 0§Œ„ 2 2 2 y − Xβ = y − Xβ +(β − β) X X(β − β) ≥ y − X β §é˜ƒ β ∈ Rp , Œ• β ¦ Q(β) ˆ 4 " 2. ² £8•§ ò β “\(7.3.1)ª§e ^ÙþŠ 0 “O Y = β0 + β1 X1 + · · · + βp−1 Xp−1 , (7.3.8) ¡•² £8•§§§£ã Y †gCþ X1 , · · · , Xp−1 Cq'X" ~7.3.1 ˜ ‚5£8"Y ´ÏCþ§•k˜‡gCþ X, §‚k‚5'X Y = α + βX + e, yé (X, Y ) Š n g* § êâ (xi , yi ), i = 1, · · · , n§Kk       y1 1 x1 e1        y2   1 x2  α  e2         . = .. ..  +  .  ⇐⇒ yn×1 = Xn×2 β2×1 + en×1 ,  ..   . .  β  ..        yn 1 xn en 13

14 . K•§•   n  yi  n xi α X Xβ = X y ⇐⇒ =  i=1 n ,  xi x2i β xi yi i=1 n n 2 n n xi | X X |= =n x2i − xi =n ¯ )2 , (xi − x xi x2i i=1 i=1 i=1 Ïd 1 x2i − xi (X X)−1 = n , n (xi − ¯ )2 x − xi n i=1 “\ K•§|Œ  n   yi  α 1 x2i − xi  , = (X X)−1 X y = n  i=1 n  β n ¯)2 (xi − x − xi n xi yi i=1 i=1 Ïdβ Úα LS O• n n n 1 xi yi − n xi yi i=1 i=1 i=1 β= n , ¯)2 (xi − x i=1 α = y¯ − β x ¯. l y = α + βx •¤‡ £8•§" 3. σ 2 LS O en×1 = yn×1 − Xβ §ò β ^ β “O í •þ O e = y − X β ⇐⇒ ei = yi − xi β, Ù¥ xi ´ O X 1i1 1•þ"^ e Š• e O§g,Ž ^ n 2 RSS = e =ee= e2i (7.3.9) i=1 Š•ïþ σ 2 Œ Ýþ§RSS ´í ²•Ú§§ Œ ‡N ¢Sêâ†nØêâ l§ Ý§Œ±y² E(RSS) = (n − p)σ §Ïd 2 1 1 1 σ2 = RSS = y − Xβ 2 = y I − X(X X)−1 X y, (7.3.10) n−p n−p n−p • σ2 Ã O§§¡• σ 2 LS O" 14

15 . e¡|^ y ∼ (Xβ, σ 2 I) OŽE(RSS). E(RSS) = E y I − X(X X)−1 X y = (Xβ) (I − PX )Xβ + σ 2 · tr(I − PX ) = 0 + σ 2 (n − p) = (n − p)σ 2 , d? PX = X(X X)−1 X •é¡˜" §´• tr(I − PX ) = R(I − PX ) = n − p. n!‚5£8 . ¥%zÚIOz 1. ¥%z 3£8©Û A^¥§·‚~~I‡r ©*ÿêâ?1¥%zÚIOz§ùé·‚?1 ÚO©Û´kÃ "P n 1 x ¯j = xij , j = 1, · · · , p − 1. n i=1 d?x¯j ´1j‡£8gCþng Š ²þê"ò .(7.3.2) ?1U • yi = β0 + β1 xi1 + · · · + βp−1 xip−1 + ei ¯1 + · · · + βp−1 x = (β0 + β1 x ¯p−1 ) + β1 (xi1 − x ¯1 ) + · · · + βp−1 (xip−1 − x ¯p−1 ) + ei = α + β1 (xi1 − x ¯1 ) + · · · + βp−1 (xip−1 − x ¯p−1 ) + ei , i = 1, · · · , n. (7.3.11) ^Ý L«•µ        y1 x11 − x ¯1 x12 − x ¯2 · · · x1,p−1 − x ¯p−1 β1 e1         y2   x21 − x ¯1 x22 − x ¯2 · · · x2,p−1 − x¯p−1  β2   e2          .  = α✶ +  . . .. ..  .. + ..   ..   .. .. . .  .   .         yn xn,1 − x¯1 xn,2 − x¯2 · · · xn,p−1 − x ¯p−1 βp−1 en d= yn×1 = α✶n×1 + Xc β(p−1)×1 + e, (7.3.12) d? α •~ê‘§β = (β1 , · · · , βp−1 ) •£8Xê•þ§Xc • n × (p − 1) §¡•¥%z O "N´ y ✶n Xc = 0. (7.3.13)   x1j − x¯j  ..  n n ù´Ï• ✶   . =  i=1(xij − x ¯j ) = xij − n¯ xj = 0, j = 1, · · · , p − 1. i=1 xnj − x¯j ò(7.3.12)ªU • α α yn×1 = (✶ | Xc ) ˜ +e=X + e, β β 15

16 .Ïd K•§• ˜ X ˜ α ˜ y ⇐⇒ n 0 α ✶y X =X = β 0 Xc Xc β Xc y k   1 α 0 ✶y = n  β 0 (Xc Xc )−1 XC y u´£8ëê LS O•  n  α= 1  yi = y¯ n i=1 (7.3.14)   β = (Xc Xc )−1 Xc y. (Øµ ¥%z .¥~ê‘ α = y¯§ β ŒÀ•d y = Xc β + e U LS •{¦Ñ )" ~7.3.2 ˜ ‚5£8(Y)§^¥%z•{¦ α Ú β LS O" ¥%z ˜ ‚5£ 8• yi = α + (xi − x ¯)β + ei , i = 1, 2, · · · , n. U¥%z•{ Xc = (x1 − x ¯, · · · , xn − x ¯) n (Xc Xc )−1 = 1 ¯ )2 . (xi − x i=1 Uúª(7.3.14) α Úβ LS O• n 1 α = y¯ = yi , n i=1 n (xi − x ¯)yi −1 i=1 β = (Xc Xc ) Xc y = n . ¯)2 (xi − x i=1 l ² £8•§ y = α + β(X − x ¯). 2. IOz ¥%z ƒ §„Œ±UY‰IOzC†" (7.3.12)ª¥•x O 1 j+1 lÑ§ Ý þ• n sj = ¯j )2 , j = 1, 2, · · · , n. (xij − x i=1 -IOzC†•µ xij − x ¯j zij = , i = 1, 2, · · · , n; j = 1, 2, · · · , p − 1. (7.3.15) sj 16

17 .ò¥%zêâ .yi = α + β1 (xi1 − x¯1 )) + · · · + βp−1 (xip−1 − x¯p−1 ) + ei Š?˜ÚU § xi1 − x ¯1 xi2 − x ¯2 xip−1 − x ¯p−1 yi = α + (s1 β1 ) + (s2 β2 ) + · · · + (sp−1 βp−1 ) + ei s1 s2 sp−1 = α + zi1 β1oyi+=zαi2+ β2oβ1+(x·i1· ·−+x¯z1 )) ip−1 o + ·β·p−1 · + β+ ei(x p−1 , ip−1 i =−1, x 2, ·)·+ ¯p−1 · ,en. i xi1 − x ¯1 xi2 − x ¯2 xip−1 − x ¯p−1 Ù¥βio = si βiy, i i==α 1, + · · · , p −(s 1.1 βÙÝ 1) + L«• (s2 β2 ) + · · · + (sp−1 βp−1 ) + ei s1 s2 sp−1 = α + zi1 β1o + zi2 β2o + · · · + zip−1 βp−1 o + ei , i = 1, 2, · · · , n. yn×1 = α✶n + Zβ o + e. βio = si βi , i = 1, · · · , p − 1. Ù¥Z = (zij )•n × (p − 1) Ý , zij d(7.3.15)‰Ñ"w,džEk yn×1 = α✶n + Zβ o + e. (a) ✶n Z = 0; n 1 (b) ZZ =Z(z=ij )R = n× ij )−• (r(p 1)(p − 1) , zij× (p − 1) • (7.3.15) §Ù¥rij = si sj (xki − x¯i )(xkj − x¯j )• k=1 ƒ'Xê§ ✶n Z =rjj0; = 1. (a) AO n 1 (b) Z Z = R ·‚éù‡ .¦ (p − 1)LS = (rij ) ëê × (p − O1) rij = si sj (xki − x¯i )(xkj − x¯j ) k=1 rjj = 1. = y¯, β1o , · · · , βp−1 αLS o ƒ §K² £8•§• α = y¯, β1o , · · · , βp−1 o X1 − x ¯1 o Xp−1 − x ¯p−1 o y =α+ β1 + · · · + βp−1 s1 sp−1 X1 − x ¯1 o Xp−1 − x ¯p−1 o y = α +p−1 + · · · +o β p−1 βp−1 ¯si1 o 1 x βi sp−1 = α− β + Xi . p−1 i=1 six¯i i p−1 i=1 βsioi = α− βio + Xi . i=1 si i=1 si ~7.3.3 ˜‡Á Nì‚ ðøA9þ§¦Ù ±ð§§eL¥gCþ X L«Nì±Œ˜ 7.3.3 X í¥ü žm²þ§Ý(e)§Y L«ü žmžÑ ðþ(L)" k25|êâ§¦² £8•§" ( ) Y (L) 25 SÒ Y (L) X(e) SÒ Y (L) X(e) Y (L) X( ) Y (L) X( ) 1 10.98 35.3 14 9.57 39.1 1 10.98 35.3 14 9.57 39.1 2 211.1311.1329.729.7 15 15 10.94 10.94 46.8 46.8 3 312.5112.5130.830.8 16 16 9.58 9.58 48.5 48.5 4 48.40 8.40 58.858.8 17 17 10.09 10.09 59.3 59.3 5 59.27 9.27 61.461.4 18 18 8.11 8.11 70.0 70.0 6 68.73 8.73 71.371.3 19 19 6.83 6.83 70.0 70.0 7 6.36 74.4 20 8.88 74.5 7 6.36 74.4 20 8.88 74.5 8 8.50 76.7 21 7.68 72.1 8 8.50 76.7 21 7.68 72.1 9 7.82 70.7 22 8.47 58.1 9 7.82 70.7 22 8.47 58.1 10 9.14 57.5 23 8.86 44.6 10 119.14 8.24 57.546.4 23 24 8.86 10.36 44.6 33.4 11 128.24 12.1946.428.9 24 25 10.36 11.08 33.4 28.6 12 12.1911.8828.928.1 13 25 11.08 28.6 13 11.88 28.1 17 17

18 . ð†§Ý 'X )µ ¥%z•{dêâ ¯ = 52.60§l y¯ = 9.424, x ~ê‘ α Ú£8Xê β • ¦ O©O•µ α = y¯ = 9.424, β = −0.079 8. u´² £8•§• Y = 9.424 − 0.079 8(X − 52.60), = Y = 13.623 − 0.079 8X. o!LS O 5Ÿ 1. £8ëê β LS O Ý ½n7.3.1 β•β LS O§Kk E(β) = β, Cov(β) = σ 2 (X X)−1 . y² d β = (X X)−1 X y Œ• E(β) = (X X)−1 X E(Y ) = (X X)−1 X Xβ = β, Cov(β) = Cov (X X)−1 X y = (X X)−1 X Cov(Y )X(X X)−1 = σ 2 (X X)−1 X X(X X)−1 = σ 2 (X X)−1 . 2. Gauss-Markov ½n c • p × 1 ‘~ê•þ§éu‚5¼ê c β §¡ c β • c β LS O"Kd½n7.3.1Œ• E(c β) = c β, Cov(c β) = σ 2 c (X X)−1 c, (7.3.16) c β ¡•Œ ¼ê" ½n7.3.2 éu‚5£8 . y = Xβ + e, e ∼ (0, σ 2 I)§Œ ¼ê c β ¤k‚5Ã O¥§LS O c β ´•˜äk• • ö" y²µ ay •cβ ?˜Ã O§u´ E(a y) = a Xβ = c β, dªé˜ƒ βp×1 ∈ Rp Ñ¤á§Óž7k aX=c. (7.3.17) 18

19 .·‚‡y² V ar(c β) ≤ V ar(a y)§é˜ƒ β ∈ Rp ¤á" V ar(a y) = a Cov(y)a = σ 2 a a, V ar(c β) = σ 2 c (X X)−1 c, K V ar(a y) − V ar(c β) = σ 2 a a − σ 2 c (X X)c = σ 2 a a − a X(X X)−1 X a = σ 2 a I − X(X X)−1 X a, ù duy²eª¤áµ I − X(X X)−1 X ≥ 0, I − X(X X)−1 X •é¡˜ §ÙA ŠšK§ÏdI − X(X X)−1 X ≥ 0, k V ar(a y) − V ar(c β) ≥ 0, ˜ƒ β ∈ Rp ¤á§… Ò¤á ⇐⇒ I − X(X X)−1 X a = 0§= a = X(X X)−1 c§•Ò´ a y = c β §•˜5 y" 3. ‚5 .e§β Ú σ 2 LS O `û5 ½n7.3.3 éc¡ ‚5 . y = Xβ + e§e e ∼ N (0, σ 2 I)§K (a) β ∼ N (β, σ 2 (X X)−1 ); (b) RSS/σ 2 ∼ χ2n−p ; (c) β † σ 2 ƒpÕá" y²µ (a) w," (b) -Z = (y − Xβ)/σ, KZn×1 ∼ N (0, I). dd¯¢Ú½n7.2.11Œ• RSS y (I − PX )y y − Xβ y − Xβ = = (I − PX ) = Z (I − PX )Z ∼ χ2m−p . σ2 σ2 σ σ (c) òRSS Úβ L«¤Y g.Ú‚5.µ RSS = y (I − PX )y = y Ay, y ∼ N (Xβ, σ 2 I), β = (X X)−1 X y = By, |^ g.Ú‚5.Õá5 ½½nŒ•§ öƒpÕá ⇐⇒ BA = 0, BA = (X X)−1 X (I − PX ) = (X X)−1 X − (X X)−1 X X(X X)−1 X = (X X)−1 X − (X X)−1 X = 0, 19

20 . Õá" 4. ¥%z£8 . β Ú σ2 LS O `û5 é¥%z £8 .µ y = α✶n + Xc β + e, ÙLS O•α = y¯, β = (Xc Xc )−1 Xc y, Kk (a) E(α) = α, E(β) =β,  1 α 0 (b) Cov = σ2  n  β 0 (Xc Xc )−1 (c) ?˜Úb½ e ∼ N (0, σ 2 I), Kk 1 2 α ∼ N1 α, σ , n β ∼ Np−1 β, σ 2 (Xc Xc )−1 , … α Ú β ƒpÕá" 5. Eƒ'Xê n P SS£ = β XC y ¡•£8²•Ú§SSo = (yi − y¯)2 ¡•? o²•Ú§½Â i=1 SS£ R2 = , SSo ¡• ½Xê§§ ½£8gCþ X1 , · · · , Xp−1 †ÏCþ y [Ü§Ý"w, 0 ≤ R2 ≤ 1"R ¡•Eƒ'Xê §R2 Œ`² Y †Ã Xi k Œ ƒ''X" ~7.3.4 Šâ² §3< pƒ ^‡§ÙÉØÂ Ø y †N- x1 !c# x2 k'§Â8 13<êâ"Áïá y † x1 , x2 ² £8•§" SÒ xi1 xi2 yi SÒ xi1 xi2 yi 1 152 50 120 8 158 50 125 2 183 20 141 9 170 40 132 3 171 20 124 10 153 55 123 4 165 30 126 11 164 40 132 5 158 30 117 12 190 40 155 6 161 50 125 13 185 20 147 7 149 60 123 )µ |^¥%z . yi = α + (xi1 − x ¯1 )β1 + (xi2 − x ¯2 )β2 + ei , i = 1, 2, · · · , 13. dþL¥êâOŽŒ 13 1 x ¯1 = xi1 = 166.8 13 i=1 20

21 . 13 1 x ¯2 = xi2 = 38.85 13 i=1 13 1 y¯ = yi = 130 13 i=1 ¥%z OÝ •   −14.08 11.15    16.92 −18.85     4.92 −18.85       −1.08 −8.85     −8.08 −8.85       −5.08 11.15      XC =  −17.08 21.15     −8.08 11.15       3.92 1.15     −13.08 16.15       −2.08 1.15       23.92 1.15  18.92 −18.85 ¥%z K•§ Xc Xc β˜ = Xc y • 2 078.92 β1 − 1 533.85 β2 = 1 607.00 −1 533.85 β1 + 2 307.69 β2 = −715.00 ) β1 = 1.068, β2 = 0.400. q α = y¯ = 130, ² £8•§• Y = α + (x1 − x ¯1 )β1 + (x2 − x ¯2 )β2 = 130 + 1.068(x1 − 166.8) + 0.400(x2 − 38.85) = −62.963 + 1.068x1 + 0.400x2 . ·‚„ŒOŽÑ ½Xê SS£ = β XC y = 1 430.276, 13 SSo = (yi − y¯)2 = 1 512, i=1 21

22 . k SS£ 1 430.276 R2 = = = 0.945 9, R = 0.972 6. SSo 1 512 Œ„ ‚5£8[Ü ûÐ" §7.4 å• ¦ O ˜! . ke ‚5 . yn×1 = Xn×p βp×1 + en×1 §e ∼ (0, σ 2 I)§Ù¥ëê β Ú σ 2 LS O 9Ù5Ÿ®3 §7.3 ¥0 L"3b u ¯KÚ˜ ¢S¯K¥§·‚~I‡¦‘ å^‡ LS O¯K§=k å^‡ Aβ = b (7.4.1) Ù¥ A • k × p Ý §b • k × 1 ‘•þ(k ≤ p)§A!b ®•§R(A) = k§…(7.4.1)ª•˜‡ ƒN•§|^‡(7.4.1) ˜‡~f´3£8 .ëê b u ¯K¥~¦e u µ H0 : β1 = β2 = · · · = βp−1 = 0. =£8•§ wÍ5u "òþª^Ý L«         β1 − βp−1 = 0 0 1 0 ··· 0 −1 β0 0               β2 − βp−1 = 0  0 0 1 ··· 0 −1  β1   0  H0 : .. ⇐⇒ H0 :  .. .. .. .. .. ..  .. = .. .   .  . . . . . .  .   .           βp−2 − βp−1 = 0 0 0 0 ··· 1 −1 βp−1 0 d= H0 : Aβ = 0§d?       0 1 0 ··· 0 −1 β0 0        0 0 1 ··· 0 −1   β1   0        A= . . . . .. .. , β =  .. , b =  .. .  .. .. .. .. . .   .   .        0 0 0 ··· 1 −1 βp−1 0 Ïdk å ‚5£8 .• yn×1 = Xn×p βp×1 + en×1 , e ∼ (o, σ 2 I) (7.4.2) Aβ = b, Ù¥ R(A) = k, =1÷•. ! å LS ) ½n7.4.1 é å ‚5£8 .(7.4.2)§β k å LS )• −1 βc = β − (X X)−1 A A(X X)−1 A (Aβ − b), 22

23 .Ù¥ β = (X X)−1 X y ´ §7.3.1 ¥¦Ñ β Ã å LS )" y²µ ò å^‡(7.4.1)©)§P     a ˜1 b1  .   .  A= .   . , b= .   . , a ˜k bk K å^‡(7.4.1)ª du     a ˜1 β b1  .   .   .  =  .  §= a ˜i β = bi , i = 1, 2, · · · , k. (7.4.3)  .   .  a ˜k β bk - Q(β) = y − Xβ 2 §¯K=z•3(7.4.3) å^‡e¦ Q(β) • Š:§^ Lagrange ¦f { E¼ê§P λ = (λ1 , · · · , λk )§K n 2 F (β, λ) = y − Xβ +2 λi (a˜i β − bi ) i=1 2 = y − Xβ + 2λ (Aβ − b) = (y − Xβ) (y − Xβ) + 2λ (Aβ − b) = y y + β X Xβ − 2y Xβ + 2λ (Aβ − b) ò¼ê F (β, λ) é β ¦ ê§ n¿-§‚ u 0 §   ∂F (β, λ)  = 2X Xβ − 2X y + 2A λ = 0 ∂β  ∂F (β, λ) = 2(Aβ − b) = 0  ∂λ d= X Xβ − X y + A λ = 0 Aβ = b d Œ± βc = (X X)−1 X y − (X X)−1 A λc = β − (X X)−1 A λc , (7.4.4) ò (7.4.4) ª‘\ Aβc = Aβ − A(X X)−1 A λc = b ⇐⇒ A(X X)−1 A λc = Aβ − b, (7.4.5) d R(A) = k Œ•A(X X)−1 A ƒ_•3§ k −1 λc = A(X X)−1 A (Aβ − b). (7.4.6) ò (7.4.6) ª“£ (7.4.4) ªŒ −1 βc = β − (X X)−1 A A(X X)−1 A (Aβ − b). (7.4.7) 23

24 .y3·‚y² βc (¢´‚5 å Aβ = b e β LS O"•d·‚•Iy²Xeü:µ (a) βc ·Ü å•§§= Aβc = b§w,¤á" (b) ‡y²é·Ü (7.4.1) å^‡e ˜ƒ β §Ñk 2 y − Xβ ≥ y − X βc 2 . •d§·‚ò²•Ú y − Xβ 2 Š©)" 2 2 y − Xβ = (y − X βc ) + X(βc − β) 2 = y − X βc + (βc − β) X X(βc − β) + 2(βc − β) X (y − X βc ) = I1 + I2 + 2I3 , Ù¥ I3 = (βc − β) (X y − X X βc ). Ï• −1 X X βc = X X β − (X X)−1 A A(X X)−1 A (Aβ − b) −1 = X X(X X)−1 X y − X X(X X)−1 A A(X X)−1 A (Aβ − b) = X y − A λc , l I3 = (βc − β) A λc = (Aβc − Aβ) λc = (b − b)λc = 0. k y − Xβ 2 = I1 + I2 ≥ y − X βc 2 , ½ny." ~7.4.1 3U©ÿþ¥§éU˜¥n‡( : ¤ n / n‡S θ1 , θ2 , θ3 ?1ÿ þ§ ÿþŠ©O• y1 , y2 , y3 §duÿþ•3Ø §¤±I‡é θ1 , θ2 , θ3 ?1 O§¦ β = (θ1 , θ2 , θ3 ) Oþ" )µ ·‚|^‚5 .L«k' þµ    y1 = θ 1 + e1    y =θ +e y = Xβ + e 2 2 2 ⇐⇒   y3 = θ 3 + e3 Aβ = b    θ1 + θ2 + θ3 = π ( å^‡) d? e1 , e2 , e3 •ÿþØ §…         y1 θ1 1 0 0 e1         y= y2   , β =    θ2  , X =   0 1 0  , e =  e2  , A =    1 1 1 , b = π. y3 θ3 0 0 1 e3 24

25 .5¿ β = (X X)−1 X y = y §d½n 7.4.1 Œ• −1 βc = β − (X X)−1 A A(X X)−1 A (Aβ − b)      −1     y1 1 1 y1           =        y2  −  1   1 1 1 In  1   1 1 1  y2  − π     y3 1 1 y3   y  1  1 3 =  y2   − 3 yi − π ✶3 , i=1 y3 1 =θi = yi − (y1 + y2 + y3 − π), i = 1, 2, 3 •θi k å LS )" 3 §7.5 2Â• ¦£GLS¤ O ˜!Úó c¡?Ø£8 . yn×1 = Xn×p βp×1 + en×1 , e ∼ (0, σ 2 I), 3Nõ¢S¯K¥Cov(e) = σ 2 I ù‡b½™7¤á§~k Cov(e) = σ 2 Σ, Σ > 0 ( ½)§dž XÛ¦ LS )º(Cov(e) = σ 2 Σ L« G-M b½Ø¤á§=§‚ Ø • ŒUØƒ §‘ÅØ ƒmŒU*dƒ')"dž .• yn×1 = Xn×p βp×1 + en×1 , E(e) = 0, Cov(e) = σ 2 Σ, Σ > 0. (7.5.1) •¦ëê Oþ§·‚F"²L· C†§òù˜²w=z¤ §7.3 ¥?ØL œ/"ò (3.5.1) ª − 21 ü>Óž†¦ Σ § 1 1 1 Σ− 2 y = Σ− 2 Xβ + Σ− 2 e = ˜ + e˜, e˜ ∼ (0, σ 2 I). y˜ = Xβ (7.5.2) 1 ˜ = Σ− 12 X, e˜ = Σ− 12 e ∼ (0, σ 2 I). (7.5.2) •˜„ Ù¥ y˜ = Σ− 2 y, X £8 .§d §7.3 Œ• β Ú σ 2 LS O• ˜ X) β ∗ = (X ˜ −1 X ˜ y˜ = (X Σ−1 X)−1 X Σ−1 y y˜−X˜ β∗ 2 ˜ β ∗ ) (˜ y −X (˜ ˜ β∗ ) y −X (y−X β ∗ ) Σ−1 (y−X β ∗ ) (7.5.3) σ∗2 = n−p = n−p = n−p ¡β ∗ Úσ∗2 •β Úσ 2 2Â• ¦ O§P•GLS O" !Ì‡(J 25

26 . ½n7.5.1 é‚5 . (7.5.1) , β Úσ 2 GLS Od(7.5.3) ‰Ñ§Kk (a) E(β ∗ ) = β; (b) Cov(β ∗ ) = σ 2 (X Σ−1 X)−1 ; (c) é ∀cp×1 , c β ∗ • c β BLU O¶ (d) σ∗2 •σ 2 Ã O" y²µ (a) E(β ∗ ) = (X Σ−1 X)−1 X Σ−1 E(y) = (X Σ−1 X)−1 X Σ−1 Xβ = β. (b) † OŽŒ Cov(β ∗ ) = Cov (X Σ−1 X)−1 X Σ−1 y = (X Σ−1 X)−1 X Σ−1 · Cov(y) · Σ−1 X(X Σ−1 X)−1 = σ 2 (X Σ−1 X)−1 X Σ−1 · Σ · Σ−1 X(X Σ−1 X)−1 = σ 2 (X Σ−1 X)−1 . (c) ò .(7.5.1) =†¤(7.5.2), d§7.3 G-M ½nŒ•§c β ∗ • c β BLU O" (d) w," 5µ dž c β ∗ • c β BLU O§ c β = c (X X)−1 X y E´ c β Ã O§Ï• −1 E(c β) = c (X X) X · Xβ = c β §Ïd•Ã O§ Ø2´• • Ã O§Ï•d½ n7.5.1(3) Œ• V ar(c β ∗ ) ≤ V ar(c β). ~3.6.1 2Â LS O•{ü œ/´ Σ = diag σ12 , · · · , σn2 , d? σ12 , σ22 , · · · , σn2 Ø ƒ §¦ β GLS O" )µ GLS O• β ∗ = (X Σ−1 X)−1 X Σ−1 y. P     x ˜1 y1  .   .     ˜i = (xi1 , · · · , xip ), y =  .. X =  ..  , x ,  x ˜n yn Kk    σ1−2 ✵ x ˜1 n  ..  .  x˜i x ˜i X Σ−1 X = (˜ ˜n )  x1 , · · · , x  .  .  =  .  2 , i=1 σ i ✵ σn−2 x ˜n    σ1−2 ✵ y1 n  ..  .  yi x ˜i X Σ−1 y = (˜ ˜n )  x1 , · · · , x  .  .  =  .  , i=1 σi2 ✵ σn−2 yn l n −1 n ∗ −1 −1 −1 x ˜i x ˜i yi x ˜i β = (X Σ X) XΣ y= · , (7.5.4) i=1 σi2 i=1 σi2 lù‡Lˆª·‚Œ±wÑ§þª¥ü‡Úª©O´x˜i x˜i Úyi x˜i \ Ú§¤^ “ ”Ñ´1 σi2 . β ∗ •¡•\ LS O" 26

27 . §7.6 ˜„‚5b u ˜!Úó •Ä ‚5£8 .µ yn×1 = Xn×p βp×1 + en×1 , e ∼ Nn (0, σ 2 I), (7.6.1) Ù¥ R(X) = p§ !?Ø˜„‚5b µ H0 : Aβ = b, (7.6.2) Ù¥AÚβ ©O•m × pÚp × 1 Ý §R(A) = m (=§1÷•). β = (β0 , β1 , · · · , βp−1 ) , ~„ u ¯K§ Œ^ (7.6.2) ªL«"~Xµ 1. u ~ê‘ H0 : β0 = 0, Ku ¯KŒL«•µ H0 : Aβ = 0, Ù¥ A = (1, 0, 0, · · · , 0), b = 0 ; 2. u k ‡£8Xê•" H0 : β1 = · · · = βk = 0, 1 ≤ k < p − 1, Ku ¯KŒL«•µ . . H0 : Aβ = 0, Ù¥ A = (✵k×1 .. Ik .. ✵k×(p−1−k) ) , b = 0; 3. u ¤k£8Xê•" H0 : β1 = · · · = βp−1 = 0, Ku ¯KŒL«•µ . H0 : Aβ = 0, A = (✵(p−1)×1 .. Ip−1 ), b = 0 . e¡ Ñb u gŽµé . (7.6.1) A^ LS {§ β•β LS O§Kí ²•Ú• RSS = (y − X β) (y − X β) = y I − X(X X)−1 X y, (7.6.3) Ù¥ β = (X X)−1 X y. RSS ‡N ¢ êâ† . [Ü§Ý" RSS §[Ü Ð" ò (7.6.1)ª \þ å^‡(7.6.2)§¼ k å^‡ LS O −1 βH = β − (X X)−1 A A(X X)−1 A (Aβ − b), (7.6.4) ƒA í ²•Ú•µ RSSH = (y − X βH ) (y − X β), (7.6.5) êâ†# . y = Xβ + e, Aβ = b ƒm[ÜÐ€^ RSSH 5ïþ"d .¥ β Cz‰Œ (Ï•b Aβ = 0§=b ˜Ü©gCþØåŠ^§XJþãb Ø¤á§= Aβ = 0 ¢S ¤á§Kù Cþ Š^8\Ø ²•Ú§¦ RSSH OŒ)" # . [ÜØ¬'Î Ð§= RSSH ≥ RSS "XJý ëê(¢÷v‚5b (7.6.2)§K\þ åÚØ\ å^‡ Ÿþ ´˜ §ùžéÃ å^‡Úk å .§êâ [Ü¬´˜ §RSSH − RSS A 27

28 . "‡L5§eý ëêØ÷v (7.6.2)ª§= H0 : Aβ = 0 Ø¤á§K RSSH − RSS A Œ§ Ïdáý H0 § ^ (RSSH − RSS) RSS ƒéŒ 5û½ (7.6.2) ª´Ä¤á´Ü· " 5µ RSS OŽúª: RSS = (y − X β) (y − X β) = y y + β X X β − 2β X y = y y + β X y − 2β X y = y y − β X y. !F u ½n7.6.1 éu ‚5£8 . (7.6.1) k RSS (a) ∼ χ2n−p ; σ2 RSSH − RSS (b) e (7.6.2) ª¤á§K ∼ χ2m ; σ2 (c) RSS † RSSH − RSS Õá¶ (d) (7.6.2) ¤áž§ (RSSH − RSS)/m FH = ∼ Fm,n−p , RSS/(n − p) ùp Fm,n L«gdÝ©O• m, n F ©Ù" RSS y y y²µ (a) = I − X(X X)−1 X ∼ χ2n−p 3½n7.3.3¥yL" σ2 σ σ (b) é RSSH ?1©) 2 2 RSSH = y − X βH = y − X β + X(β − βH ) 2 2 = y − Xβ + X(β − βH ) + 2(β − βH ) X (y − X β) 2 RSS + X(β − βH ) + 2I, Ù¥ I = (β − βH ) (X y − X X β) = 0. ù´du β ÷v K•§ X X β = X y § I = 0 ¤á" k RSSH − RSS = (β − βH ) X X(β − βH ). −1 ò β − βH = (X X)−1 A A(X X)−1 A (Aβ − b) “\ þª RSSH − RSS −1 −1 = (Aβ − b) A(X X)−1 A A(X X)−1 X X(X X)−1 A A(X X)−1 A (Aβ − b) −1 −1 = (Aβ − b) A(X X)−1 A · A(X X)−1 A · A(X X)−1 A (Aβ − b) −1 = (Aβ − b) A(X X)−1 A (Aβ − b), (7.6.6) Ïd- Z = Aβ − b§K3H0 ¤áž Z ∼ Nm 0, σ 2 A(X X)−1 A = Nm (0, Σ). 28

29 .Ï• R(A(X X)−1 ) = m, d½n7.2.11Œ• −1 −1 (Aβ − b) A(X X)−1 A (Aβ − b) ZΣ Z= ∼ χ2m . σ2 (c) Ï• −1 RSSH − RSS = (Aβ − b) A(X X)−1 A (Aβ − b), ò Aβ − b = A(X X)−1 X (Xβ + e) − b = (Aβ − b) + A(X X)−1 X e “\þª RSSH − RSS −1 = e X(X X)−1 A + (Aβ − b) A(X X)−1 A A(X X)−1 X e + (Aβ − b) −1 = e X(X X)−1 A A(X X)−1 A A(X X)−1 X e −1 +(Aβ − b) A(X X)−1 A (Aβ − b) −1 +2(Aβ − b) A(X X)−1 A A(X X)−1 X e = e M e + ∆ + 2c e, d? −1 M = X(X X)−1 A A(X X)−1 A A(X X)−1 X , −1 ∆ = (Aβ − b) A(X X)−1 A (Aβ − b), −1 c = (Aβ − b) A(X X)−1 A (X X)−1 X . du ∆ †Ø •þ e Ã'§ Ù´š‘Å "eP N = I − PX , PX = X(X X)−1 X §5¿ N X = 0§K RSS = y (I − PX )y = (Xβ + e) N (Xβ + e) = β X N Xβ + 2e N Xβ + e N e = e N e. Ïd§‡y RSSH − RSS † RSS Õá§•‡y e M e † e N e Õá!c e † e N e Õá=Œ"Ï • e ∼ N (0, σ 2 I)§d §7.3.2 'uü‡ g.Õá5Ú‚5.† g. Õá5 O•{§• I y M N = 0!c N = 0. Šâ M, N Ú c ½Â9 N X = 0§N´ µ −1 M · N = X(X X)−1 A A(X X)−1 A A(X X)−1 X (I − PX ) = 0; −1 c N = (Aβ − b) A(X X)−1 A (X X)−1 X N = 0, RSSH − RSS † RSS Õá" 29

0点赞

0收藏

1下载