“最优“回归方程的选择”
当变量中含有对Y影响不大的变量时,可能因为误差平方和的自由度减小而使方差的估计增大,从而影响回归预测的精度,适当的选择一个变量建立一个最优的回归方程十重要。此处采用逐步回归法。
逐步回归法计算
#水泥热量与四种成分的关系
cement<-data.frame(
X1=c( 7, 1, 11, 11, 7, 11, 3, 1, 2, 21, 1, 11, 10),
X2=c(26, 29, 56, 31, 52, 55, 71, 31, 54, 47, 40, 66, 68),
X3=c( 6, 15, 8, 8, 6, 9, 17, 22, 18, 4, 23, 9, 8),
X4=c(60, 52, 20, 47, 33, 22, 6, 44, 22, 26, 34, 12, 12),
Y =c(78.5, 74.3, 104.3, 87.6, 95.9, 109.2, 102.7, 72.5,
93.1,115.9, 83.8, 113.3, 109.4)
)
lm.sol<-lm(Y ~ X1+X2+X3+X4, data=cement)
summary(lm.sol)
Call:
lm(formula = Y ~ X1 + X2 + X3 + X4, data = cement)
Residuals:
Min 1Q Median 3Q Max
-3.1750 -1.6709 0.2508 1.3783 3.9254
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 62.4054 70.0710 0.891 0.3991
X1 1.5511 0.7448 2.083 0.0708 .
X2 0.5102 0.7238 0.705 0.5009
X3 0.1019 0.7547 0.135 0.8959
X4 -0.1441 0.7091 -0.203 0.8441
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.446 on 8 degrees of freedom
Multiple R-squared: 0.9824, Adjusted R-squared: 0.9736
F-statistic: 111.5 on 4 and 8 DF, p-value: 4.756e-07
#回归系数没有一项通过检测
#用step( )做回归分析
lm.ste<-step(lm.sol)
Start: AIC=26.94
Y ~ X1 + X2 + X3 + X4
Df Sum of Sq RSS AIC
- X3 1 0.1091 47.973 24.974
- X4 1 0.2470 48.111 25.011
- X2 1 2.9725 50.836 25.728
<none> 47.864 26.944
- X1 1 25.9509 73.815 30.576
Step: AIC=24.97
Y ~ X1 + X2 + X4
Df Sum of Sq RSS AIC
<none> 47.97 24.974
- X4 1 9.93 57.90 25.420
- X2 1 26.79 74.76 28.742
- X1 1 820.91 868.88 60.629
用全部变量做回归分析时,AIC值为26.94。接下来显示如果去除X3,则AIC = 24.97,去掉x4则为25.01,去掉X3可以使AIC达到最小,R软件自动去掉x3.
summary(lm.ste)
Call:
lm(formula = Y ~ X1 + X2 + X4, data = cement)
Residuals:
Min 1Q Median 3Q Max
-3.0919 -1.8016 0.2562 1.2818 3.8982
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 71.6483 14.1424 5.066 0.000675 ***
X1 1.4519 0.1170 12.410 5.78e-07 ***
X2 0.4161 0.1856 2.242 0.051687 .
X4 -0.2365 0.1733 -1.365 0.205395
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.309 on 9 degrees of freedom
Multiple R-squared: 0.9823, Adjusted R-squared: 0.9764
F-statistic: 166.8 on 3 and 9 DF, p-value: 3.323e-08
回归系数显著性水平有大提高,但是X2,X2系数检验不理想。从step()可以看出去掉x4,AIC从24.97变为25.42,是增加的最少的,除AIC准则外,残差平方各也是逐步回归的重要指标之一,从直观上看,拟合越好的直线,残差平方和应该最小,去掉x4后,残差平方和上升了9.93,也是最少的。从这两项指标看,应该去掉x4.
lm.opt <- lm(Y ~ X1+X2,data = cement);
summary(lm.opt)
Call:
lm(formula = Y ~ X1 + X2, data = cement)
Residuals:
Min 1Q Median 3Q Max
-2.893 -1.574 -1.302 1.363 4.048
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 52.57735 2.28617 23.00 5.46e-10 ***
X1 1.46831 0.12130 12.11 2.69e-07 ***
X2 0.66225 0.04585 14.44 5.03e-08 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.406 on 10 degrees of freedom
Multiple R-squared: 0.9787, Adjusted R-squared: 0.9744
F-statistic: 229.5 on 2 and 10 DF, p-value: 4.407e-09
这个结果还算满意
Y = 52.58 + 1.46831*X1 + 0.66225*X2
改变step( )中的某些参数,可能得到不同的结果。
lm.ste<-step(lm.sol, trace=0, k=3); lm.ste
Call:
lm(formula = Y ~ X1 + X2, data = cement)
Coefficients:
(Intercept) X1 X2
52.5773 1.4683 0.663
直接去掉X3和X4。
从增加变量的角度考虑逐步回归
lm0<-lm(Y~1, data=cement)
lm.ste<-step(lm0, scope = ~X1+X2+X3+X4, k=4)
Start: AIC=73.44
Y ~ 1
Df Sum of Sq RSS AIC
+ X4 1 1831.90 883.87 62.852
+ X2 1 1809.43 906.34 63.178
+ X1 1 1450.08 1265.69 67.519
+ X3 1 776.36 1939.40 73.067
<none> 2715.76 73.444
Step: AIC=62.85
Y ~ X4
Df Sum of Sq RSS AIC
+ X1 1 809.10 74.76 34.742
+ X3 1 708.13 175.74 45.853
<none> 883.87 62.852
+ X2 1 14.99 868.88 66.629
- X4 1 1831.90 2715.76 73.444
Step: AIC=34.74
Y ~ X4 + X1
Df Sum of Sq RSS AIC
+ X2 1 26.79 47.97 32.974
+ X3 1 23.93 50.84 33.728
<none> 74.76 34.742
- X1 1 809.10 883.87 62.852
- X4 1 1190.92 1265.69 67.519
Step: AIC=32.97
Y ~ X4 + X1 + X2
Df Sum of Sq RSS AIC
- X4 1 9.93 57.90 31.420
<none> 47.97 32.974
- X2 1 26.79 74.76 34.742
+ X3 1 0.11 47.86 36.944
- X1 1 820.91 868.88 66.629
Step: AIC=31.42
Y ~ X1 + X2
Df Sum of Sq RSS AIC
<none> 57.90 31.420
+ X4 1 9.93 47.97 32.974
+ X3 1 9.79 48.11 33.011
- X1 1 848.43 906.34 63.178
- X2 1 1207.78 1265.69 67.519
这里取k4,最后还剩下x1与x2。
在R中,还有两个函数可以做逐步回归,一个是add1( )函数,用于增加变量,一个是drop1( )函数,用于减小变量。事实上,step( )就是使用这两个函数来自动增加和减小变量。
add1(lm0, scope = ~X1+X2+X3+X4, test="F")
Single term additions
Model:
Y ~ 1
Df Sum of Sq RSS AIC F value Pr(>F)
<none> 2715.76 71.444
X1 1 1450.08 1265.69 63.519 12.6025 0.0045520 **
X2 1 1809.43 906.34 59.178 21.9606 0.0006648 ***
X3 1 776.36 1939.40 69.067 4.4034 0.0597623 .
X4 1 1831.90 883.87 58.852 22.7985 0.0005762 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
打算减少变量
drop1(lm.sol, test="F")
Single term deletions
Model:
Y ~ X1 + X2 + X3 + X4
Df Sum of Sq RSS AIC F value Pr(>F)
<none> 47.864 26.944
X1 1 25.9509 73.815 30.576 4.3375 0.07082 .
X2 1 2.9725 50.836 25.728 0.4968 0.50090
X3 1 0.1091 47.973 24.974 0.0182 0.89592
X4 1 0.2470 48.111 25.011 0.0413 0.84407
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
根据每步计算的结果情况,人工选择增加还是去掉某些变量。
数据分析咨询请扫描二维码