Heteroscedasticity

What heteroskedasticity is. Recall that OLS makes the assumption that V j (ε ) = σ2 for all j. That is, the
variance of the error term is constant. (Homoskedasticity). If the error terms do not have constant variance,
they are said to be heteroskedastic. The term means “differing variance” and comes from the Greek
“hetero” ('different') and “skedasis” ('dispersion').

When heteroskedasticity might occur.

1) Errors may increase as the value of an IV increases. For example,

consider a model in which annual family income is the IV and annual family
expenditures on vacations is the DV. Families with low incomes will spend
relatively little on vacations, and the variations in expenditures across such
families will be small. But for families with large incomes, the amount of
discretionary income will be higher. The mean amount spent on vacations
will be higher, and there will also be greater variability among such
families, resulting in heteroskedasticity. Note that, in this example, a high
family income is a necessary but not sufficient condition for large vacation
expenditures. Any time a high value for an IV is a necessary but not sufficient condition for an observation
to have a high value on a DV, heteroskedasticity is likely. Similar examples: Error terms associated with
very large firms might have larger variances than error terms associated with smaller firms. Sales of larger
firms might be more volatile than sales of smaller firms.

2) Errors may also increase as the values of an IV become more extreme in

either direction, e.g.
with attitudes that range from extremely negative to extremely positive. This
will produce
something that looks like an hourglass shape:

3) Measurement error can cause heteroskedasticity. Some respondents might provide more accurate
responses than others. (Note that this problem arises from the violation of another assumption, that
variables are measured without error.)

4) Other model misspecifications can produce heteroskedasticity. For example, it may be that
instead of using Y, you should be using the log of Y. Instead of using X, maybe you should be using X2, or
both X and X2 . Important variables may be omitted from the model. If the model were correctly specified,
you might find that the patterns of heteroskedasticity disappeared.

5) Heteroskedasticity can also occur if there are subpopulation

differences or other interaction effects (e.g. the effect of income on
expenditures differs for whites and blacks). (Again, the problem arises
from violation of the assumption that no such differences exist or have
already been incorporated into the model.) For example, in the following
diagram suppose that Z stands for three different populations. At low
values of X, the regression lines for each population are very close to each
other. As X gets bigger, the regression lines get further and further apart.
This means that the residual values will also get further and further apart.

Consequences of heteroscedasticity

1 Heteroscedasticity does not alter the unbiasedness and consistency properties of OLS estimators.
2 But OLS estimators are no longer of minimum variance or efficient. That is, they are NOT best linear
unbiased estimators (BLUE); they are simply linear unbiased estimators (LUE).
3 As a result, the t and F tests based under the standard assumptions of CLRM may not be reliable, resulting in
erroneous conclusion
4 In the presence of heteroscedasticity, the BLUE estimators are provided by the method of weighted least
squares (WLS).

Presence pf heteroscedasticity: Ubiased – yes, consistent – yes, efficient - yes

Data: Abortion rates in US.

A priori, we would expect ABR to be negatively related to religion, price, laws, picket, education, and positively
related to fund and income. We assume the error term satisfies the standard classical assumptions, including
the assumption of homoscedasticity.

We assume the error term satisfies the standard

classical assumptions, including the assumption of
homoscedasticity.

- For every 1 dollar increase in price there is a corresponding 0.04236 decrease in the percent of
abortion in thousands.
- States with restriction (laws) has less 0.8731 abortion than states without restriction (w/o laws)
- For every 1% increase in the religion of a state, there is a corresponding 0.02007 increase in abortion
- For every 1% increase in picketing, there is a corresponding 0.1168 decrease in abortion

As these results show, on the basis of the t statistic, price, income, and picket are statistically significant at the
10% or lower level of significance, whereas the other variables are not statistically significant, although some of
them (laws and education) have the correct signs. But remember that if there is heteroscedasticity, the estimated
t values
may not be reliable.

The R2 value shows that 58% of the variation in the abortion rate is explained by the model. The F statistic,
which tests the hypothesis that all the slopes’ coefficients are simultaneously zero, clearly rejects this
hypothesis, for its value of 8.199 is highly significant; its p value is practically zero. Again, keep in mind that the F
statistic may not be
reliable if there is heteroscedasticity.

As noted, a commonly encountered problem in cross-sectional data is the problem of heteroscedasticity. In our
example, because of the diversity of the states we suspect heteroscedasticity.

VISUAL TEST FOR HETERO:

From Gretl output, SAVE fitted values and square of residuals. Go to main Gretl and plot (Fitted at X and
Square resid at Y).

It seems that there is a systematic relationship between the squared residuals

and
the estimated values of the abortion rate, which can be checked by some formal
tests of heteroscedasticity.

DETECTION:
Breusch-Pagan – Now, for BP test, the null assumes homoskedasticity. So if p_val < 0.05 (or your chosen
alpha value); you reject the null and infer the presence of heteroskedasticity and if p_val > 0.05 (or your
chosen alpha value); you fail to reject the null and conclude there may not be heteroskedasticity.
Note: A weakness of the BP test is that it assumes the heteroskedasticity is a linear function of the
independent variables. Failing to find evidence of heteroskedasticity with the BP doesn’t rule out a
nonlinear relationship between the independent variable(s) and the error variance.

Regress e2 on IVs. (Regress the e2 on all the IVs.).

H0: Error terms are homoscedastic

p-value = .0166, therefore REJECT Ho. Heteroscedastic.

NOTE: same output if we do TESTS – Hetero – BP test

White’s test: White test provides a flexible functional form that’s useful for identifying nearly any pattern
of heteroskedasticity. It allows the independent variable to have a nonlinear and interactive effect on the
error variance. So most commonly used test for homoskedasticity is White test.
Regress e2 on: all regressors, their squares, pairwise cross product

Complete White test: Note we don’t reject H0 (homoscedastic). Chi-square is sensitive to many IVs.
The null hypothesis for White’s test is that the variances for the errors are equal. In math terms, that’s:
H0 = σ2i = σ2.
The alternate hypothesis (the one you’re testing), is that the variances are not equal:
H1 = σ2i ≠ σ2.
The only different between White’s test and the Breusch-Pagan is that its auxillary regression doesn’t
include cross-terms or the original squared variables.
Other than that, the steps are exactly the same.

When to use White test?

If your data set has many explanatory variables, the test
may be challenging to calculate. Unless you have a
specific reason for running the White Test (i.e. you
need your independent variable to have a interactive,
non linear effect on the variance), you should use the
simpler Breusch-Pagan. White’s test is an asymptotic
test, which it’s meant to be used on large samples. For
smaller samples, interpret the results with caution.
One issue with White’s test us that it can return
a significant result even if the variances of the errors
are equal. This happens because the model is a general
one, and may pick up other issues in your data
(although it won’t specify what those issues are!).
According to Richard Williams, one of the reasons the
test is more general is because of added terms that test
for more types of heteroscedasticity, like adding
squares of regressors (i.e. the independent variables) to
try and identify non-linear shapes like an hourglass.

In the test above, p = 0.5667 >.05, hence we do not reject H0, thus we infer homoscedastic error terms. As
this exercise shows, White’s chi-square test is sensitive to whether we add or drop the squared and cross-
product terms from the auxiliary regression. Remember that the White test is a large sample test. Thus,
when we include the regressors and their squared and cross-product terms, which results in a loss of 33
df, the results of the auxiliary regression are likely to be very sensitive, which is the case here.

White’s test – squares only of IVs and not cross – product ( still not signif, thus NOT reject H0), with the p
valeue = 0.123 > .05
White’s test with ONLY fitted value and squared fitted values as regressors (proxy of all the other IV and
their cross-product)

To avoid the loss of so many degrees of freedom, White’s test could be shortened by regressing the
squared residuals on the estimated value of the regressand and its squares. That is, we regress:
ei2 = α1 + α2 * Abortion + α3 * Abortion2 + v i

Now we see p-value is signif, thus reject H0.

REMEDIES
Knowing the consequences of heteroscedasticity, it may be necessary to seek remedial measures. The problem
here is that we do not know the true heteroscedastic variances, σi2 , for they are rarely observed. If we could
observe them, then we could obtain BLUE estimators by dividing each observation by the (heteroscedastic) i
and estimate
the transformed model by OLS. This method of estimation is known as the method of weighted least squares
(WLS). Unfortunately, the true σi2 is rarely known. Then what is the solution?

1) If the true error variance is proportional to the square of one of the regressors, we can divide both
sides of
the equation by that variable and run the transformed
regression. Suppose in the equation on the left, the error
variance is proportional to the square of income. We therefore
divide the equation by the income variable on both sides and
estimate this regression. We then subject this regression to heteroscedasticity tests, such as the BP and White
tests. If these tests indicate that there is no evidence of heteroscedasticity, we may then assume that the
transformed error term is homoscedastic.

2) If the true error variance is proportional to one of the regressors, we can use the so-called square
transformation, that is, we divide both sides of (5.1) by the square root of the chosen regressor. We then
estimate the regression thus transformed and subject that regression to heteroscedasticity tests. If these
tests are satisfactory, we may rely on this regression.

AD HOC Transformations

3) Divide DV by its FITTED VALUE – abortion / abortion

Note that we do transformations to get rid of heteroscedasticity

If you do White’s or BP, still heteroscedastic

4) Logarithmic transformation - Get the log of regressand (abortion) (Note: You can only take logs of
positive numbers)

LOG of Regressand Output

The reason for this is that the log transformation

compresses the scales in which the variables are
measured, thereby reducing a tenfold difference
between two values to a twofold difference.

For example, the number 80 is 10 times the

number 8, but ln 80 (= 4.3280) is about twice as
large as
ln 8 (= 2.0794).

The one caveat about using the log transformation

is that we can take logs of positive numbers
only.

Regressing the log of the abortion rate on the variables included in Eq. (5.1), we obtain the following results.
Qualitatively these results are similar to those given in Table 5.1, in that the price, income, and picket variables
are statistically significant. However, the interpretation of the regression coefficients is different from that in
Table 5.1. The various slope coefficient measure semi-elasticities – that is, the relative changes in the abortion
rate for a unit change in the value of the regressor.16 Thus the price coefficient of –0.003 means if price goes up
by a dollar, the relative change in the abortion rate is –0.003 or about –0.3%. All other coefficients are to be
interpreted similarly.

When this regression was subjected to Breusch–Pagan and White’s test (without squared and cross-product
terms), it was found that this regression did not suffer from heteroscedasticity. Again, this result should be
accepted cautiously, for our “sample” of 51 observations may not be large enough. This conclusion raises an
important point of about heteroscedasticity tests. If one or more of these tests indicate that a problem of
heteroscedasticity,
it may not be heteroscedasticity per se but a model specification error.
ROBUST (Valid in LARGE SAMPLES)
If there is HETERO, remodel and click ROBUST.

Still, we reject H0. (Sample is small, only 51).

REVISIT WAGE FUNCTION and CHECK FOR HETERO

Then correct for one if there is.

Notes on Weighted Least Squares (WLS) – used when assumption of constant variance assumption is
violated

Estimation is different in WLS v.v. OLS

In OLS, we don’t have the W terms.
In OLS,
β matrix = (XTX)-1 (XTY)
JULY 30 ABORTION

BP – squared residual (usq2)