More than a decade ago, back in the advanced econometrics course at George Mason, Thomas Stratmann taught us about the weak instruments problem. Looks like one of these was worse than I’d thought.
Suppose you wanted to know whether education affected earnings. And you saw that people with more education had higher incomes. But suppose that you also thought that people who were smarter (or more conscientious) would earn more regardless of education AND that those same people were more likely to select into more education. You then have an endogeneity problem: just running a wage regression with education on the right hand side doesn’t really give you the effect of education on earnings. It tells you how much more those with more education earn compared to those with less education, but it’s a total effect rather than a partial: it includes all the effect of selecting into more education as well as the effect of the education.
So, then, what to do? You need something that will affect the amount of education that somebody gets that isn’t itself a predictor of income. Something exogenous to income that affects education. If you have that, you can use an instrumental variables approach to try to estimate the effect of education on income via the effect of the exogenous instrument on education.
And the typical early example is the old Angrist and Krueger measure of quarter of birth as instrument for education. Compulsory schooling laws in the US meant you couldn’t quit school until you were some minimum age. Quarter of birth could then affect how much schooling you got: those who turned 16 early in the school year could drop out earlier than those who turned 16 later in the school year.
Then Prof Stratmann told us about the weak instruments problem. Quarter of birth is only really weakly correlated with education – it explains very little of actual differences in education levels. And so bad stuff can happen in the estimation.
So we’ve known for a while that quarter of birth might not be a great instrument. But it would still get used. Bound, Jaeger and Baker complained about it in 1995; I think we went through their critique in our 1999 metrics class. Jaeger got angry in 2009 when quarter of birth was still being cited as a decent instrument.
Season of birth is associated with later outcomes; what drives this association remains unclear. We consider a new explanation: variation in maternal characteristics. We document large changes in maternal characteristics for births throughout the year; winter births are disproportionately realized by teenagers and the unmarried. Family background controls explain nearly half of season-of-birth’s relation to adult outcomes. Seasonality in maternal characteristics is driven by women trying to conceive; we find no seasonality among unwanted births. Prior seasonality-in-fertility research focuses on conditions at conception; here, expected conditions at birth drive variation in maternal characteristics, while conditions at conception are unimportant.
If different types of parents select different seasons in which to have kids, then there goes the exclusion restriction: IV just plum don’t work right if the instrument affects outcomes via channels other than the one specified. I think that Dubner at Freakonomics understates the importance here when he says the finding’s interesting because so many studies have found an association between birth month and later outcomes: the importance is rather that we have something much more serious than a weak instruments problem when using quarter of birth to get around endogeneity problems in estimating the effect of schooling on income.
@KevinMilligan points out that folks have moved toward regression discontinuity design for this kind of work anyway.
Agreed! But I still wonder whether loud cries of “WE TOLD YOU SO!” were coming from Jaeger, Bound or Baker’s offices when the Buckles and Hungerman paper came out….
Update: I’m going to give the last word on this one to Dr. Phil:
Finding a solid IV is a grueling task. Make it a little more fun by turning it into a scavenger hunt for the kids.
— Dr.Phil of Economics (@DrPhilofEconomi) August 13, 2013