Bryan Caplan points to a study showing that the cognitive effects of upbringing evaporate over time.
If the only result from this study had been the "IQ is heritable," it would have been just another study. But its special methodology - studying adoptee's development from birth to adulthood - confirmed a shocking finding: As children grow up, the heritability of IQ rises, and the influence of family environment on IQ literally vanishes. . . .
We naturally think about the effects of family as cumulative: The longer you're in a family, the deeper the impression. At least for IQ, though, this "natural" thought turns out to be wrong. Family affects the very young, then fades out.
In hindsight, should this pattern really have been so surprising? Yes and no. Consider the parallel case of church attendance.
For a young child, family has near-absolute control over church attendance (unless you're Damien in The Omen, of course!): If your parents go, so do you; if they don't, you don't. As you get older, though, you gain some independence - and with it, a chance to show your true colors. By the time you're an adult, you only go to church if you want to. So it's not surprising that family matters less over time.
Even so, though, you would expect attending church to be at least somewhat habit-forming. Adults only go to church if they want to, but what they want has something to do with what they've experienced. The surprising thing about family influence on IQ is that the effect actually goes to zero. As you grow up, you find your own cognitive level, and the cognitive level you're looking for has nothing to do with the cognitive level you grew up with.
This is pretty well replicated for both home environment and early childhood education. There are lots of things that early childhood environment does affect: the Perry Preschool Project, for example, produced significant reductions in criminality, while improving high school graduation rates and modestly increasing future income. But we're talking about moving from Popeye's to a steady job in a warehouse or at the Post Office, not mass movement into the professional class.
I confess I still don't understand it--if IQ is so heritable, why is it ever plastic? But I'd say that dealing with this problem is the biggest social policy problem America has, whether we're dealing with true heritability or some masking factor.