I did get a chance to read the paper. They seemed to take a fairly reasonable approach to the issue, but I’m a little confused. It seems like they were not using the data from the first grade follow up (I may be wrong). That paper came out before the final report on the HHS study (2008 vs 2010). I think the biggest disappointment for head start supporters was that almost complete elimination of effects by the 1st grade, not just the small size of the initial effects. I think HHS is already crunching the numbers on the 3rd grade follow up, so maybe we can see some evidence of longer term impacts at that point. I can’t really comment on you statistics questions… I really would have to look at my stats books again… and I don’t think I can handle that on an emotional level… I told myself after my dissertation defense that I would never have to open a stats book again. I guess my final take on the issue is that the results were not promising, but there does need to be a continued effort to identify and encourage effective programs.
I appreciate the discussion on the issue, I certainly know more about it now than I did last week. The Brookings Institution, does good work on these issues (one of the authors of the article you linked was from Brookings). The original impetus for this discussion was related to why people would wan to cut funding for these programs. You wrote “I don't think many people would support a program that has been shown to NOT actually help (I guess some might want to continue the funding with an overall of the program). I'd be curious if you can point to a single program that has been shown to NOT be affective and somebody argued we need to continue to fund it.”
Here is one take from the Brookings Institution following the final report.
“A rigorous study found that the program, after producing some initial gains during preschool, had almost no effect on children's cognitive, social-emotional, or health outcomes at the end of 1st grade, compared with a control group of children whose families had access only to the usual community services.”
The author goes on to say
“This is the 10th instance since 1990 in which an entire federal social program has been evaluated using the scientific "gold standard" method of randomly assigning individuals to a program or control group. Nine of those evaluations found weak or no positive effects, for efforts such as the $300 million Upward Bound program (academic preparation for at-risk high school students), and the $1.5 billion Job Corps program (job training for disadvantaged youths). Only one - Early Head Start, a sister program to Head Start for younger children - was found to produce meaningful but modest effects.”
My thought is that well meaning and informed people can be for cutting and eliminating these types of programs. We need to keep looking for answers, but maybe on a smaller scale until the data come in.