If Baby College doesn’t make a difference, to what to you attribute the enormous difference between the 3rd and 4th grade scores?

I realize it’s a complex program, but Baby College is enrolling 150 people a year, which is not a small number.

HCZ is a complex initiative, with multiple components. Only a fraction of the students attending the HCZ Promise Academy schools participated in Harlem Gems or the Baby College, so one cannot conclude that the third-grade reading and math achievement of the students in the Promise Academy are due to these programs. Dobbie and Fryer note that, unlike the lotteries used for selection into the Promise Academy schools, children self-select into Harlem Gems and Baby College in ways that make it difficult to assess the causal impact of these programs. With many caveats, they report that their point estimates for participating in these programs suggest that they decrease third grade math and ELA scores, although these decreases are not statistically significant.

I doubt if participation in Harlem Gems of Baby College actually has a negative impact on test scores, but there’s certainly no evidence that they increase test performance, and thus no evidence that they can account for the reduction of the achievement gap. Overall, Dobbie and Fryer conclude that the HCZ community programs are unlikely to explain the test performance of students attending the Promise Academy schools, and that therefore it’s either the schools themselves or the interaction of the schools and the community investments that explain the test performance.

The real test will be whether or not this is sustained for that cohort on a large scale. Even if it’s not, it will be important to note whether the achievement gap has narrowed by some degree - reasonably, it would take a few classes moving through the Promise Academies before the kinks get worked out and we can see the results we all want.

I agree in the broad sense, but it’s really hard to know whether teachers and schools are doing a good job if we have to wait 20 years to find out in retrospect.  

Wouldn’t it be nice if we had a proxy that could give us some idea now whether teachers and schools are managing to impart any knowledge to children?  Say, whether those children were able to compute math problems and read passages about history and science (as on many reading tests)?  

The older students may receive some services from HCZ, but have not been exposed to the full program. So, rather than cast doubt on the claim that HCZ is closing the achievement gap, these data actually show that HCZ’s full program is working remarkably - and that Baby College is an essential part of its success, as Mr. Canada says in the TAL story.

Canada himself admits that his students did not do nearly as well on the more objective Iowa Test of Basic Skills. Why is this? Were the children drilled on the state test? If so, those results are invalid.

Privileged children learn a great deal from home. Many are read to on a daily basis, taken on yearly vacations and given many opportunities to pursue their interests. Are Canada and others implying that these family activities don’t count?

Finally, as the Scarsdale grandmother suggests, it is foolish to compare populations based on test scores alone. As most of us know (or should know) many privileged children learn so much, so quickly that it often dazzles the mind. There is a virtual explosion of knowledge, most of which is probably untested and untestable.
My own grandchildren, ages 8, 6, and 2 are actually beginning to know some things that I don’t know because of their exposure to their scientist dad and mom.

What Mr. Canada is doing is, in my opinion, the very best that can be done for the children he serves. However, somewhere I read that his own son is in a middle-class school mainly because of the intangibles that the school (and its population?) provides. Exactly.

You’re looking at the April, 2009 Dobbie & Fryer paper, which goes through the 2007-08 school year. The November, 2009 version includes the 2008-09 school year, which is the year that I report in my graphs. I haven’t looked yet to see if the data line up.

The paper is available here: http://scienceblogs.com/appliedstatistics/upload/2009/11/001_hcz%25204.15.2009.pdf.

The 8th grade year on Dobbie and Fryer’s graph should be the same 8th graders in your graphs. They seem to totally contradict each other. Somebodies got to be incorrect here….

Add to the layers of irony the liklihood that “poor black kids in Harlem” are more likely to be both victims of crime as well as confronted by police — in school even — than “affluent white kids in Scarsdale.”

Hey fellow New Yahkers, got Brown v Board of Ed?

Even though I think he often gives short shrift to schools which are legitimately driving huge gains in achievement, he’s right to police claims of “fully” closing the gap.

Our charter school’s state test scores are indeed higher than white and Asian students (and gigantically higher than the “lottery losers” who applied to our school and didn’t get in). But…

But our SAT is lower than national average. And our AP Calc scores are at national average (though that is with unbelievable AP Calc teacher…not sure we can hold that performance if he retires).

Our actual college success rate (black and Hispanic kids from low-income families) seems to be higher (white and Asian national averages), but too early to tell. No idea about labor market outcomes.

Per the earlier commenter, I suspect when one of our younger kids plays cops and robbers, none would invoke habeas corpus (more realistically depict an arrest). This stuff is hard to fully measure.

I’m not talking about the Fryer and Dobbie claim about closing the achievement gap after controlling for income and gender. Please show me where in the 60 Minutes piece there was any attention to controlling for income in the claims about closing or eliminating the achievement gap. You may think it cynical for a sociologist to attend to the persistent and pervasive effects of social class on socioeconomic outcomes, but these effects are real, and it’s what we study. Surely “leveling the playing field” does not mean ignoring family income.

As is true of most research, the question is, “compared to what?” Compared to how the children and youth attending HCZ Promise Academies might have fared in the absence of HCZ, the evidence that Fryer and Dobbie present is powerful: this intervention is having a large impact on the academic achievement on these youth (at least in terms of state assessments that we now know have lots of problems). Compared to the performance of the typical Asian and white student in New York City, or the typical student in Scarsdale — with whom the HCZ youth will be competing for entry into college, among other things — the data I report show the HCZ youth well behind. These data are a snapshot, to be sure, and HCZ is an evolving program; but they are not, in my view, a distorted response to the question to which they are addressed.

The Fryer and Dobbie piece says the HCZ closed the black-white achievement gap after controlling for income and gender. Your graphs are misleading because they include income, so what shows up is the black/white AND the low-income/ high-income gaps. It’s like comparing apples to apples plus oranges.

More here: http://tinyurl.com/yfslxna
And here: http://tinyurl.com/yhr2czl

Sure, apples and oranges in the sense that Fryer’s study is addressing a different research question than the data I present. I’m not commenting on his study, part of which is about the impact of attending HCZ Promise Academy charter schools on student outcomes. I’m commenting on the claim that the students attending the HCZ Promise Academy charter schools have eliminated the achievement gap. The best evidence on that point is the direct comparison of the performance of HCZ students with citywide white and Asian students, and with Scarsdale students.

When students compete for access to selective colleges, or other desirable positions, we don’t look at their regression-adjusted performance. We look at their actual academic records. That’s what I’m commenting on.