For those observers who feel that one cannot develop effective programs unless one has children currently in the system, Liebman brought additional credibility as a parent – in the parlance of many money managers, at least he was eating his own cooking.

ED Hirsh, an iconic education leader for many, has observed that these bubble tests that many parents so decry are among the more reliable and valid tests we have.

That said, the fundamental flaw, and one that the Department has been either unable to understand or unwilling to address, is the use of the NY State ELA and Math exam scores for purposes for which they were not intended. The critical issue is that the exams are not vertically linked – that is comparable across grade levels. So one cannot compare students scores from one grade to the next higher grade. NY State Ed Dep’t officials know this, external testing experts know this, and (I bet) even people in the Board of Ed’s testing group know this.

Thus the fundamental conceit of the progress reports – that they track an individual child’s progress from 3rd to 4th to 5th grade and so on, is flawed. However fancy one wants to make the mathematics beyond this point, it just doesn’t matter.

As a non-mathematician and non-testing expert, I never expected Liebman to understand this – I saw his role as more sketching the big picture and leaving to other, more qualified folks, the nuts and bolts of implementing the design. But it is a shame that someone as smart, influential, and respected as Liebman did not use this power to demand that a proper progress metric be developed. Surely with the money available to hire all the smart folks working in the office of accountability, this could have been done.

Liebman once told a parent audience that he never intended the progress reviews as the only metric of a school’s quality, There are the qualitative reviews done by the outside experts (and increasingly trained staff from the Board of Ed), comments by outside groups like Inside Schools, and a parents own two eyes and ears that should come to bear. Nevertheless, the Board and City Hall spent significant time, attention, and money on progress reviews that at their core do not measure progress.

As a city, and a nation, we face real challenge in developing good systems that measure student progress, and ultimately a teacher’s role in that progress. It is a shame that as well regarded a system as New York City and as committed a fighter for civil rights as Jim Liebman, couldn’t demand real solutions, instead of talking points for elected officials. And that none of the media watchdogs and bloggers called the Board’s bluff.