GothamSchools — daily independent reporting on NYC public schools

Dear Aaron: Next time you consider writing something so remarkably idiotic, I would encourage you to give me a call so I can spare you the embarrassment. Here’s your logic: 1) Chris Cerf was interviewed for a book; 2) A passage in the book says ” … ” 3) therefore 3) The passage can be attributed to Chris Cerf. Your leap to this conclusion on the ground that “nothing in the public record” supports the statement. Even if the logic of your reasoning were sound — and it is demonstrably silly — you are completely wrong about the latter point. (Had you called I could have explained in detail your misunderstanding of the sequence of events and corresponding public statements.) But, here’s the bottom line: I personally brokered and drafted the letter that says that teacher data reports won’t be used in tenure decisions. That is the simple truth then and now. Even by the low journalistic standards embraced by Skoolboy, this one was a lulu.

More dispassionate analysis from skoolboy, the master of evidence analysis. Except when there is no evidence and he becomes the master of making things up. Either way, ideological manipulation is always the outcome.

Socrates,

I think that you might be reading to much into what Aaron wrote.

He is not suggesting that the Klein/Bloomberg want to take all hiring or firing authority from principals. Rather, he is focusing on one particular decision at one particular time.

Moreover, I don’t think that Klein/Bloomberg are interested in giving full autonomy to principals. The are happy to control the aims and goals of education for themselves. They are happy to set the standards by which school are evaluated. They are intent upon equating test scores and learning.

I think make an excellent point that using a this kind of system to evaluate teachers contradicts their principal as CEO model. However, I think that their system of for school evaluation also contradicts that. A CEO might be judged by total profit, by % marekt, by expanding market share, by controlling costs. Depending on the conditions of market and the company’s position in the market, the CEO might have different goals. But the Klein/Bloomberg model does not recognize that CEO’s need to have a say not only in selecting their methods, but also in selecting the the appropriate goals (and even metrics).

So, I don’t think that Klein & Bloomberg’s support of principal as CEO is entirely complete, and given their belief in test scores, it would not surprise me at all if they put in this kind of teacher evalatution system at one critical point. The fact that it is — and likely will remain — so quite difficult for principals to remove their predecessor’s tenured teachers, they view this sort of system as the lessor of two eveils.

Moe, Chubb and Cerf got to put all their theories into practice with Edison Schools, the for-profit privatization fad of 10 years ago that’s now treated by its former cheerleaders like an embarrassing odor in the room. (OK, Moe wasn’t directly involved, but it was still his principles being put into practice — and Chubb and Cerf were running the show.)

Why do any of them still have any credibility at all?

I can see the point — judging the quality of their arguments, ideas and evidence rather than their past record. On the other hand, there IS a point at which someone’s credibility is called into question by the failures of their past ideas.

Terry and I have no dog in the gotch fight with Joel Klein and Chris Cerf that animates this discussion. They are quite capable of explaining how and why they wish to use test score data to improve the city’s schools. In our book we applaud their efforts to gather all kinds of data that informs the development of students, teachers, and schools. We also acknowledge, indeed underscore, the interests at stake in using data and the accomodations and compromises that must be made to employ data when the stakes are high. Klein and Cerf must balance those interests every day on this issue and many others. For our part, we only want to make clear that we believe sound decisons about teaching and teachers should absolutely consider objective evidence of their effects on student achievement–especially on the standardized tests for which schools are accountable. Afterall, would we not use objective evidence to evaluate a piece of instructional software or online program? Of course we would. But research also makes clear that teacher performance should be evaluated with other indicators as well as test scores. We say this explicitly in chapter three, well before we tell the NY story in chapter four. We do not see test score data as a panacea to the improvement of teaching. At the same time, we find it indefensible that test scores be off-limits in evaluating teaching. Happily for our kids, the truth always wins out.

Did Chris Cerf really write that? Looks like someone has more gonads than he is usually accusesd of having in the public sphere!

Ceolaf,

I don’t see the inconsistency. Mr. Chubb clearly states on this very site that he doesn’t believe test scores are everything but that robust data collection is a virtue of the Klein/Cerf administration. In the quote you cite he rejects (many years ago, but perhaps still today) the notion that test scores alone are sufficient, and even suggests that test scores used in isolation are completely insufficient. He cites 3 problems: 1. They leave some types of knowledge untapped, 2. Schools and teachers will focus too narrowly, and 3. They depend on more than simply school/teacher quality.

In his much more contemporaneous quotes, Chubb issues a possible solution to these 3 problems: use of more robust data and sophisticated evaluations. This potentially mitigates the 3 problems by tapping the other knowledge types, focusing teachers on a broader set of indicators, and accounting for issues that are beyond a school’s scope of control. I don’t see Chubb contradicting himself here at all.

Caroline, of course, chimes in with her typical anti-charter vitriol, unaware that her position is diminished by the fact that it would never change regardless of which way the evidence pointed. We get it, Caroline, all charter schools and people who run them are evil.

Socrates,

I think that you misread the passage I quoted. It does not say that tests are not sufficicent. Rather, it says that they are bad. “The danger is…that they will actually hurt the schools more than help them over the long run.”

He wrote that they are inherently “imperfect yardsticks,” they *will* becomes the official yardstick by which schools are assessed — which as come to pass — and that they will thereby distort the work of teachers and schools. I am curious what changed his mind about this. It appears that his prediction came true, so why the shift?

He cites a problem of people jumping to unwarranted conclusions about schools, based on tests. Does he think that will no longer happen? Does he have so much faith in value-added, despite the narrow range that tests address, that he no longer believes that people will “jump to conclusions — unwarranted conclusions.”? Or is there some other reason why this is no longer a concern for him?

Chubb and Moe have long had concerns with bureaucracy, and I believe that they still do. In the passage, he write, “They create still more bureaucracy, and they unleash new bureaucratic pathologies that divert people and resources from the pursuit of quality education.” Does he no longer believe this? If not, I’d love to understand his shift.

Let’s be clear here. They did *not* write that tests are “insufficient.” They, instead, wrote that despite tests’ appeal, there are “they are clearly deficient as solutions to the problems they are addressing, and they stand little chance of improving schools in any significant way.”

You offer that they call for “more robust data and sophisticated evaluations.” What does this mean, and does Chubb believe that that is sufficient to address the concerns he wrote of nearly 20 years ago? Does he think that we are capable of gathering robust enough data and doing sophisticated enough evaluations? Are were there, yet?

I realize that my comments appeared to veer off topic. But my point is to question whether Mr. Chubb should be treated as a credible commentator at all when a project he was in charge of disseminated a long and jaw-dropping series of false and misleading claims on his watch. Over the long run, of course, Edison Schools’ “miracle” claims did not hold up, so that backdrop adds credibility to my question.

I learned about the very concept of “juking the stats” at the knee of Edison Schools. That’s largely WHY I’m a critic of charter schools and other “it’s a miracle!” education reforms — I learned from following Edison how the magician performs the illusions. I don’t see how Mr. Chubb can just switch hats and go back to being a credible commentator on education after engaging in that show. Now, the book he COULD write on “How to Juke the Stats” — THAT I’ll read.

Socrates says: “Caroline, whether Edison juked the stats or not I have no idea. The problem is, you take that to mean that all charters that are successful juke the stats, and you’ve set about to prove it, regardless of the counter-evidence.”

No, I don’t say that all charters that are successful juke the stats. Some do. The playing field is unlevel, though, because of the creaming factor and other advantages conferred on charters — so in some sense the stats juke themselves. I do indeed write a lot about that, because it’s an important point in the debate about education “reform.”

And, Socrates, you do have an idea of whether Edison juked the stats, because I gave you details, should you choose to pay attention to them.

Edison Schools was an extreme case of active — and flagrantly dishonest — juking.

Socrates says: “Caroline herself is the one who says that if the messenger is flawed, the message is unworthy of attention.”

Every messenger is flawed, and that’s not what I say. But when the messenger has in the recent past presided over a large-scale deception, I think there’s a “fool me once, shame on you; fool me twice, shame on me” message that needs to get across. A clean slate is not appropriate in this extreme a case.

Socrates says: “That’s why she changed the subject in this thread – to discredit the messenger. So what I gave her was a dose of that medicine – she, too, has a “long and jaw-dropping series of false and misleading claims.” ”

I am indeed attempting to point out why this messenger has discredited himself. It’s an extreme case and needs to be illuminated. However, regarding my honestly, I strongly disagree and object — that’s an unfair and untrue flame, and you can’t back it up, Socrates.

In addition, I’m an unpaid amateur volunteer mommy blogger. Chubb is a well-known paid think-tanker who was in a paid leadership role in a for-profit operation when those Edison Schools stats were being so vigorously juked. He had a financial interest in the juking. There’s no possible analogy here.

Lets get back to the matter at hand, Aaron why haven’t you redacted your assertion that the statement in question comes from Clef?

RAND did indeed release a research report on Edison, which gave the standard “mixed results” conclusion. Prior to the release of the report, Edison spokespeople and supporters repeatedly claimed, inaccurately, that Edison’s inaccurate and misleading claims about its schools’ achievements came from RAND.

I co-wrote an analysis of Edison’s claims, including RAND’s denial that the organization was behind those claims, posted June 15, 2002, on the website www dot pasasf dot org. Readers with sufficient interest may judge for themselves whether a social scientist is on firm ground standing behind the methodology of the analyses described below.

Mr. Chubb is correct in stating that Edison offered transparency in giving breakdowns of its claims. That raised questions of whether the deception involved in the claims described below was deliberate. And that raises yet more questions of its own. All information in the press release below was correct as of June 15, 2002.

Parents Advocating School Accountability, San Francisco
June 15, 2002

RAND Denies Link with Questionable Edison Success Claim

Spokespeople for the RAND Corp. say the respected research organization has no connection
with an Edison Schools claim about the company’s purported success, despite widespread
insistence by Edison representatives and supporters that RAND has confirmed the claim.

The controversial, for-profit school management company contends that 84 percent of its schools made “positive gains” in the 2000-01 school year. The figures Edison gives for that claim do not work out to 84 percent, and the assertion fails to hold up under scrutiny in other aspects. But Edison spokespeople and supporters have continued to repeat the figure and have frequently attributed it to RAND.

The inaccurate RAND attribution has appeared, uncorrected, on the Wall Street Journal op-ed
page and has been cited by Edison representatives at community meetings in Philadelphia, where Edison is set to take over 20 schools in the fall. It has also reportedly been used by Daniel J. Whelan, a member of Philadelphia’s School Reform Commission.

RAND’s website explains that the organization does not yet have results to report on its study of
Edison. “In the summer of 2000, RAND began a three-year, independent evaluation of Edison
Schools, which will examine student achievement outcomes as well as the implementation of
Edison’s academic program …” the website explains. “When that evaluation is complete, RAND
will release public reports indicating our findings, whether favorable or unfavorable to Edison.
“We do not yet have results to report from our ongoing study,” the website adds.

David Tillipman of RAND confirmed that statement in a May 22, 2002, e-mail to Keith Newman
of Philadelphians United To Support Public Schools. “We have not completed our evaluation
project,” Tillipman said. “We are not able to offer any opinions on Edison’s success until we
complete our effort.”

Edison Schools elaborates on its “84 percent success” claim in its 2000-2001 Annual Report on
School Performance (“Exhibit 5: Average Achievement Gains Since Year School Opened,” page
18, available on Edison’s website). The company lists 62 schools that purportedly showed
“positive achievement gains.” But Edison says it ran 113 schools that year. Sixty-two of 113
schools would be 54.86 percent, not 84 percent. Edison has never explained the discrepancy.
The “Exhibit 5” page lists a number of schools exempted from the calculation, but the list leaves
23 of Edison’s 113 schools unmentioned and unaccounted for.

Claims of success at the 62 listed schools are also highly questionable. The “positive” list includes at least 12 schools for which contracts with Edison have been or are being severed. Many more are struggling or plagued with low performance. One example is Henry E.S. Reeves in Dade County, Fla., which has just received a “D” rating on Florida’s A-F school assessment system for the fourth year in a row, barely escaping an “F.”

Among other examples, the “positive” list also includes Feitshans-Edison in Springfield, Ill., and
Northmoor-Edison in Peoria, Ill., though test scores at both schools have dropped since Edison
took them over.

The list displays further inconsistent methodology. For example, it specifically excludes eight
schools because they were in their first year of Edison operation (“the following schools have only baseline scores”). Yet the “positive” list includes 17 schools that were also in their first year of Edison operation. Edison has reportedly not responded to media inquiries about the discrepancies and inconsistencies.

Edison Schools, which runs more than 100 schools nationwide, is a nationwide school
management firm with stock publicly traded on the NASDAQ, though the 10-year-old company
has never made a profit. Edison Schools’ stock has recently plummeted, losing more than 90
percent of its value.

Edison has attracted ideological support from backers of privatization and school vouchers, and
from such powerful conservative bast ions as the Wall Street Journal editorial board and the
Hoover Institution.
##
Edison schools on “positive gains” list for which contracts have been or are being severed:
– Edison-Ingalls Partnership School, Wichita, Kan. (severed as of end of 2001-02 school year)
– Granville Charter School, Trenton, N.J. (severed as of end of 2001-02 school year)
– Boston Renaissance Charter School, Boston (severed as of end of 2001-02 school year)
– Wintergreen Interdistrict Magnet School, Hamden, Conn. (to be severed at end of 2002-03
school year)
– Edison/PPL School, Minneapolis (severed as of end of 2001-02 school year)
– Mid-Michigan Public School Academy, Lansing, Mich. (severed at end of 2000-01 school year)
– Edison-Isley Partnership School, Wichita, Kan. (severed as of end of 2001-02 school year)
– Timberview-Edison Junior Academy, Colorado Springs, Colo. (severed at end of 2000-01
school year)
– Elm Creek Elementary School, San Antonio/Atascosa, Texas (severed as of end of 2001-02
school year)
– Kriewald Road Elementary School, San Antonio/Atascosa, Texas (severed as of end of
2001-02 school year)
– McNair-Edison Junior Academy, San Antonio/Atascosa, Texas (severed at end of 2000-01
school year)
– Edison Charter Academy, San Francisco (contract with school district severed at end of
2000-01 school year; Edison received a charter from the California state Board of Education and now operates the school under the state charter)

The contention that RAND is behind the Edison Schools 84 percent success claim appeared in the
Wall Street Journal op-ed page in a column by Pete du Pont, “Bureaucrats First, Kids Second:
That’s the Ethos of America’s Public School Establishment” (April 18, 2001).

Dr. Chubb,

Oh, yeah. I left one out.

You wrote, “The reality is that people look at a school’s average test scores and jump to conclusions-unwarranted conclusions-about school performance.” Why do you think that would be any less true today than 20 years ago? Why do think that parents today would be less drawn to jumping to unwarranted conclusions about school performance based on schools’ test scores than whomever you were referring to (“people”) 20 years ago?

As you believe that those old challenges still stand, and given the rise of choice/market mechanisms, wouldn’t such unwarranted conclusions be of greater concern today?

Keep articles like this coming, please.