GothamSchools — daily independent reporting on NYC public schools

I’ve been relatively quiet in the ongoing debate about how best to evaluate teachers in New York City and across New York State. I’m not close to the negotiations and can claim no expertise on the political machinations outside of public view. At its heart, this seems to me a dispute over jurisdiction: Who has the legitimate authority to regulate the work of an occupation that seeks the status of a profession—but one that is in a labor-management relationship?

The laws of New York recognize the labor-management fault line, but they do little to guide a collective-bargaining process toward agreements in the many districts in which teacher-evaluation systems are contested. Each side brings a powerful public value to bear on the disagreement.

For the employers, it’s all about efficiency. It’s in the public interest, they argue, to recruit, retain and reward the best teachers, in order to maximize the collective achievement of students. A teacher-evaluation system that fails to identify those teachers who are effective, and those who are ineffective, can neither weed out consistent low-performers nor target those who might best benefit from intensive help. Rewarding high-performing teachers can, in the short run, help keep them in their classrooms, they claim, and, in the long run, can help expand the pool of talented individuals who enter the occupation. (more…)

What counts as a “fact”? New York State Supreme Court Justice Cynthia Kern’s ruling on the release of the New York City Teacher Data Reports reflects a view very much at odds with the social science research community. In ruling that the Department of Education’s intent to release these reports, which purport to label elementary and middle school teachers as more or less effective based on their students’ performance on state tests of English Language Arts and mathematics, was neither arbitrary nor capricious, Kern held that there is no requirement that data be reliable for them to be disclosed. Rather, the standard she invoked was that the data simply need to be “factual,” quoting a Court of Appeals case that “factual data … simply means objective information, in contrast to opinions, ideas or advice.”

But it is entirely a matter of opinion as to whether the particular statistical analyses involved in the production of the Teacher Data Reports warrant the inference that teachers are more or less effective. All statistical models involve assumptions that lie outside of the data themselves. Whether these assumptions are appropriate is a matter of opinion. Among the key assumptions that are necessary to make inferences about teacher effectiveness from student performance on the state tests are the following:

• The tests are valid measures of students’ mastery of English Language Arts and mathematics.
• A student’s performance on the test, which is taken on a particular date, reflects how that student would perform on the test on other dates.
• The student, classroom and school-level variables taken into account in the value-added model underlying the Teacher Data Reports are appropriate for inferring that a particular teacher caused the test-score gains experienced by that teacher’s students.
• Test-score gains observed on tests administered in the middle of one year and the middle of the following year can be properly apportioned to the prior-year teacher and the current-year teacher.

The fact that reasonable people might disagree about these assumptions makes clear that they are a matter of opinion. (more…)

Mayor Michael Bloomberg’s selection of Hearst Magazines chairman Cathie Black as chancellor of the New York City public schools has hastened a crisis over how to assess expertise in a complex educational system. Does Black have the expertise necessary to assume leadership of a school system with a budget of $23 billion, 135,000 employees, and 1.1 million students? The mayor certainly thinks so. He has described the job as being able to “solve complex problems in the face of controversy, motivate staff, communicate with and bring together diverse constituents, manage labor relations, use data in decision making, and sustain a culture of change and excellence.” Black’s experience in publishing, he has argued, has demonstrated her bold vision, capacity to make tough financial decisions, skills in negotiation and building support among constituents, and knowledge of state and federal laws. In the eyes of the mayor, these skills — none specific to the field of public education — constitute the expertise required to do the job.

The state of New York has a different conception of the expertise needed to be a school district superintendent. State law specifies that to obtain a professional school district leader certification, school district leaders (i.e., superintendents) must have completed a School District Leader program authorized by the state; accumulated a minimum of 60 semester hours in graduate courses approved by the state commissioner of education; and have at least three years of teaching experience. The certification also includes a full-time, 15-week clinical component of school-building leadership experience or its equivalent, and requires passing two written School District Leader assessments.

The content of the School District Leader assessments provides some purchase on the kinds of expertise that the state views as necessary to successful practice. The standards expressed in these assessments include applying knowledge of skills for engaging building leaders, board members, community members, parents/guardians, students and school staff in an ongoing dialogue regarding core values, goals, policies, practices and achievements; demonstrating knowledge of the New York State Code of Ethics for Educators and the role of values and ethics in district leadership; demonstrating knowledge of factors to consider in comprehensive, long-range planning, including the importance of involving all key stakeholders in planning processes; analyzing concepts, principles and best-practice applications of developmental and learning theories, curriculum development, instructional delivery, and classroom organization and practices with regard to the diverse needs of all students (e.g., special-education students, English-language learners, gifted and talented students); analyzing strategies for developing staff capability through the supervision and evaluation of teachers and building leaders, effective staff assignments, and systems of mentoring, support, and development; and demonstrating knowledge of processes of collective bargaining and contract management that support and extend the educational vision, to name just a few.

If the various requirements of the School District Leader certification are indicators of the expertise that New York state requires of school superintendents, and Cathie Black has not met those requirements, how are we to judge if she has the requisite expertise? (more…)

Here’s what you need to know about “Waiting for ‘Superman.” It’s not a film — it’s a propaganda campaign.

That’s not necessarily a bad thing.

The term “propaganda” has gotten a bad rap, ever since its association with 20th-century totalitarian governments promoting troubling political objectives. But there is a long and honorable tradition of propaganda in the genre of documentary films. In its original formulation, “propaganda” is simply a deliberate effort to change what people know, understand and value, for a particular purpose. Propaganda can rely on many different media and symbols to carry its message. Documentary films have often sought to activate a sense of urgency about a social problem or condition that needs our attention. The medium of film is especially powerful because propaganda often appeals to emotion as much as reason, and film is very effective at evoking an emotional response. Much better than, say, a speech by Al Gore, Arne Duncan or Bill Gates.

I had the opportunity to view Waiting for “Superman,” the new documentary by Academy Award-winning filmmaker Davis Guggenheim, at a pre-release screening at Teachers College last week. Based on the early buzz from proponents and detractors alike, I expected to see a film that lived up to its billing as “stirring” or “moving.” (more…)

I’ll admit it: When I hear the phrase “charter school miracle,” my antennae go up. It’s not that I think that charter schools can’t possibly be good schools, or that they cannot surpass traditional public schools in the measured achievements of their students. The evidence is pretty clear that there are many fine charter schools, just as there are many struggling charter schools.

No, it’s that I think miracles are exceedingly rare phenomena. And the current narrative about miracles in school reform relies heavily on a “great man” theory, replete with outsized personalities. Witness the contemporary stage, on the cusp of the release of Waiting for “Superman”: Geoffrey Canada, Michelle Rhee, even — God help us — Bill Gates and Joel Klein being anointed as miracle-workers who, by dint of their commitments, hard work and personalities, are overcoming entrenched bureaucracies and transforming the life-chances of poor and minority children across America’s urban landscape.

It was against this backdrop that I read Caitlin Flanagan’s stirring op-ed that graced the gatefold of Sunday’s New York Daily News. Flanagan, a former prep-school teacher who now writes for The Atlantic and other publications, singles out Mike Piscal, who founded a charter management organization called the Inner City Education Foundation (ICEF) that now operates 15 elementary, middle and high schools in south Los Angeles. Flanagan and Piscal were colleagues, once upon a time, in the English department of the elite Harvard-Westlake School.

Flanagan’s argument goes something like this: the ICEF schools are extraordinarily high-performing; in fact, the elementary schools have eliminated the achievement gap. (more…)

I’ve become increasingly alarmed at the growing divide between the news and editorial functions of major metropolitan daily newspapers (e.g., in New York City, the New York Times,  New York Daily News,  and the New York Post;  in Washington, DC, the Washington Post).  The functions are largely independent, and that is as it should be;  the ideological proclivities of the publisher and editorial board should not be shaping what counts as or is reported as news. 

To be sure, the editorial page of a newspaper should express a point of view, and a typical reader will likely agree with some viewpoints, and disagree with others.  But it’s a very dangerous thing when the editorials of a newspaper are not informed by the daily reporting of its journalists.  Ignoring the news, reported with a minimum of spin by “beat” reporters, leads to simple-minded and ignorant editorializing on complex matters of public policy.  It’s also insulting to the profession of journalism, and to the many reporters whose goal is simply to understand the news and get the story right.  (I talk to some of the reporters to whom I’m referring.)

A case in point is yesterday’s Daily News editorial, “Truth in testing.”  The editorial is an effort to shore up claims about the success of school reform in New York City under Mayor Mike Bloomberg and Chancellor Joel Klein.  Last week’s revelations that the state testing system was dramatically overstating student growth and the closing of the achievement gap rocked the New York City Department of Education on its heels.  The Daily News editorial board, which has long supported these reforms, came out firing, citing four “facts”:  (1) The State Education Department defrauded parents and students;  (2) Regents Chancellor Merryl Tisch and Education Commissioner David Steiner owned up to the deception;  (3) The drop triggered bogus charges that the schools have made no progress;  and (4) Only radical action will give New York’s kids a shot at the quality education they need. (more…)

New York State is releasing the results of the 2010 state assessments in reading and math tomorrow.  We’re told that the 2010 tests were more difficult than those in previous years, and less predictable, the first steps towards a new assessment system that provides a realistic picture of student proficiency.  Testing experts such as Dan Koretz, Jennifer Jennings and Howard Everson presented evidence to the Board of Regents that being judged proficient on the state’s tests in grades three through eight or on the Regents exams did not always predict later success in high school or in college.  This evidence strongly suggested that the threshold for proficiency was set too low;  students who were classified as proficient in eighth-grade math had only a 30% chance of earning a Regents score of 80, which many colleges in the state judge to be the bare minimum for college readiness, had a high chance of scoring below 500 on the SAT, and were likely to be placed in remedial classes if they entered college.  And, based on this chart prepared by the NYC Department of Education, of uncertain provenance, a student who is at the minimum threshold for proficiency on the eighth-grade tests has only about a 55% chance of earning a Regents diploma in high school, the state’s minimum standard for high school graduation for all students who entered 9^th grade in 2008 or later.

 Last week, the Board of Regents voted to adjust the cut scores that determine proficiency on the state’s readingand math assessments in grades through eight.  They didn’t say by how much, but we have a clue from Merryl Tisch’s assertion that the “inflation rate” on the state tests has been about 20% in recent years. Twenty percent of what is not clear.  But I’m going to assume that the cut score for Level 3, which represents proficiency in a subject at a particular grade level, is going to rise substantially at all grades for both reading and math.  What are the likely consequences?

 We’ll see tomorrow, but here’s my prediction, focusing on eighth-grade math.  First, I’m assuming that the distribution of scale scores for 2010 will be the same as it was for 2009.  If the tests were more difficult in 2010, the average scale score might go down a bit;  if students were actually learning more in 2010 than in 2009, the average scale score might go up a bit.  For my little prediction exercise, I’m assuming that these two things cancel each other out. (more…)

Charles Murray is a very confused guy.  His op-ed piece in today’s New York Times uses the dreary impact of the Milwaukee Parental Choice Program on student achievement to justify policies expanding school choice.  Let’s get over the fact that school choice plans don’t show big impacts on students’ performance on standardized tests, he argues.  After all, we’ve known for a long time that it’s hard for schools to overcome the family advantages of cognitive ability and motivation.  Rather, he proposes, we should support school choice because it can allow a small number of parents to choose a curriculum that’s better than that offered to students in traditional public schools.

Setting aside some of the most remarkable inconsistencies—Charles Murray, 2010 edition, doesn’t think that test scores are meaningful measures of academic performance?  Has he met Charles Murray, 1994 edition, who was quite comfortable in The Bell Curve reducing the whole of human intelligence to a single score on the Armed Forces Qualification Test?—Murray fundamentally misunderstands the historic logic of the charter schooling movement—an exchange of autonomy for accountability.  We can argue over the scope of that autonomy and accountability, but even those who have disagreed on this site about whether charter schools are properly labeled as public or private schools generally agree that it’s appropriate to hold them accountable for their students’ performance on assessments measuring standards that are the de facto public curriculum of the state in which they are located.  Certainly, the charter movement gains energy from studies showing that students in charter schools may outperform their counterparts in traditional public schools on state assessments.  Charter schools may strive to expose students to a curriculum that’s more ambitious, but the standards of the state cannot be ignored. (more…)

Writing in the pages of today’s New York Post, Marcus Winters, Senior Fellow at the Manhattan Institute, argues that charter schools might improve the chances for Black and Hispanic students to enter New York City’s prestigious exam high schools.  The key evidence for this is the fact that 2.4% of the Black and Hispanic eighth-grade students who attended charter schools in 2009 were offered admission to the eight exam schools, compared to 1.5% of the Black and Hispanic eighth-graders attending traditional public schools.  Comparing these rates, he states that Black and Hispanic eighth-graders in charter schools are 60% more likely to obtain a seat in the exam schools than their counterparts in traditional public schools.

It’s true that 2.4% is 60% more than 1.5%.  But both percentages round to the same whole number, 2.  So it’s hard to say that the likelihood of admission is dramatically different for students in charter and traditional public schools.  And, although Winters pays lip service to the notion that these data are solely descriptive, there’s no mistaking his desire to use these data to argue that the quality of charter schools is in fact responsible for this small increase.  “Charter schools could,” he writes, “increase minority access to the city’s esteemed high schools by offering a higher quality elementary and middle school education than is available in the traditional public schools system.”  Yep, that’s true, they could.  They could also be successful in recruiting some talented minority students with families that are highly motivated to help them succeed in school.  In the latter case, the primary dynamic is selection into charter schools, not their academic consequences.

By focusing on the relative rates of minority access to New York City’s specialized exam high schools for students in charter and traditional public schools, however, Winters has buried the lead.  The real story here is the fact that, in a system that is overwhelmingly made up of Black and Latino students, very few are getting into the most prestigious high schools.  71% of the eighth-graders in New York City’s traditional public schools are Black or Latino, but only 16% of the students offered seats in the specialized exam schools are Black or Latino.  Another way of representing the same information is to look at the probability of admission to the exam schools for members of different racial/ethnic groups.  As Winters noted, 1.5% of the Black and Latino eighth-graders in traditional public schools were offered admission to the specialized exam schools.  But 19% of the white and Asian eighth-graders attending such schools scored high enough on the entrance exam to be offered admission. (more…)

GothamSchools Editor Elizabeth Green’s cover story in the March 7th edition of the New York Times Sunday magazine tackled the problem of preparing teachers for K-12 classrooms in the United States.  Embellished with the provocative title “Building a Better Teacher,” Elizabeth’s piece profiled two approaches to teacher preparation:  a grassroots approach emerging outside of the academy which focuses on a set of techniques that teachers can use to increase learning time and improve learning environments, and a research-based approach developed in colleges and universities emphasizing the knowledge and skills that enable teachers to teach particular school subjects effectively.  Elizabeth’s story opened with a description of Doug Lemov, who has developed a taxonomy of 49 instructional techniques that he and others believe are critical to effective teaching, and especially to closing the achievement gap between poor, minority children and their more advantaged peers.  If we were to judge the relative merits of the two approaches based on the amount of ink devoted to each in her article, we’d conclude that, in the battle for the minds of education policymakers and practitioners, classroom management (i.e., Lemov’s taxonomy) had won, and pedagogical content knowledge (i.e., the work of Deborah Ball on Mathematical Knowledge for Teaching) had lost.

The disproportionate emphasis on Lemov’s approach in Elizabeth’s article surprised me.  To be sure, he’s a fine human-interest story, and the schools he works with have shown remarkable performance on state achievement tests.  But Elizabeth briefly acknowledged the lack of a research basis for Lemov’s approach, writing:  “And while Lemov has faith in his taxonomy because he chose his champions based on their students’ test scores, this is far from scientific proof.  The best evidence Lemov has now is anecdotal…”  Why would she and the Times choose to feature an approach with so little evidence to back it up?

Lemov’s book, “Teach Like a Champion:  49 Techniques that Put Students on the Path to College,” was published two weeks ago, and currently ranks #30 on Amazon’s bestseller list.  I wanted to see what he had to say about the research evidence underpinning the techniques.  A thin research base does not, of course, mean that the techniques are not valuable—I expect to learn quite a bit from studying them, and seeing if there are opportunities to adapt them for teaching my graduate students (who will tell you that classroom management is not my strong suit.)  And, of course, who wouldn’t want to be a champion teacher?  Because it is, after all, a competition, right?  (more…)