GothamSchools — daily independent reporting on NYC public schools

An analysis of nearly 15,000 districts’ test scores turned up suspicious patterns that suggested that some cheating might be taking place in New York City schools.

The analysis was conducted by a team of reporters and researchers at the Atlanta Journal-Constitution, the newspaper that covered last year’s revelations about a far-ranging cheating scandal in Atlanta’s schools. The team looked at changes in students’ test scores from year to year, reasoning that large increases or decreases in groups of students’ test scores would be unlikely without an unusual intervention such as cheating.

The analysis does not identify instances of cheating, only places and times when cheating is considered more likely to have occurred.

Most of New York’s 32 school districts fell well within the normal range, with around 5 percent of classes showing unusually large score climbs or falls. But in a few places, the analysis detected swings in more than 10 percent of classes, a level that experts told the AJC team was highly improbable under normal circumstances.

In Brooklyn’s District 16, for example, 7.95 to 12.82 percent of classes between 2009 and 2011 showed suspicious test score swings. Between 2008 and 2011, the percentage of classes flagged in Manhattan’s District 2, which includes many middle-class students, ranged from 7.41 to 12.5 — significantly higher than in neighboring districts.

The newspaper crunched the numbers for individual schools but published only district-level data online. One of several stories about the analysis hints at school-level findings for the city. It says, ”Though high-poverty city schools were more likely to have suspicious tests, improbable scores also showed up in an exclusive public school for the gifted on the Upper West Side of Manhattan.”

The AJC analysis comes at a time when New York State officials have turned their attention to test security, galvanized by cheating scandals in Atlanta and elsewhere. Last week, the state for the first time appointed a test integrity czar whose job is to lead a team in rooting out cheating through on-the-ground investigations and statistical analysis. But a proposal to fund a sophisticated tool to detect likely cheating, erasure analysis, appears poised to be shut out of the state budget.

City Department of Education officials have long maintained that the city’s test security practices exceed what is required by the state, although the officials have said more could be done — at a steep cost — to ensure that city test scores are sound.

Unlike “value-added” analyses of the type that will factor into New York’s teacher evaluations, the AJC analysis does not control for students’ demographics or even assure that the same students are in the tested groups each year. Gary Miron, an education researcher who reviewed the data, said the analysis was less informative than one conducted last year by USA Today into test scores in Washington, D.C.

“We all need to be concerned about cheating and its implications,” Miron wrote on The Answer Sheet, a Washington Post blog that is frequently critical of school reform efforts. “At the same time, we need to be leery of sensational attempts to secure headlines with weak and incomplete analyses.”

The analysis found red flags twice as often at charter schools, but the district data do not reflect charter schools’ scores.

Eric Gordon, the head of Cleveland’s schools, issued a striking rejoinder to suggestions that cheating had taken place in his district.

“In all candor, if I thought we had a widespread cheating problem in the district, I would expect our achievement to look quite a bit better than it does,” Gordon told the Dayton Daily News.