GothamSchools — daily independent reporting on NYC public schools

Hoxby's study examined 43 charter schools throughout the city. The schools she researched are noted on this map with red stars.

New York City charter school students are performing so well on state tests that they may soon catch up to students in Scarsdale, the upscale suburb north of the city, according to an extensive update of a multi-year charter study released today.

The optimistic projection stems from researchers’ finding that the boost charter schools give does not taper off, but is steady throughout elementary school and middle school and even into high school.

“It seems to be really stable as an effect,” said Stanford University economist Caroline Hoxby, who directed the study.

Hoxby and her team studied 43 charter schools in New York City serving elementary, middle and high school students. They compared students who applied and were accepted into charter schools in 2000 by random lottery to those who applied but did not receive a seat.

By the time charter school students reached the eighth grade, in 2008, they scored on average 30 points higher on state math tests than students who remained in traditional public schools, the researchers found.

That’s almost the equivalent of closing the average achievement gap between students in traditional public schools in Harlem and students in Scarsdale, the affluent New York suburb north of the city where students take the same standardized tests. The average Harlem-Scarsdale math score gap is between 35 and 40 points, so the charter school students close that gap by about 86 percent.

Researchers found that charter schools had closed the Harlem-Scarsdale gap by a smaller but still substantial 66 percent on the state English test. They also scored nearly 3 points higher on average on their high school Regents exams for each year they attended a charter school.

Students who were not accepted into charters by lottery and continued in the traditional public school system continued to score at grade level but did not raise their scores enough to narrow the gap significantly, the study found.

The study also concluded that charter school students were demographically no different than students at surrounding traditional public schools and that the lottery process truly selected students at random. (See demographic details below.)

The study reports aggregated data for New York charters and did not track the performance of students at individual schools.

Researchers have not come to a solid conclusion about whether charter schools are more effective than public schools. Hoxby has published several other studies about charter schools. Her research regularly finds positive effects of school choice.

In an interview, Hoxby addressed many of the criticisms leveled against charter schools in recent years. Her analysis found that lottery admissions were truly random and that in general, students who applied for charter school lotteries shared much in common with traditional public school students who did not seek out charters.

Hoxby did note that parents of students rejected from charter school lotteries are more likely to transfer out of traditional public schools, whether to parochial schools or to schools outside of the five boroughs. “It’s possible their parents may be more motivated or committed to the idea of school choice,” she said.

But Hoxby downplayed criticisms that charters cater to a savvier population of students than the surrounding traditional public schools.

For example, some analysts have concluded that charter schools under-serve higher need populations such as special education students and English-language learners. Hoxby said she does not trust those studies. She thinks the differences might reflect different ways charter schools track special programs like English-language learner services and special education.

Hoxby pointed to figures estimating that charter school students are more likely to be African-American, and far more likely to be poor, than the average New York City public school student, but she noted that these are not final figures and do not include students enrolling in charter schools in kindergarten.

As in 2007, Hoxby estimated that charters schools enrolled just slightly fewer special education students. She also noted that significantly fewer English-language learners enrolled in charters, a figure that Hoxby linked to the proportionately fewer Hispanic students in charter school populations. (Search through special education data comparing charter school students and district students.)

When the first iteration of the study was released two years ago, one of the biggest lingering questions was why charter schools spurred these gains. Hoxby said that her findings could not show any causal relationship between various charter school policies and practices, but a number of school characteristics seemed to be strongly associated with high achievement.

Longer school days and school years, some form of teacher merit pay and mission statements emphasizing academic achievement were all statistically linked to high student achievement.

Hoxby emphasized that there was no guarantee that those factors caused student achievement, or that they were replicable. But that doesn’t mean that other schools cannot take lessons from the charter schools’ success, she said.

“I don’t know if I took a traditional public school in Harlem, and I said to them, ‘you’re going to have a long school year and a long school day,–I don’t know that it would have the same effect,” Hoxby said. “But there’s no reason they shouldn’t try.”

The study presents quite detailed information on charter school demographics and programs; I’ve pulled out some of the more interesting sets of data and presented them below.

Demographics of charter school applicants, enrolled students and traditional public school students: The study found no statistically significant differences between students who received spots through the lotteries and those who did not, confirming that the lotteries are truly a random selection process, Hoxby said.

Prior special program participation of charter school applicants and traditional public school students: Because charters and traditional public schools track special education, English-language learners and free- and reduced-lunch students differently, Hoxby examined students who were already enrolled in these programs when they applied for admission to a charter. Hoxby acknowledged that her sample cannot be truly representative of the charter school population, but said that given its limitations, her data still shed some light on students served by charter schools.

Policies and characteristics of New York charter schools: Hoxby and her researchers found that certain elements of charter schools’ educational programs were statistically linked to gains in student achievement. Programs with longer school days and years, academic school missions and some form of teacher merit pay were found to have the strongest correlation to student achievement. Many elements, including types of math and reading curriculum, were not found to be linked statistically. Programs that reserved seats for parents on charter school boards were found to have a slightly negative correlation, though Hoxby cautioned that association may be more indicative of other management problems.

The full report is below, and can be downloaded alongside the original 2007 report here.

How NYC Charter Schools Affect Achievement Sept2009