GothamSchools — daily independent reporting on NYC public schools

Most students taking this week’s state reading test are doing so under the watchful eyes of their regular classroom teacher. Teachers proctor their own students’ exams in most schools, in an arrangement that is logistically simple and keeps students calm  — but also represents a soft spot in the state’s efforts to prevent cheating.

As part of its recent efforts to safeguard against fraud, New York State has reduced educators’ access to tests before they are administered and increased scrutiny on tests after they are returned to see whether answers were changed unusually often. The latter measure, known as erasure analysis, helped investigators uncover adult cheating in Atlanta, Philadelphia, and Washington, D.C., in recent years.

But even as the state has taken steps to prevent improprieties at a time when ensuring that scores accurately reflect student performance is increasingly important, it has left proctoring relatively unregulated. Erasure analysis and pre-test security can’t reveal whether students were advised to check their work on specific questions or, more egregiously, were actually given the answers while they took the tests.

“Test administration with educators proctoring their own students is one of the weak links in the testing process,” said Greg Cizek, a professor at the University of North Carolina who specializes in educational measurement and test security. (more…)

An independent research group with access to a trove of the city’s education data concluded that most of the Bloomberg administration’s claims of high school progress are credible.

But in a different report commissioned by a nonprofit group that manages some city high schools, researchers found that the city’s tools for evaluating schools do not treat schools with higher-need students fairly.

The two reports come as the Bloomberg administration concludes a three-term spree of policy changes meant to spur improvement in the city’s high schools. The spree included dozens of school closures and the creation of hundreds of new high schools, along with accountability metrics such as the annual “progress report” to make school performance transparent. Whether to continue the policies and accountability measures will be a major choice facing the next mayor. (more…)

Minutes after the close of business hours today — a summer Friday already packed with education news — the city released the first set of required reports about students who left middle school and high school last year without graduating.

Some students leave their schools for good reasons, such as when their families leave the city. But others are dropping out.

In 2011, an audit by the state comptroller found evidence that the city might have underreported its dropout rate by classifying many dropouts as “discharges,” the term for students who have provided good reasons for leaving school and evidence to support their explanations. The audit followed a 2009 report by a researcher and an advocate that suggested that the city was increasingly exploiting the reporting loophole to inflate the graduation rate.

Alarmed by the reports, the City Council took up the cause and a year ago passed a local law requiring the Department of Education to report annually on how many students leave school and why. The first reports were due today. (more…)

When Education Secretary Arne Duncan urged the country’s education commissioners this summer to ensure their standardized tests were as secure and reliable as possible, he specifically recommended four measures that would help them do so.

Here in New York State, officials for the most part heeded his advice. Last week, Commissioner John King’s proposal to upgrade testing and scoring procedures included three of the four measures.

But state officials ignored one Duncan recommendation: to conduct “unannounced, on-site visits during test administration.” That raised a red flag for Kathleen Cashin, a member of the Board of Regents who supervised schools in Brooklyn and Queens for many years.

“That is a preventive way, if someone is thinking of cheating, they might think twice if they knew someone was in the building touring,” Cashin said at last week’s Board of Regents meeting.

Principals and teachers report they rarely or never see test monitors in their schools, but it wasn’t always that way. (more…)

ALBANY — Members of the Board of Regents today endorsed an independent review of the state’s procedures for investigating cheating.

The independent review is a first step in a complete overhaul of the state’s test security procedures that a State Education Department task force recommended last week. The Regents are reviewing the recommendations at their monthly meeting today and tomorrow.

Today’s vote to pursue the independent review came from the P-12 Committee, which supervises education from preschool through high school. With the committee’s endorsement, the measure is expected to pass easily when the entire board votes on it tomorrow.

Approval will trigger an “immediate” review, just as soon as the state finds an entity to conduct it. Education Commissioner John King said the state would look for entities to participate in the review at low or no cost to the state.

That review is likely to generate ideas for how SED can expand its investigative arm, which officials characterized as muddled. (more…)

The first recommendations of the state task force to boost test security are out, and they suggest that big changes could be coming to the way tests are administered and graded.

Next week, the Board of Regents will vote on a measure to start an immediate, independent review of how the state handles allegations of cheating.

No action is set yet on the rest of the recommendations. But they provide a blueprint of what the state might do to prevent cheating scandals like those that have gripped Atlanta, Philadelphia, and other cities.

To improve the current system, the state could prohibit teachers from proctoring their own students’ exams and even exams in the subject they teach; bar teachers from grading their own students’ exams, as many currently do; and keep completed exams on hand for longer than a year so they can be checked if cheating is alleged, the recommendations say. The Regents could turn those recommendations into official policy as soon as next month.

But a more substantive revision of the testing system would be even more secure, the working group concluded. The task force wants permission to sketch out — and cost out — a centralized, statewide scanning system that includes erasure analysis and other measures to check for irregularities in test results.

City officials say they support the changes — as long as the city doesn’t have to foot the bill.

“We applaud the state for proposing to centralize and strengthen security on its exams,” Chief Academic Officer Shael Polakow-Suransky said in a statement. “Their proposals make a lot of sense, provided the costs are not passed on to districts like New York City, where we now spend more than $20 million a year to score state exams.” (more…)

Board of Regents Chancellor Merryl Tisch said she believes the city’s schools have improved, but urged the Department of Education should do more to prove that its test scores are “bulletproof.”

Tisch made the comment at this morning’s City Hall News and GothamSchools “On Education” panel: ”I think the city has an obligation to show the public that what they’ve done here is real,” she said, noting that she had “had conversations with the city on this issue.”

Department of Education Chief Academic Officer Shael Polakow-Suransky, who also sat on the panel, defended the department against the suggestion that cheating is widespread, noting that investigations substantiate very few cheating allegations. He said that the department plans to release a more complete accounting of its internal investigations into cheating allegations, similar to the one released earlier this week by the Special Commissioner of Investigation, an independent office.

But Polakow-Suransky also said that more could be done to tighten test controls and that the city “would welcome more scrutiny.” He told GothamSchools that the city has offered to chip in to help the state create a “distributed scoring system” whereby students’ Regents exams could be electronically sent to other schools to be graded by teachers with no relationship to them. Currently, teachers grade their own students’ exams.

That system would be the best option for preventing teachers from changing exam grades, Polakow-Suransky said, but the cost — which officials pegged as high as $20 million — is too much for New York City to undertake alone.

“I think the state has an obligation to pay for that,” he said. “We’ve offered to help with some of our Race to the Top money, and we’re looking into some models that we can begin to test in hopes that they will take it over statewide. That’s the real solution to this.” (more…)

Three months after announcing that he would take a taxpayer’s suggestion and audit the Department of Education’s online data system, Comptroller John Liu is asking the system’s most frequent users for feedback.

Liu announced in March that he would audit the Achievement Reporting and Innovation System, the department’s data warehouse known as ARIS, which has attracted no shortage of critics for its $81 million price tag and early glitches. In June, Liu’s office distributed a satisfaction survey to some ARIS users, including teachers.

“As part of the audit, we are evaluating whether the system meets users’ needs,” read an email containing the survey sent June 14 by Vince Liquori, director of financial audit in the comptroller’s office. A high school teacher who received the survey sent it to GothamSchools after the deadline to complete it, June 24, had passed.

The 21-question survey asks respondents for details about how they use ARIS and whether they think they system is helping them boost student achievement. The survey also includes a free-response section for respondents to list what they like and dislike about the system and identify which of its features they would change.

The survey comes as ARIS continues to contend against lower-budget competitors for teachers’ attention. (more…)

The City Council is requiring the education department to provide more transparent reporting to support claims for two of its signature achievements: higher graduation rates and fewer failing schools.

In the midst of finalizing next year’s city budget, the council managed to pass two bills that target the Department of Education’s bookkeeping. One of them requires the department to disclose more detailed information about students who leave the system without graduation. The second mandates the release of information about students who do not graduate when their high schools close.

Under the first bill, the DOE will be forced to provide more detailed data about student discharge rates, which critics say is overused by schools in order to inflate graduation rates. In 2009, Leonie Haimson, of Class Size Matters, released a report that found discharge rates steadily climbed since 2000. That prompted a state audit that concluded the dropout rate was in fact higher than claims made by the DOE.

Out of 88,612 students from the 2004-2008 cohort, 19 percent – or 17,025 – were discharged and 10 percent – or 9,323 – dropped out, according to the audit.

“This bill will for the first time allow us to know what happened to the thousands of students every year who are discharged from high schools,” Haimson said. “It will make it possible to see if they’re honestly reporting discharge rates. (more…)

Principals who worried that new, toughened state math and English exams would hurt their performance reviews had good reason: Far fewer principals earned high marks from the city last year.

Data on principals’ performance ratings, which GothamSchools obtained through a Freedom of Information Law request, show that the number of principals who “substantially exceed” expectations fell by roughly 60 percent from 2009 to 2010. (A full list of all principals and how they scored is at the end of this post.)

The decrease parallels a drop in test scores and fewer schools earning “A” grades on their progress reports. The percentage of elementary and middle schools to get A’s on their city-issued report cards fell from 84 to 25 percent — a drop precipitated by more students failing the exams and the city grading schools on a curve.

With fewer principals earning the city’s highest rating, more fell into the middle. Principals can earn one of five ratings: does not meet expectations, partially meets, meets, exceeds, or substantially exceeds. The number of principals rated as “exceeding” expectations rose from 465 to 608 and the number who “meet” expectations climbed from 114 to 376.

The number of principals earning substandard marks also rose. In 2009, only five principals were rated “does not meet” expectations, but that number more than quadrupled to 21 in 2010. Even with the increase, the percentage of principals earning the lowest rating is now only 1.4 percent of the 1449 on the city’s list. (more…)