GothamSchools — daily independent reporting on NYC public schools

What does it mean for tests to get easier? And is that really what happened to New York’s tests?

The analysis that has spurred that idea in the last few weeks actually found something slightly different. The tests aren’t necessarily easier, in the way that a kindergarten spelling bee is easier than the SAT. Instead, between 2007 and 2009, students who hadn’t learned much came out looking like they had.

This is an important distinction because it points to a different culprit behind the dropping standards than simply the individual test items themselves. Instead, Harvard professor Daniel Koretz – the lead author of the analysis commissioned by the state education department — names two possible causes: a phenomenon called “score inflation” and a possible psychometric error tied to an obscure state law.

The actual questions on the test play a role in both, but just as important is the practice of prepping students extensively for tests. Another key is a state law that forces New York to release all test items publicly, making it easier for teachers to practice test prep and making it harder for officials to keep tests consistent over time. (more…)

State Board of Regents Chancellor Merryl Tisch is calling for state exams to be more “defensible,” but a study investigating test score credibility requested a year ago by the state’s testing oversight board has still not received a go-ahead.

The committee first formally asked the state education department to join an academic study on the state tests in the fall of 2008, said chair Howard Everson. The education department declined but did not rule out future participation. Since then, Everson has received no requests to revisit the idea, he said in an interview yesterday.

“It’s hard to trust the data right now,” said Everson, a psychometrician who is also a senior fellow at the City University of New York. Everson’s committee, the state Technical Advisory Group, is charged with monitoring the state testing process. (more…)

State school officials will announce the results of this year’s state English tests this morning, at 11 a.m. To watch the news conference live, either begin driving to Albany immediately or visit one of these Web sites, depending on which video player you have:

Windows Media

Real Player

Chancellor Joel Klein will give his interpretation of the city’s results at a separate press conference at Tweed Courthouse this afternoon, at 1 p.m. Come back here for the latest intelligence.

And, while you’re waiting, consider obtaining a big fat grain of salt, bigger yet given that it’s an election year, and political opponents of Mayor Bloomberg’s are zeroing in on his claims of improved student performance. Last year, New York City scores shot up to historic highs, and Bloomberg and Schools Chancellor Klein said the improvements were evidence that they are succeeding at turning around the public schools.

But testing experts said the increased scores on state tests didn’t necessarily mean that students were learning more than students in the past. The experts pointed to relatively smaller gains — and in some cases, no gains at all — on the national benchmark test, the National Assessment of Educational Progress, plus a phenomenon called “score inflation,” in which practices like test prep make test scores rise faster than actual learning.

One highlight of the mayoral control panel put together by the parent commission Friday night was testimony by Robert Tobias, the former city testing czar and now New York University professor. Tobias has often been quoted expressing concerns that the Bloomberg administration inflates its record of educational improvement.

But the analysis Tobias presented Friday, explaining exactly what progress he thinks happened (“real” improvements in math) and what he thinks did not (any narrowing of a longstanding gap between the state and city students’ scores on reading tests), was the most succinct summary I’ve ever heard him deliver — not to mention a striking counterpoint to the sanguine evaluations of Chancellor Joel Klein, Mayor Bloomberg, and even Caroline Kennedy.

Here’s what Tobias said:

Tobias also tempered the fact of the improvements in math scores with a warning about score inflation, the phenomenon by which test-prepping, in his words, can “undermine” the meaningfulness of the test as an indicator of what students know, versus how well they have been prepped. (Harvard Graduate School of Education’s Daniel Koretz has written the most on score inflation that I know of. For more on the topic, see this story I wrote for the Sun and these posts by Eduwonkette.)

Tobias’s remarks on score inflation are below the jump. Thanks to David Bellel for sending me the video. (more…)