GothamSchools — daily independent reporting on NYC public schools

Schools Chancellor Joel Klein defended the city's results on the NAEP math exams this afternoon at Tweed Courthouse.

Frustrated with criticism that city students made no progress on a national math exam in the past two years, Chancellor Joel Klein instead focused on a shrinking performance gap separating city students from their state and national peers today.

Speaking at Tweed Courthouse this afternoon, Klein argued that the city has made greater gains in fourth and eighth grade math than the rest of New York State and the United States overall.

City fourth graders improved their math scores by 11 points since 2003, Klein said, compared to a rise of one point in the state and five points in the nation. He pointed to similar patterns in eighth grade scores.  The percentage of students scoring at or above a proficient level also rose faster in New York City than in the state or nation.

Klein said that because other states like Massachusetts have state standards that hew more closely to what is tested on national exams, it is difficult to compare New York City’s results to those of other major urban areas like Boston. The city ranked third out of the eighteen urban districts tested by NAEP in fourth grade scores and sixth in eighth grade scores. (more…)

Fourth grade students' scores were flat this year, but have increased since 2003 and 2005.

City students have made no significant improvements on a national math test in the last two years, but years of two and three-point gains have led to a general trend of modestly increasing scores.

Eighth grade students did not make meaningful gains this year. Reflecting a pattern of fourth-graders outperforming eighth graders, the older students have seen fewer score gains since 2003.

Fourth and eighth grade students’ scores on the National Assessment of Educational Progress, known as the nation’s report card, have been statistically flat since 2007, though both groups have made gains since 2005 and 2003.

NAEP scores are typically released on a state-by-state basis, but in 2002 several large cities agreed to have their own figures reported separately. The data does not include test scores from students in charter schools. Compared to students in other large cities, New York City’s fourth-graders beat the average score, while its eighth grade students’ scores met the average — a pattern that has held constant since 2003. (more…)

A schedule conflict could mean that students’ scores on this year’s state standardized tests may not play a role in whether their teachers get tenure. Nevertheless, if the city does use the scores, it could land in court with the union on the other side.

Citing a loophole in state law, Mayor Bloomberg ordered the city’s Department of Education last week to begin using students’ test scores in tenure decisions this year. But the results of this year’s state math and English tests will not be available until after the deadline for submitting tenure decisions has passed.

The state changed its timeline for administering math and English exams this year, pushing both exams to the spring. Previously, they were given in January and March. Though principals have to make decisions about whether to grant teachers tenure by May 1, this year’s tests will not even finish going through the scoring process until weeks after that deadline.

This schedule conflict could leave principals to make tenure decisions using two years of test scores rather than three, and those could be two easier years. Chancellor of the Board of Regents Merryl Tisch has said that this year’s tests will be “less predictable” than in previous years. (more…)

The New York State Education Department is failing to ensure that Regents tests are properly scored, according to an audit published today by the state comptroller’s office.

The exams are given to high school students, who have to pass five in different subject areas in order to receive a Regents diploma. Teachers normally administer and score the tests under the supervision of each school’s principal, and the school district is responsible for reporting scores to the state.

The audit focused on the review process the state uses to ensure the scoring is accurate and consistent. In these reviews, a group of teachers and NYSED officials re-score a random selection of exams and compare them to how the tests were originally scored to judge the accuracy. The review team then makes recommendations to the state and to schools about how to improve the scoring process.

In the most recent review, completed in 2005, the scores awarded by schools were routinely higher than the scores given by the reviewers, and reviewers reported that school scorers frequently assigned full credit to student answers that were “vague, incomplete, inaccurate or insufficiently detailed.”

But auditors found little to suggest that the state followed up to improve the process, the report says.

“For example, we found no evidence actions were taken to implement the Review team’s recommendations to improve scoring training and enhance quality control during the scoring process. We also found no evidence actions were taken to bring about improvements at particular schools,” the auditors write. (more…)

Reacting to differences between the state’s own testing data and the results of a national math assessment, Commissioner David Steiner called for the state to review and redesign its tests to make the questions less predictable.

“The New York State NAEP scores in mathematics, released today, are of great concern to the Board of Regents and to me,” Steiner wrote in a statement. “We are struck by the contrast between results on the NAEP and on New York State’s own math tests.”

The call from Steiner is the strongest language a state education official has used since critics began challenging the state tests in 2007. (more…)

Here’s one more reason state tests might be getting easier to pass: a longstanding State Education Department practice of publicly releasing every question on each year’s exam.

The unusual practice makes it harder for test-makers to gauge how difficult a test is, said Howard Everson, chair of the state’s Technical Advisory Group, an oversight committee that monitors state testing.

Many states release some test questions but keep others private so they can be used again to compare one year’s test to another’s, said Daniel Koretz, a Harvard University education professor who studies testing. But New York has long had a practice of releasing every single test question to the public soon after students sit for the exams. (more…)

One interpretation of rising pass rates on state math and reading tests is that the tests have grown easier. But are they so easy that students can earn a passing score without showing that they know anything at all?

The answer is yes, at least sometimes, writes teacher Diana Senechal in the GothamSchools community section. Senechal conducted an experiment to test whether guessing alone could yield a 2, the lowest passing score. She writes:

I first tried my experiment with the sixth grade ELA test. I “guessed” all the answers on the multiple-choice portion and left the written portions blank. Or, rather, I didn’t “guess,” but filled in the answers as follows: A, B, C, D, A, B, C, D, and so on, all the way through the 26 questions. I didn’t read one of them. …

… I got a 2 without looking at a single test question or writing a single word.

Read Senechal’s entire column for a more detailed explanation of her experiment and its results.

President Obama’s Department of Education has vowed to invest federal money in building better tests, but the dollars may be held up until the country can hash out some “common standards.” The new Board of Regents chancellor, Merryl Tisch, is also zeroing in on state tests, but it’s not yet clear exactly how that will happen.

Meanwhile, in England, they’re off and running. Computerized assessments staggered throughout the year will replace written end-of-course exams within the next 15 years, a senior testing official, Simon Lebus, told The Guardian over the weekend:

Exam boards are investing millions of pounds in developing the technology – and, Lebus claimed, it’s not “science fiction”.

He said: “The likelihood is that in the next 10 to 15 years it will change almost out of recognition in that by the end of that period of time you’ll be able to do exams more or less on demand, on screen.

“You can make the learning more valid and the technology can enhance the way people engage in the subject. It’s very expensive, complex stuff to do. But it is achievable. It’s not a vision based on a sort of science-fiction type fantasy.”

This Education Sector report, “Beyond the Bubble,” explains how technology can innovate testing.

A Daily News report this week cast doubt on the validity of the state’s math scores. A major problem the News pointed to is that the math tests seem to repeat themselves, broken-record style, making it easy for teachers to coach their students on how to give correct answers — without necessarily understanding the underlying math. A second problem is that the tests may be getting easier over time, the story said.

Here’s a graphical portrait of what this means in practice, courtesy of Jennifer Jennings, the doctoral student at Columbia University whose analysis informed the News’s story.

A math question seventh-graders answered in 2009:

A math question for seventh-graders in 2008:

And finally a question from the same test’s 2007 version, assessing the same concept, but in a much more difficult way: (more…)

Next year, the state’s English tests could be missing one crucial component: writing.

That’s the conclusion that educators are drawing after the Board of Regents weighed a proposal earlier this month to eliminate the open-ended question section of the state’s standardized tests — the only part of the third through eighth grade testing regime that asks students to write out their answers in sentences.

The proposal is one of several ideas the Board of Regents, the state panel that sets New York’s education policy, is considering in order to speed up the test-grading process, following a new federal regulation ordering states to tell schools sooner whether or not they are meeting states standards. (State test scores play a large part in making that decision.) Changing the way the tests are graded could also cut costs.

The Regents have been studying how to meet the new federal requirement for almost a year. The prospect of scrapping writing first surfaced publicly when the Regents published the findings of a survey the board conducted to study the question. Of 22,000 parents and educators surveyed, 85% said the essay questions should remain. (more…)