GothamSchools — daily independent reporting on NYC public schools

Teachers College professor Celia Oyler has posted a quiz about the NYC school progress reports, featuring true-and-false, multiple choice, and even essay questions:

From a psychometric point-of-view, New York State achievement test scores offer a reasonably adequate tool to measure progress of one learner from one year to the next. TRUE FALSE

Explain why you agree or disagree with the following statement by Chancellor Klein: School grades “are giving parents and the public clearer information than they’ve ever had before about the strengths of their schools. They have also become a tool schools use to pinpoint the specific areas where they need to improve.”

If you took last year’s quiz, compare your scores and see if you’ve made any progress. (Via Eduwonkette)

“Has he ever spent time with any five-year-olds?” said one parent at yesterday’s demonstration against testing, refering to Chancellor Klein’s plan to test children in kindergarten through second grade. Schools piloting the plan can choose among several testing options, including assessments based on teacher observations and written tests of up to 90 minutes. Parents and community leaders questioned the developmental appropriateness of such tests, and expressed concern that schools are too focused on testing and test preparation, Edwize reports.

Weprin and his son with Mayor Bloomberg at a parade last year.

State Assemblyman Mark Weprin spoke at the rally, saying that his 8-year-old son has learned some startling lessons about tests:

Recently his son told him confidently that if he ran out of time on a test he’d just check off the C answers on all the rest of the questions. Why? Weprin asked. “Statistically C is most often the right answer,” the child told his dad. “I know what millions of other parents know,” Weprin said. “We are spending too much time on testing and test prep. And it’s not just teaching to the test. I mean cheating.”

And it’s not just children who learn to game the tests. (more…)

Between city- and state-mandated assessments, New York’s 3rd and 4th graders average a standardized test every 13 school days, and 8th graders every 14 school days, says Public Advocate Betsy Gotbaum. And that doesn’t include teacher-designed tests and quizzes used to guide instruction and compute report card grades, or non-mandated practice tests given by science and social studies teachers to prepare students for standardized tests in those subject areas. With all this testing, when do teaching and learning occur?

Example question from NY Intermediate Level Science exam sampler.

This year, the city is rolling out new science materials for grades 5, 7, and 8, building on the curriculum introduced last year in grades 3, 4, and 6. Yet new tests based on the curriculum have been delayed for the second straight year, the Post reported yesterday.

A 2004 report by the City Council Committee on Education stated, “The most striking aspect of science in elementary schools is how rarely it is taught. Students are fortunate to get 45 minutes of science once a week for half the year.” The report made a number of recommendations for recruiting highly-qualified science teachers, increasing the profile of science education, and holding schools accountable for science.

In response to this and other reports that accountability in reading and math was pushing aside science and social studies instruction, the city introduced its new scope and sequence for science, based on state standards. Schools across the city select from a kit-based approach, a textbook-based approach, or a “blended” model which combines the textbooks and kits, or they may use approved alternatives. Yearly testing based on the curriculum was supposed to push school administrators to increase time spent on science and support teachers’ implementation of the new curriculum.

The delay in introducing the new tests poses a catch-22 for teachers fighting for attention, time, and resources for science education, but hoping to avoid the pressures and pitfalls of yearly standardized testing. Although many educators and students are undoubtedly relieved to avoid adding another exam to the already-full assessment calendar, others see the test as necessary to raise the profile of science education. At an August 2007 professional development workshop related to the new curriculum, some science teachers reported that their principals said they’d increase time for science once science tests started to matter for school accountability.

Many teachers are also waiting to see what the tests emphasize. Will they focus more on content, reasoning skills, or laboratory skills? The state science exams currently given in 4th and 8th grade include multiple choice, constructed response (short answer), and performance (lab-based) sections. What will the new tests look like? (more…)

Yesterday, LA, Denver, and Houston. Today, Baltimore, DC, Chicago. The tour continues…

First stop, Baltimore. Maryland School Assessment test results – proficiency levels only – are available in a giant PDF report. But the state DOE saves the day with a data navigator that lets you check off groups you’re interested in and view graphs of proficiency data based on your choices. Two screenshots should give you a sense of the range of data available here.

Screenshot of the Maryland Report Card data tool.

Screenshot of county-level demographic data.

(more…)

Mind the gap, by Marcin Wichary

The internet has seen a flurry of activity recently over the DOE’s claim that it has reduced the achievement gap between black and Hispanic students and their white and Asian peers. Testing that claim, the New York Sun submitted the ELA and math scale score data for students in grades 3-8 to three independent analysts, who concluded that the gap has decreased in ELA, but has stayed flat since 2002 in mathematics, confirming much of Eduwonkette’s analysis.

The new analysis emphasizes the difference between closing the proficiency gap by comparing the percentage of students who score at a level 3 or 4 on state tests, and closing the achievement gap by comparing mean scale scores.

(more…)

The story of the Ohio principal who is apologizing for his school’s focus on testing made the rounds of education blogs last week, but I wanted to take another look at it. In particular, I want to point out that while most of the criticisms made by the principal, David Root of Rocky River Middle School, are valid, one throwaway point at the end of the Cleveland Plain Dealer column is not. Unfortunately, education reformers who advocate increased testing have used that point to justify their agenda, and Root would have been wise to consider how they might receive his apology.

Most of what Root told the columnist about the effects of testing in his school struck me as real, serious, and disturbing. Among many other offenses, he apologized for not suspending a student who assaulted another student during testing days; for not reporting absences because they would count against the school; for reducing art, music, and physical education in his school; and for “arranging for some students to be labeled ‘at risk’ in front of their peers and put in small groups so the school would have a better chance of passing tests.” It’s enough to send even the most moderate education observer running to join a Time Out From Testing protest.

But I was put off by the end of the column, when Root expresses nostalgia for the days before testing. “We don’t teach kids anymore,” he said in the column. “We teach test-taking skills. We all teach to the test. I long for the days when we used to teach kids.”

Testing might not be the answer — and certainly here in New York City, where independent analysis shows that increased testing hasn’t narrowed the achievement gap at all, it’s not — but neither is nostalgia. For all of its faults, No Child Left Behind requires states to examine the performance of all of their students, and for many states this attention is new and greatly needed. The good old days when schools just taught without high-stakes testing were characterized by a level of failure, particularly among minorities and poor children, that no one should accept. I’ve heard Chancellor Klein cite this history as proof that his reforms, which include frequent testing, both with and without stakes, are necessary. Why lend him and similar reformers support when the arguments against their style of testing program are so strong?

Be on the lookout tomorrow for a GothamSchools look at schools’ real good old days.

After some back and forth between bloggers and the DOE press office, NYC has released scale scores and standard deviations broken down by race for the past seven years of English Language Arts and Math tests. In Eduwonkette’s analysis, they show that the racial achievement gap in the city has increased during the Bloomberg administration, and in 8th grade ELA, the one area where the gap has decreased, it’s because white and Asian scores have declined.

This note on the spreadsheet, coupled with concerns that the tests may have gotten easier, makes you realize just how tricky it is to get a clear picture of how the kids are doing:

As of 2006 the New York State Education Department expanded the ELA and mathematics testing programs to Grades 3-8. Previously, state tests were administered in Grades 4 and 8 and citywide tests were administered in Grades 3,5, 6, and 7. State tests at Grades 3-8 include both multiple-choice and extended response questions. Citywide tests were composed of multiple-choice questions only. As a result of the changes in the testing program, scale score results from 1999 to 2005 cannot be compared with scale scores from 2006 to 2008 because the state changed the scale scores and its corresponding ranges with the introduction of state tests in ELA and math in grades 3-8.

Personally, I’m hoping for some visuals to help bring the numbers to life… (hint, hint).

test books by menlophoto

From the Washington Post, a glaring example of why it’s so important for educators, parents, and concerned citizens to turn a critical eye on education reporting, especially reporting about test scores:

Today, the paper ran a story about across-the-board improvements in test scores in Maryland’s Anne Arundel County, where reading test scores increased by 4 percentage points, to 86 percent proficient, and math test scores increased by 3 percentage points, to 84 percent proficient. Sounds pretty good, right? School leaders attribute the gains to the fact that they broke down test score data to focus on the particular skills individual students needed to improve and to teachers’ increased cultural sensitivity to how children learn. The Post reporter takes her sources at their words, writing, “Countywide results reflect that effort.”

But there is more to this story, and much of it appeared in the Post last week, in an article about statewide test results and the skepticism with which they have been received. First, Anne Arundel County’s scores may have increased, but they didn’t keep pace with the average gain in Maryland — statewide, reading scores increased by six percentage points, and math scores by four percentage points.

In addition, the Post reported just last weekend that Maryland test was shorter this year than in the past and that the state dropped its practice of drawing some test questions from a national exam. That important information didn’t make it into today’s article. A shorter test could have reduced fatigue that might have inhibited students from performing at their best in previous years. And because the test contained fewer items, the results should be considered less reliable than in past years. Finally, by including only questions devised by Maryland teachers in line with the state’s standards, test makers increased the likelihood that students would be able to answer more questions. Together, these changes could account for much, or even all, of the rise in test scores, but not surprisingly, state education officials are uninterested in examining whether that’s true.

Students in Anne Arundel County — home of the elementary school profiled in Tested, Linda Perlstein’s expert 2007 book about the effects of high-stakes testing on students, classrooms, and schools — might be doing better than they ever have before. But the county’s test scores don’t tell us that, and neither does the Washington Post.

Yesterday, I took an initial look at the Manhattan Institute’s study, “Building on the Basics.” Today, I want to look at Florida’s state science exam, the focus of the study. A common criticism of standardized tests is that they all, to some degree, test reading ability. What does the Science FCAT look like? What skills would you need to perform well on it? I’ve only seen the NYS Science exams, so I decided to download a Florida sample test and take a look. The first thing that surprised me about this test was the reading level, which seemed high. Many of New York City’s fifth graders would (for better or for worse) stumble over sentences like, “Florida has many limestone caves containing formations called stalactites.” I tracked down a site of readability analyzers and entered text from test items.

Question 1: Melissa’s school rings a bell to alert students that it is time to start class. When the bell rings, it vibrates. The use of vibrations to send messages is an example
of which type of energy?

This one ranged from 4.72 to 10.07 in estimated US grade level required to understand it, which certainly calls into question the reliability of the readability analyzers, but also the ability of average 5th graders to understand this question.

(more…)