Post a comment about the budget cuts at your school on our interactive comment map. more »
GothamSchools — daily independent reporting on NYC public schools
Primary Sources
June 12, 2009
New York’s annual math tests are repeating themselves
A Daily News report this week cast doubt on the validity of the state’s math scores. A major problem the News pointed to is that the math tests seem to repeat themselves, broken-record style, making it easy for teachers to coach their students on how to give correct answers — without necessarily understanding the underlying math. A second problem is that the tests may be getting easier over time, the story said.
Here’s a graphical portrait of what this means in practice, courtesy of Jennifer Jennings, the doctoral student at Columbia University whose analysis informed the News’s story.
A math question seventh-graders answered in 2009:
A math question for seventh-graders in 2008:
And finally a question from the same test’s 2007 version, assessing the same concept, but in a much more difficult way:
Here’s another set of “clones,” as Jennings calls them.
From the 2009 seventh-grade math test:
From the 2007 seventh-grade test:
-
June 12th, 2009 5:25 pm
And some schools still failed these tests? What are they teaching these kids?
-
June 12th, 2009 9:45 pm
B-b-but the improvement in test scores is because Mayoral Control WORKS, doncha know?
Like I said before, why bother “teaching to the test” when you can teach to the answer key (er, um, I mean LAST YEAR’S test.)
Just think, if DOE spent half as much on PR and twice as much on test prep, we might show even MORE grade inflation.
(/sarcasm)
BTW, is it “mass” or “weight” being measured? Inquiring minds want to know.
Next, please note the scale question is on the SEVENTH grade math test. Shouldn’t we be expecting a weee bit more?
Last, am I the only one chuckling over the movie theater question? I mean, isn’t self-congratulatory-metric a DOE specialty?
-
June 12th, 2009 10:06 pm
Of course there are going to be similar questions. These tests are supposed to be created based on the standards set forth by the state, and those standards haven’t changed.
Also, not all of the questions on the 7th grade test are necessarily from the 7th grade. They can be from previous years of schooling. So just because this question is on the 7th grade test it doesn’t necessarily mean that it is a topic they would have covering in the 7th grade.
-
June 12th, 2009 10:21 pm
An annual 7th grade math test stays the same, not a real problem.
A perennial 7th grade student,now that’s a problem. The implicit assumption is a different set of 7th graders annually learning essentially what the 7th graders previously and subsequently should learn.
If you don’t want teachers teaching to the test then do away with high stakes testing. If you want each grade level to master certain concepts and demonstrate them then certain questions will be recycled.For instance adding -15+-5 is exactly the same recycled concept as -14+-4. So do you want to stop all arithmetic questions because they appeared on the test the year before??????
That’s like telling a history teacher not to ask when the Declaration of Independence signed, where, and why because it was asked on last year’s test.
Since I didn’t read the Daily News article or
Ms. Jennings analysis, I can only comment on this abstract and it really seems “much ado about nothing” -
June 13th, 2009 5:31 pm
Why is the Daily News suddenly “discovering” that math tests are redundant? These are the same tests that New York State (and before that, NYC) have been administering since the mid-1980s and the problem is not with the tests, it is with the NEW YORK STATE CURRICULUM for elementary school on which they are based. It’s terrible and not nearly comprehensive enough such that when a standardized test is based on it, this is always the result, BORING, uninspiring, standardized tests. What I disagree with, however, is that idea that my child was “coached” to take these tests. I’m am very outspoken with my son’s school about the amount of time spent on standardized test prep and his school does not waste time on test prep, they spend time teaching. I think a week before the test they prep the kids by telling them that the next week they are going to have it, go over some test taking strategies, etc. This is the same as when I was in school and I’m fine with that. My son’s school is very middle class, so I imagine at poorer schools those kids might need more test prep. Mine does not.
-
June 14th, 2009 7:06 am
Read the apologists’ comments can be so amusing. If there were reports that all failing tests were deleted before being reported, they would say, “There must be a storage problem in keeping all that negative data on the hard drives. Much ado about nothing.”
-
June 14th, 2009 9:46 am
The entire situation is one more reason to discredit this administration but the billionaire publisher Mort Zuckerman, and his felow billionaire publishers as well as fellow billionaire Regent Meryl Tisch are all lined up behind their billionaire neighbor “Mike” who they feel can do no wrong. We are doomed to another 4 years of this tyrrany due to the media’s refusal to look into what is really happening. I only ask that the record reflect that the current eteachers and sueprvisors of the NYC education system who actually work in schools not be blamed when these poor schildren and their families try to move on to colllege and find out they are not prepared because THEY HAVE BEEN LIED TO BY BLOOMBERG AND HIS ALLIES IN THE MEDIA AND THE BUSINESS COMMUNITY. (Do you notice he doesn’t go anywhere without Katherine Wylde who claims to speak for “The Business Community” , Dennis Walcott to add a little color to the equation and then he spits off a few broken and poorly pronounced phrases in Spanish- with an accent that brings to mind Frank Purdue when he tried selling chickens “en espanol” decades ago.
Suddenly every student in NYC is a level 3 or 4. All the parents are happy- and yet SAT scores for NYC students have fallen since King Bloomberg took over with absolute and dictatorial “Mayoral Control”. Bloomberg and Klein trumpet that they have eliminated the “acheivement gap” between white, asian and black and hispanic students, yet if I walk by Stuyvesant High School, a school that refuses to alter its very difficult enterence exam-It looks like you are in Hong Kong. So who are they kidding? Only the poor black and Hispanic communities who will face a rude awakening when our children apply to Ivy League and Tier I colleges only to be told that their SAT scores of under 1000 ( combined) leave them better suited for work at McDonald’s or if they are lucky- a shot at remedial classes in a community college, one probably somewhere in Pennsylvania because they wont be able to afford the rent in NYC as the Mayor is hell bent on rezoning the City for more condo development and gentrification.
-
June 14th, 2009 5:07 pm
Allie,
Stability of the standards is really no excuse for such similar questiions.
* There are far more standards than can tested on a single exam. Tests should not be based on the same subset of those standards each year, but rather should be based on a random (i.e. unpredictable) subset of the standards.
* There are multiple ways to address any specific standards. Question construction and the thought process to come to answer should not be too similar from year to year. If standards are really learned, students would be able to apply their learning to different kinds of questions and to different aspects of the problems they face.
* There certainly is no excuse for the parallel wording of these questions.
This are great examples of the problem of teaching to the test, rather than the standards. Knowing that questions like this will appear on test — as teachers figure after a few years — pushes them to prepare for thiese kinds of questions, rather than the working with students to master the standards as they might be used in a variety of contexts.
The big problem is that we know that this happens all over the country, not just in New York. This is not actually news, per se, to those who pay attetention to testing. The only news here is that the particular examples came from the 2009 NYS math exam.
(This is not to say that Elizabeth should have run this. A local paper ran the story, which is timely even if it is not new, and this post gives great examples of this phenomenon.)
This kind of test contruction encourages inappopriate test preparation. It discourages teaching the standards, underlining the authority of standards settings bodies and shifting that authority to test developers/publishers.
-
June 15th, 2009 6:08 pm
Admittedly I am no science major (I took Physics in HS about 46 years ago), but I remembered that mass and weight were not the same thing. So even before seeing Jacob’s question, I looked up mass to remind myself why this question wasn’t easy for me.
Mass and weight are not the same. Here’s from About.com:
“In most common instances, mass is determined by weighing the object and using the force of gravity to calculate the value automatically - which is why you can get on a scale and have read your mass. . . .
Because of the relationship between weight and mass, these concepts are frequently confused. You can, in fact, convert exactly between weight and mass on the Earth’s surface. This confusion is heightened by the fact that in much of metric world, weight is not dealt with, and mass is used in its place almost exclusively. The main difference is that if you were to leave the Earth and go to the Moon, your weight would change but your mass would remain constant.”And if you need a little more science, this from an online encyclopedia:
“Mass (symbolized m ) is a dimensionless quantity representing the amount of matter in a particle or object. The standard unit of mass in the International System ( SI ) is the kilogram ( kg ).
Mass is measured by determining the extent to which a particle or object resists a change in its direction or speed when a force is applied. Isaac Newton stated: A stationary mass remains stationary, and a mass in motion at a constant speed and in a constant direction maintains that state of motion, unless acted on by an outside force. For a given applied force, large masses are accelerated to a small extent, and small masses are accelerated to a large extent. The following formula applies: F = ma where F is the applied force in newton s, m is the mass of the object or particle in kilograms, and a is the resulting acceleration in meters per second squared. The mass of an object can be calculated if the force and the acceleration are known.
Mass is not the same thing as weight. ”In other words, unless the cheese or the penny is being thrown across the room, we’re not going to be measuring its “mass.”
Perhaps we should be questioning a lot more than just the repetition of questions.
-
July 10th, 2009 1:46 am
Meryl Tisch has publicly stated skepticism about reports of rising math scores. The following post inaccurately implies otherwise:
<>
Leave a Reply
Mapping the Budget Cuts
10 Comments
Subscribe to comments with RSS or TrackBack