May 4, 2012
Even when it seemed that everyone had something to say about “The Hare and the Pineapple,” the seemingly nonsensical story that appeared on New York’s eighth-grade reading exam, the company that created the test remained silent.
Now, a leaked memo makes it clear that state officials sought an explanation from Pearson of why the story appeared on some exams — and that Pearson offered a vigorous defense of its test-construction choices.
“The Hare and the Pineapple” and associated items had been field tested in New York State, yielded appropriate statistics for inclusion, and it was aligned to the appropriate NYS Standard,” a Pearson vice president wrote to Kenneth Slentz, the state’s interim testing czar. The story was meant to test students’ ability to interpret characters’ traits and behavior and to “elicit supporting detail,” according to the memo, which Time Magazine published today.
The memo was obtained by Andrew Rotherham, head of the nonprofit Bellwether Education Partners, who said it was given to him by a state government employee. A State Education Department official confirmed the authenticity of the memo, which was signed by John Twing, a Pearson vice president who serves as the company’s “chief measurement officer.”
The memo, which is written in dense test jargon, describes a chain of psychometric decisions that resulted in the “The Hare and the Pineapple” winding up on the test.
The state contracted with Pearson to include “norm-referenced” questions, tested elsewhere, to help illuminate how well students did compared to each other. But the state turned down one set of questions that Pearson typically uses for that purpose because the questions were not in line with New York’s standards. So Pearson cobbled together custom norm-referenced questions and others that it had used elsewhere — including the Pineapple story, which had been used in six states and three large districts since 2004.
Some New York students saw the questions last year as part of field testing of questions for this year’s exam. Students in New York performed identically to students elsewhere, Twing notes.
Twing also defends the questions that earned the most ridicule: the one that asked students to identify why the animals ate the pineapple and the one that asked for the moral to the absurdist fable. He writes:
Finally, the owl declares that “Pineapples don’t have sleeves,” which is a factually accurate statement. This statement is also presented as the moral of the story, allowing a careful reader to infer that the owl is the wisest animal.
To Rotherham — who discloses in his weekly Time column that he has consulted for Pearson in the past — the moral of the saga is that adults should seek to understand state tests thoroughly before lampooning them.
But he also says that test-makers should pull back on maintaining tight controls over who sees exam content. The state tossed the passage amid media scrutiny, saying that having the questions in public view meant they would not be valid when students who missed the regular test took make-up exams.
“Making some test materials public would drive up costs because they couldn’t reuse that material, but it could strike a better balance between helping people understand how tests work and maintaining the integrity of assessments,” Rotherham writes.