GothamSchools — daily independent reporting on NYC public schools

I suppose it was wishful thinking on skoolboy’s part to hope that we could escape a release of new scores on the National Assessment of Educational Progress (NAEP) with just a soundbite from Margaret Spellings, who served as Secretary of Education from 2005 to 2009.  Today’s Washington Post features her op-ed piece arguing that the No Child Left Behind Act (NCLB), which she once referred to as “99.9% pure,” is the reason that test scores are rising.  A corollary is than any significant alteration to the NCLB reforms is a cowardly retreat from “the real accountability that is working in our elementary and middle schools.”

Key to Madame Secretary’s argument is that scores have risen more rapidly in the NCLB era in both reading and math than in the three decades before that.  “Consider,” she writes.  “In the 10 years since 1999, reading scores for 9-year-olds have risen eight points;  in the nearly three decades before that, scores rose only four points.  In the past 10 years, math scores have increased 11 points, while in the nearly three decades prior, scores rose only 13 points.”

I argued last week that the long-term trend NAEP data are a poor basis for evaluating the effectiveness of reforms such as NCLB, in part because the skills tested on them are not aligned with contemporary curricular standards.  (Math, for example, is weighted towards computation;  and although computation is important, we now expect students to engage in more challenging mathematical content in elementary, middle and high school.)  The main NAEP is a better indicator of trends in recent performance over time.

The first figure below shows reading scores in grades 4 and 8 on the main NAEP from 1998 to 2007.  (Keep in mind that the NAEP scale goes from 0 to 500, so this figure may exaggerate the size of the trends over time.)  In the fourth grade, the average score increased by six points between 2000 and 2002, then stabilized, and finally increased by two points from 2005 to 2007.  The 2007 average score is significantly higher than the scores in 2005, 2003, 2002 and 2000.  In the eighth grade, the average score held steady between 1998 and 2002, and really hasn’t budged much since then.  The 2007 average of 263 is significantly higher than the 2005 average of 262, but is identical to the 2003 average of 263, and is actually a point lower than the 2002 average of 264.  The best summary here is that the scores have been essentially flat since 1998.

The second figure shows math scores in grades 4 and 8 on the main NAEP from 1996 to 2007.  In the fourth grade, there was a huge increase of 10 points from 2000 to 2003, and continued gains in 2003, 2005 and 2007.  The 2007 average score of 240 is five points higher than the average of 235 recorded in 2003, and a whopping 14 points higher than 2000′s average score of 226.  The eighth-grade math pattern is similar, although not as dramatic.  Scores increased by an average of five points from 2000 to 2003, and have crept up since then, reaching a high of 281 in 2007.  The 2007 figure is two points higher than the average of 279 recorded in 2005, and three points higher than the 278 observed in 2003.  It’s also eight points higher than the average of 273 found in 2000.

These figures summarize the trends over time in achievement.  But what portion of those trends can be attributed to NCLB?  Margaret Spellings refers to changes since 1999, which is convenient for her story, because there were sharp increases in grade 4 reading between 2000 and 2002, and in grade 4 and grade 8 math between 2000 and 2003.  But NCLB was signed into law in January, 2002;  the first final regulations dealing with assessment were issued in December, 2002;  and initial state accountability plans were approved by the U.S. Department of Education no later than June, 2003.  The 2003 main NAEP was administered between January and March of 2003.  Is it realistic to claim that NCLB affected scores before the 2003 NAEP administration?  I, and a great many other analysts, think not.

If we look at what’s happened since 2003—i.e., using 2003 as a baseline—we see increases of three points in fourth-grade reading;  flat scores in eighth-grade reading;  an increase of five points in fourth-grade math, and a rise of three points in eighth-grade math.  With the exception of eighth-grade reading, these trends are encouraging, and provide some support for the claim that NCLB is working. 

However, part of Spellings’ argument is that scores have been increasing faster in the NCLB era than they were before.  And the evidence here is not as supportive of NCLB.  In fourth-grade reading, scores rose one point from 1998 to 2003, and three points from 2003 to 2007, consistent with the claim of faster growth under NCLB.  In eighth-grade reading, scores declined by a point from 1998 to 2003, and have been unchanged since then-no growth to speak of over the 1998-2007 period.  In math, however, fourth-grade scores rose from 224 in 1996 to 235 in 2003, and this 11-point gain, all of which occurred prior to NCLB, dwarfs the five-point increase observed between 2003 and 2007.  Similarly, the six-point gain in eighth-grade math between 1996 and 2003 is twice as large as the three-point gain we see between 2003 and 2007.  On balance, the evidence does not support the claim that NAEP scores have increased faster under NCLB than before.

Only in Margaret Spellings’ world can NCLB affect NAEP scores for the four years before the law was passed and implemented.  Now that’s wishful thinking.