The City, more than the State tests, but both show variation in difficulty year to year. Instead of comparing grade 4 2005 to grade 4 2006 to grade 4 2007… look at grade 4 2005, grade 5 2006 and grade 6 2007… much too bumpy. And the bumps match those in a comparison of other cohorts as they move from 2005 to 06 to 07.

In other words, I’ll test (as time allows) not the idea that the tests are getting easier, but rather the idea that the difficulty changes each year by amounts that overwhelm any real differences.

If someone else wants to do the work, set the cohorts by creating a new column on the spreadsheet, and subtracting the grade from year, and then sort on the new column.

If you follow NYC schools, here’s a new must read blog for you – Gotham Schools. When the Open Planning Project lined up two of NYC’s most talented education bloggers – Philissa Cramer (formerly of Inside Schools) and Kelly Vaughan……

Love the site!

Re this, “scale score results from 1999 to 2005 cannot be compared with scale scores from 2006 to 2008 because the state changed the scale scores and its corresponding ranges with the introduction of state tests in ELA and math in grades 3-8″: this is not a problem for comparing *gaps* but a potential problem for comparing the raw scale scores. Thus far, no one has presented a compelling argument about how rescaling would, for example, differentially affect the scores of black versus white students; only in such a case would the comparison of gaps over time be a problem. It is also worth noting that the DOE has no problem with comparing proficiency rates over time, but has put forth all of these caveats about scale scores – most likely because they reveal what’s really been going for the last 5 years.

What is reasonable, however, is that comparing the earlier city tests with the later state tests is likely a problem, so best to look at grades 4 and 8 for now…