GothamSchools — daily independent reporting on NYC public schools

preview (updated)

New progress reports shift some weight from scores to grades

For the first time since introducing school progress reports in 2007, the Department of Education has reduced the weight of state test scores in determining middle schools’ scores on their state test scores.

The change is slight, allocating just 5 percent of the calculation toward the grades schools hand out, but it reflects a significant shift within the Department of Education. After years of saying that the state’s current tests are not the ideal measure of students’ abilities, the department is — to a limited extent — putting its metrics where its mouth is.

Until now, 85 percent of elementary and middle schools’ scores have come from crunching the scores in different ways. But on the 2011-2012 progress reports, which are coming out today, that proportion has dropped slightly for middle schools, to 80 percent. The difference will be made up by schools’ course passage rates in the core subjects of English, math, science, and social studies.

The change, which the department promised a year ago, makes year-to-year progress report score comparisons hard to make yet is unlikely to dramatically alter schools’ scores on its own. Still, it signals that the city is projecting onto middle schools growing concerns about the mismatch between how city students perform on some high-stakes accountability metrics and how well prepared they are to take on more challenging work.

The mismatch has fueled a new aim for “college readiness” rather than simply graduation eligibility at the high school level, but it is just starting to have an influence in the primary grades. This year, for the first time, middle schools will get credit for the proportion of students who pass their core classes, considered an important precursor for high school success.

Recognizing that the accountability change introduces a new incentive for schools to hand out, or even mandate, potentially unearned passing grades, the department says it is on the lookout for schools where many students pass their classes but cannot pass the state’s math and reading tests.

“If we find cases where course passing rates are far out of alignment with both state exam performance and state exam progress we may redistribute points from the course metrics to the exam metrics for those schools,” reads a document that principals received last month to help them interpret their schools’ progress reports, which they received in advance of today’s public release.

Plus, the document warns, “The DOE is increasing oversight of schools’ grading polices. Schools may be asked to provide documentation of grading policies for review to justify student course performance results.”

The city has also awarded extra credit to middle schools where many students have passed a high-school level course or exam before leaving eighth-grade. One goal of the city’s 2007 middle school reforms, which quickly lost steam but got new attention after Chancellor Dennis Walcott announced a middle school overhaul last year, was to expand high school-level coursework in middle schools.

Next year, middle schools will also be scored on how their graduates perform in ninth grade, when they are confronted with higher expectations and different grading standards. For now, that data will appear on the progress report, offering a view into middle schools that might have promoted students who were not ready for high school, but won’t be figured into schools’ final scores.

In an education department that’s driven by data, what gets measured is a clear expression of values, and other changes to the progress report signal that the department is placing a new emphasis on the success of individual students rather than simply on schools as organizational units.

In one example, schools have long been eligible for “extra credit” based on how quickly they boost the test-scores of various groups of high-needs students. But in the past, schools have been ranked according to how much extra credit they earned, and only the top 40 percent received any progress report score boost. This year, every school with students who meet the standard for extra credit will earn some, and schools with more high-needs students will get more extra credit.

The city is also giving more weight to third-grade test scores, with a large portion of elementary schools’ scores going to an “early grade progress” metric that weights scores according to students’ demographics. And schools will also be able to earn extra credit for English language learners who advance quickly. Until now, extra credit has gone to schools that move low-performing students and black and Hispanic students forward, but there has not been a separate category for ELLs.

And in another non-graded change, this year’s progress reports will show how students performed in each grade, although the total score will still be based on the aggregated scores. “We have received feedback that reporting this kind of additional, concrete information about student achievement in the Progress Report would be useful to school staff and families,” the city’s guidance to principals reads.

None of the elementary and middle school changes alone is as as dramatic as the one that will hit high school progress reports when they come out later this month. Students’ college-readiness rate will count for 10 percent on this year’s high school reports, inducing potentially significant score declines for schools that have successfully gotten students to graduation but not prepared them with even the most basic college-level skills.

Concerns about the graduation-skills mismatch were one reason that the department last year introduced new policies to guard against soft grading on high school Regents exams and limit students’ ability to make up missed credits without retaking failed courses.

The city uses the progress reports as part of its determination of which schools to close and which principals to reward.

UPDATE: This article originally stated that elementary schools, which do not have core courses, were subject to the change in the way progress report scores are calculated.

  • Michael M. (parent still)

    Wowzers. 
    A whole 5%?  A whole third of the “school environment” score?  An entire half as much as the parent survey?
    /sarcasm

    Note that year-to-year comparisons have been just as problematic in past years in which the formula has been changed, or in years in which “progress” masked significant grade inflation correction on the state tests.

    Going back about 5 years, I recall my jaw dropping, when, in response to parents like myself questioning the “progress” category outright, DOE RAISED the percentage of the formula pinned to “progress,” and then had the chutzpah to credit “input from parents.”

    As of 2010-2011, the weight on “progress” was 60 (out of 100), and “performance” was only 25.  Not sure what’s pending for 2011-2012.  I’d still like to see my, and others’, originally requested shift AWAY from “progress,” as well as to a more balanced “peer”/”citywide” split.

    And don’t get me started on the broken promise of using multiple years of data to reduce the annual pendulum swings.

  • East Sider

    Is NYC part of NYS?  The Department and State Ed have totally different methodologies in measuring school progress.  While the data embedded in the reports are important the life-death weight of the letter grades are deplorable. School planning should reflect the data in the reports, with whomever supervises schools these days, SLTs should address the data in driving school budgets … unfortunately the reports are used as hammers to punish school rather than tools to assist schools … unfortunately the mantle over Tweed is still”The Beatings Will Continue Until Morale Improves.”

  • http://twitter.com/MrPortelos Mr. Portelos

    My troubled school just dropped down to a “C”. That gives us even more of a reason to unite as a community and inquire “Where’s the support?”

    http://www.occupywarrenstreet.org

    Francesco Portelos
    Parent
    Educator
    http://www.protectportelos.org

  • http://twitter.com/BNiche B

    My school went up by 2 points over (50 to 52), but tanked in percentage points (by about 22). Either way, my school’s a B school, but still… color me confused?

  • Michael M. (parent still)

    I believe the change only applies to MIDDLE school grades — NOT elementary school grades.

    But this is to miss the forest for a few twigs.  The reports are still grossly flawed and should be TOSSED.

  • Michael M. (parent still)

    Just goes to show how little stability there is in the system, a common problem.

  • Michael M. (parent still)

    As you may have noticed, under the DOE’s method, your school missed a chance to score points in Math Progress, especially in the lowest third, or pick up bonus points in Closing the Achievement Gap.

    And if the DOE’s method had weighted more on “performance” than “progress,” your school’s English scores within peer group would have helped the overall score.  But when there’s only 10 points out of 100 to be had for either english or math PERFORMANCE, you can share my conclusion that the reports are meaningless.
     
    Note that if your school had only gotten 4.4 more points (e.g., on the Enviornment Survey, by encouraging parents to have given a half point higher on each of the judgment scales), I.S. 49 would have gotten a “B” overall, and landed in the 40th percentile.

  • Tim

    I don’t disagree that they’re a mess, mostly because they mirror the mess that is NCLB, but I’d hate to see them tossed entirely. It’s only a tiny smidgen of power, the parent survey, but at least it’s something.

  • Michael M. (parent still)

    But the parent survey results in no changes, no policy input, and just RANDOMIZES the Report grades.  Feh.

  • Michael M. (parent still)

    Mr. P,  Please see my “reply” above. 

  • Tim

    You might be right about policy input and randomization (though I’m not sure about that second one), but a lousy environment score absolutely gets attention from elected officials, school communities, and even the DOE. 

    I’m making a fine distinction here: I’m not saying that this is a tolerable or optimal way for parents to have input, but it is certainly better than nothing at all, which is where we’d be if the reports/surveys were tossed.

  • Michael M. (parent still)

    Lousy environment scores also get school administrations to encourage parents to pump the results.  It’s a form of gaming the outcomes, IMHO. 

    Per my comments re IS 49, and taking it a step further, it’s easier to game the enviro survey than teach math to the lower third.  And that ain’t right.

  • Stefan Koster

    This grading system does a lousy job of identifying the best schools based on academic achievement. That’s actually what parents want to know. 
    According to these new results District 3 doesn’t have a single K to 5 elementary school worthy of an A! That’s ridiculous.  PS 9, PS 199 and PS 87 all receive the same overall B that other schools with much lower proficiency levels receive. 

    It gets even better on the middle school level. MS 54 – Booker T. gets the same B as MS 860 – FDA II, which goes from an F to a B, even though MS 54′s peer index is 3.43 and that of MS 860 is 2.09. BTW, within the group of 6-8 middle schools MS 247 receives an A with a peer index of 2.36 as does MS 258 with a peer index of 2.16. 

    The peer index for 6 to 8 Middle schools operates on a 1.00–4.50 scale and is calculated using the following formula: Peer Index = (Average 4th grade English and Math proficiency) – (Percent students with disabilities). 

    If I’m moving to D3 in Manhattan from anywhere in the country, or the world, and I would use DOE’s Progress Report to get a quick overview of the best schools in the district, chances are very high that I would not get the information I’m looking for.

    Why can’t a B be a real B?

  • East Sider

    The reports are useful guides to SLT teams in determinig how to allocate school resources … the sharp jumps or declines in letter scores are a failing of the reports, and, reports are not hard to “game” by targeting particular cohorts of kids. Worse, the determine school closing on a grade is horrendous … the letter grades are foolish and all based on reward/punismnent rather than a guide.

  • http://twitter.com/leoniehaimson leonie haimson

    I wish all schools would get a “B” so no one would attempt to use these ridiculous grades to assess quality.  With 85% based on one year’s worth of test scores, and a large part of that  gains or losses in scores from the year before, which are up to 80% random, these grades are so volatile as to be wholly unreliable.  Not to mention the DOE still has not figured out how to properly control for the student population, with schools that enroll large nos. of students w/ disabilities unfairly punished.  And to imply that rating middle schools on how many students pass their courses is somehow an improvement …absurd!  Instead, this is likely to lead to the same sort of grade inflation as it has occurred in HS across the city, where teachers are told to pass the vast majority of their students or else.  Check out my 2007 DN oped, which I wrote when the grades were first announced.  It continues to hold true today: http://shar.es/5qG1m 

  • Michael M. (parent still)

    As you might have noticed, the Peer Index is defined differently for elems than middles. Elems consider a new “Economic Need Index,” as well as Students with Disabilities (as per middles), and also Black/Hispanic and ELL.

    Two points: 
    1) I don’t know why Black/Hispanic should NOT be part of the middle school peer grouping, and
    2) This formula is DIFFERENT than last year (Eligible for Free Lunch has morphed into a more nuanced Economic Need Index which blends Temp Housing and HRA-eligible with Free Lunch) … which means schools could well have changed peer groups over the summer, further confusifying year-over-year comparisons and adding to volatility.

  • Philip Nobile

    Philissa, what’s with the Orwellian terminology in your score/grades shift story today? What you call “soft grading” on Regents exams, we teachers call tampering, a crime under state law, that everybody knows, including you, was the “dirty little secret” that DOE, UFT, and Regents covered up in their fashion for years and years. Cui bono?
    There is one and only one reason for the rushed reforms in Regents test security–the Wall Street Journal’s devastating exposé  headlined “Students’ Regents Test Scores Bulge at 65” (Feb. 2, 2011). This story ripped the integrity of Bloomberg’s and Klein’s grad rate claims. Nobody at City Hall or Tweed dared to contest the paper’s findings. When the retired Chancellor Klein was asked for comment, he shamefully declined.
     
    And now the DOE is shifting weight from state exams to course grades in progress reports. Isn’t that swell? Teachers and principals who cannot be trusted to be honest with their students’ Regents and credit recovery will get the last word, as they always have, with course grades. The new dirty little secret is pass quotas set by principals who will U-rate teachers who don’t go along. I missed Chancellor Walcott’s memo on that cheating trend. While preparing my first report card at Washington Irving in 2001, I was ordered to pass 80% of my Latin students. When I refused, I was fired on a trumped up corporal punishment charge which I forced Principal Robert Durkin to retract at a Step 2 grievance. The more things change …

  • Michael M. (parent still)

    On a fishing trip for volatility, I started checking school scores in D2.  It didn’t take but 5 minutes and three schools checked to come up with this insanity:

    According to the DOE, PS 234 deserved to be in the 69th percentile in 2010 (good enough for a B), the 81st percentile in 2011 (good enough for an A), and this year…. drum roll please)….

    THE FOURTH PERCENTILE.  Not a typo.  The number 4.  Out of 100.

    But hey, they were saved by a new free-fall floor for schools in the “top third citywide”, and only got a C, rather than the D which the multi-gazillion dollar black box would have assigned — F’s being reserved for only the lowest TWO percentiles.  Wouldn’t want to be accused of going soft, eh?

    The DOE gets the F.  As in FRAUD.

  • BloombergMustGo

    I’m sorry, “power”?  Parents have all the power in the world to affect student scores.  Instead of spending all day coming up with excuses and teaching children to bash teachers, try the following:
    1) Feed your children healthy food.
    2) Make sure your children are getting sleep and not playing video games until 2 AM.
    3) Make sure your children are properly clothed when you send them off to school.
    4) Make sure your children read at home and learn to appreciate the power of literacy.
    5) Make sure your children do their homework as instructed.  No, don’t do it for them.
    6) Teach your children to respect knowledge and education.
    7) Harass your mayor until he stops robbing schools of money and overcrowding classes.

    These are the true foundations of power.  I promise that if every parent practiced them, the children would perform miracles.

  • http://twitter.com/leoniehaimson leonie haimson

    As opposed to the overly credulous account here, see the analysis from InsideSchools, showing that “severe instability
    persists” in these grades, despite the DOE claims. Fully 24% of the
    schools that got failing grades this year earned A’s and B’s on their
    2010-11 Progress
    Reports. http://insideschools.org/blog/item/1000471-quarter-of-failing-schools-were-on-top-last-year

    As Michael points out, some highly regarded schools like PS 234 would have fallen even further in their grade — from an “A” to an “F” — if it weren’t for the built-in failsafe system that the DOE has devised to prevent this.  Does anyone really believe that the quality of this school, with the same principal and staff, fell so low in one year?

    How anyone can take this grading system seriously, or argue that any sort of intelligent decision should be based on grades so
    volatile and unreliable, is beyond me.  As I have pointed out many times, these grades are determined primarily by one year’s
    worth of change in test scores, which experts have found to be 40-80% random — not to mention how essentially flawed the state exams are in themselves, esp. at the lower and higher levels.
     

  • I noticed that…

    “The DOE is increasing oversight of schools’ grading polices. Schools may be asked to provide documentation of grading policies for review to justify student course performance results.”
     
    If students are given online credit recovery or they’ve been assigned to online learned such as “blended learning”, how does the DoE oversee this type of learning?  What are the standards and the rigors when students only have to guess the answers on their computer?  What is the grading policy when many students used the DoE’s “vendor-approved” Aventa?  Once again the DoE is talking out of both sides of their mouth!

Tips, questions, feedback?

Contact us at .

Word from Our Sponsor

Follow GothamSchools

RSS
Subscribe to the daily email digest:

Chalk It Up

Recent Comments

3 comments so far today

Archives

May 2013
M T W T F S S
« Apr  
 12345
6789101112
13141516171819
20212223242526
2728293031