GothamSchools — daily independent reporting on NYC public schools

roland fryer returns

March 7, 2011

Study: $75M teacher pay initiative did not improve achievement

by Elizabeth Green, at 3:56 pm

New York City’s heralded $75 million experiment in teacher incentive pay — deemed “transcendent” when it was announced in 2007 — did not increase student achievement at all, a new study by the Harvard economist Roland Fryer concludes.

“If anything,” Fryer writes of schools that participated in the program, “student achievement declined.” Fryer and his team used state math and English test scores as the main indicator of academic achievement.

Schools could distribute the bonus money based on individual teachers' results, but most did not. Most teachers received the average bonus of $3,000.

The program, which was first funded by private foundations and then by taxpayer dollars, also had no impact on teacher behaviors that researchers measured. These included whether teachers stayed at their schools or in the city school district and how teachers described their job satisfaction and school quality in a survey.

The program had only a “negligible” effect on a list of other measures that includes student attendance, behavioral problems, Regents exam scores, and high school graduation rates, the study found.

The experiment targeted 200 high-need schools and 20,000 teachers between the 2007-2008 and 2009-2010 school years. The Bloomberg administration quietly discontinued it last year, turning back on the mayor’s early vow to expand the program quickly.

The program handed out bonuses based on the schools’ results on the city’s progress report cards. The report cards grade schools based primarily on how much progress they make in improving students’ state test scores. A so-called “compensation team” at each school decided how to distribute the money — a maximum of $3,000 per teachers union member, if the school completely met its target, and $1,500 per union member if the school improved its report card score by 75%.

The deal was seen as a landmark in 2007 when Mayor Bloomberg announced it with then-United Federation of Teachers President Randi Weingarten by his side. “I am a capitalist, and I am in favor of incentives for individual people,” Bloomberg said then, while Weingarten emphasized that schools could decide to distribute bonuses evenly among educators. She called the program “transcendent.”

In his study, published as a National Bureau of Economic Research working paper, Fryer writes that researchers were surprised to see that schools that won bonuses overwhelmingly decided to distribute the cash fairly evenly among teachers. More than 80 percent of schools that won bonuses gave the same dollar amount to almost all of the eligible educators.

Researchers were also surprised to find that middle school students actually seemed to be worse off. After three years attending schools involved in the project, middle school students’ math and English test scores declined by a statistically significant amount compared to students attending similar schools that were not part of the project.

The study adds to a research literature on teacher incentive pay that is decidedly more lukewarm than much of the popular conversation about teacher pay. Fryer, himself a strong early advocate of experimenting with financial incentives to improve student achievement, calls the literature “ambivalent.” While programs in developing countries such as India and Kenya have had positive effects, few teacher incentive pay efforts in the United States have been deemed effective.

Almost all schools gave nearly all of their teachers the same sized bonus.

Nevertheless, a person’s position on teacher merit pay has become a litmus test for her reform credentials in many education circles. During his campaign, President Obama used his support for merit pay — traditionally scorned by teachers unions — as evidence that he was willing to challenge traditional Democratic Party thinking. Now, the Obama administration has boosted support for the Teacher Incentive Fund, a program that funds local experiments in incentive pay.

What explains the discrepancy between programs in the U.S. and elsewhere? Fryer rejects several explanations. He argues that the $3,000 bonus (just 4 percent of the average annual teacher salary in the program) was not too small to make a difference, citing examples of effective programs in India and Kenya that gave out bonuses that were an even smaller proportion of teachers’ salaries. He also rejects the possibility that schools’ decisions to use group, rather than individual, incentives was the problem, citing a 2002 study of a program in Israel that used group incentives.

Instead, he says the challenge is that American plans aren’t clear about what teachers can do to receive the reward. In New York City, the bonuses didn’t come simply if students’ test scores rose; the test scores had to rise in comparison to a group of similar schools. So did other measures considered by the city report card, including the surveys that ask students, teachers, and parents for subjective opinions about schools.

Fryer argues that the complexity made it “difficult, if not impossible, for teachers to know how much effort they should exert or how that effort influences student achievement.”

Print Print
← Older Comments
  • floyd99

    They are doing this in Mexico and Brazil and apparently the results are stunningly positive. According to a times article a few months ago, if the trends continue, the US will have more inequality between rich and poor than Brazil. Think about that for a moment. But, just imagine anybody trying to pass more programs that invest money into the poor here. I’m constantly amazed at how we have so many clear examples of social programs working all around the world and yet so many voters here refuse to acknowledge reality.

  • floyd99

    They are doing this in Mexico and Brazil and apparently the results are stunningly positive. According to a times article a few months ago, if the trends continue, the US will have more inequality between rich and poor than Brazil. Think about that for a moment. But, just imagine anybody trying to pass more programs that invest money into the poor here. I’m constantly amazed at how we have so many clear examples of social programs working all around the world and yet so many voters here refuse to acknowledge reality.

  • floyd99

    If I offered to pay you a million dollars to jump over the empire state building (legit, no tricks, just from the strength of you muscles), you could train all you want but it just isn’t physically possible.
    Merit pay implies that you can do better if you had more incentive. No amount of incentive will allow you to jump over a skyscraper. Teachers, like all sane human beings, would love to get paid more for their jobs. Merit pay just doesn’t work because the issue isn’t that teachers aren’t paid enough to be motivated. At the same time, if you pay teachers so little they have to get a second job or can’t pay their bills, you will force teachers to be martyrs to do the job and you will have trouble attracting anybody to the profession. Michelle Rhee and Klein claim they care about the kids but were/are making six figures a year.
    Do you also question their love of kids, or, does it only ring hollow when folks making a fraction of their salary feel like they deserve to be compensated fairly?

  • JEichman

    “The program had only a “negligible” effect on a list of other measures that includes student attendance, behavioral problems, . . ”

    What is the relationship between a teacher receiving merit pay and student attendance and behavioral problems? Please don’t answer with some idealistic twaddle about how a change in teachers’ behavior will somehow magically entice students to do what they ought to be doing in the first place. When the American people can face the fact that we have become a sick, dissolute society, when Americans re-enshrine industry, discipline, self-control, and personal accountability, rather than entertainment and pleasure-seeking, as the ruling principles of our society, then we may see positive change in student achievement. If you want to cure the problem, get to its root; don’t just spray the leaves.

  • Rayfolk

    What are we to make of this information? As it is explained here, it is a peice of journalistic garbage. Why did student achievement decrease? Are yousaying it had something to do with the incentive program. This would imply that an increase teacher’s income will have a negative impact on education. Maybe the decrease in studetn acheivement more to do with budget cuts in education.

  • Tortugadreams

    Which is why it’s so interesting that the results of the study are not in glowing support of merit pay.

  • Jkleincollins

    When you break a promise, like many states have, do you expect the person you affected to then make up for your lack of follow through? That’s what states want teachers and other municipal workers to do. Teachers and other state employees will, in the end, compromise. State legislators are like the children we teach – short-sighted and lacking impulse control.

  • PiledHd

    How about an alternative approach, that would hold teachers to the same standards as everyone else in the workplace: Your students do poorly, and you get fired.

  • drjb

    As a researcher, clinician, and educator of urban teachers, I find numerous flaws in just this cursory description of the results of this study. It doesn’t make sense to use studies examining samples from Kenya, India, and Israel as a comparison point to try to make sense of the results yielded in this study of teachers in NYC. Clearly there are a number of confounding variables that make analyzing the results of this study murky at best. The researcher stated that a majority of teachers in his sample ended up splitting their earned bonuses among all colleagues at the school, which likely amounted to fairly small individual amounts that likely would have little impact on one’s performance! I’d be interested to examine the performance of the teachers who were able to keep the total amount earned (if there were any?) and see what impact that had on student outcomes. Its appalling that the effects of merit-based pay is being assessed in studies that are not actually examining true implementation of merit-based pay! During a time when our education system is finally getting at least a small percentage of the attention and scrutiny it so badly needs, and when there is greater push to try to “reform” our broken system by applying a business model and using management procedures found to be effective in the board room, in the classroom, it is astounding that this seemingly methodologically questionable study could be conducted, much less published. The implications of this study are potentially far reaching; thus, greater caution should be taken when reporting on the findings, and more emphasis should be placed on careful scrutiny of these significant studies.

  • Spiraltri

    It’s very simple really. The scores didn’t go up because they aren’t low in the first place (primarily) because of “bad” teachers. The scores are low in districts with insane poverty rates. Folks cannot concentrate on academics when they are facing the social and emotional stressors that come with poverty.

    If you look at statistics student achievement is tied to socioeconomic class. If you are white, black or otherwise in a middle class family you have a good chance of graduating and going to college. If you are poor, black white or otherwise, there is a good chance that you will not complete high school. Still, based on more than a century of policies and laws that have kept significant numbers of people of color without access to wealth (which means no wealth to pass on to the next generation), we can see this as an issue that is attached to race. You are more likely to experience poverty if you are a person of color than if you are white.

    The idea that noticing that there is a correlation between poverty and low achievement in school is somehow making excuses is ridiculous. How can you explain away a correlation that is so strong? Stating that a few charter schools (that have more control over which students they take in…maybe not on paper, but they do) are able to do better with the same demographics is insane. Those are small schools of choice and small schools of choice DO work.

    Large urban districts do not give teachers the chance to teach in small schools of choice. Instead teachers are micro-managed and left with rules that are anything but realistic. You want everyone to read better…when a kid can’t read at 16 stop the standardized tests and focus all of your energy on teaching the kid to read…but that isn’t allowed.

    If you force pay by standardized test scores on teachers you will soon have a real quandary…another one. Who in the world will work in the neediest schools? I mean, I’ll never make any money if I work with the kids who need the strongest teachers the most. Hmmmm, seems like a set up to keep the poor with nothing and the folks who have in positions of power. Merit pay=a mass exit of teachers from large urban district as they can’t possibly get paid a living wage=the anti-teacher movement pointing their finger and saying “see, urban teachers are lazy and bad” (those are the only ones that will be left)=a justification in the mind of those in power for the power structure that benefits them…the status quo. I mean, they tried right? What else can they do if these lazy kids want to be poor and those bad, lazy, overpaid teachers don’t even respond to proof that they are bad and lazy?

  • Twkoppusa

    Merit pay is the great lie told by those who really don’t believe there is any merit in the system at all. It’s another way of bashing hard working teachers who happen to be in the way of the business moguls who want to make education a profit-making venture. My state gives me a bonus for earning my national board certification, a proven measure of accomplished teaching;. It will probably disappear this year due to budget cuts. Why should anyone believe these reformers mean business when they offer to pay for “merit”? And where in the conservative lexicon does it say it is okay to give “bonuses” with tax dollars? What serious conservative thinker says that? I think it’s all smoke and mirrors to hide the real agenda of destroying unions and public schools. Who benefits when the brick and mortar crumbles? The same businessmen who brought us our current health care system.

  • Epnev5

    Vouchers will do the same thing. Give an equal amount of money to a wealthy family and a poor family. The wealthy family can afford to pay up for the best schools and the poor family will be stuck in a school system that is underfunded with underpaid personnel .

  • Dschneider1

    Unlike the workplace that you refer to, the public school system must “Hire” every student who walks through the front door, and is not allowed to “Fire” any of them. How would you like to work under those conditions, knowing that YOU can be fired when these workers do not perform well?

  • boji

    This study simply confirms merit pay studies done on education for nearly 200 years. Pay isn’t a great motivator. It seems so logical, but it simply does not produce the required results. Daniel Pink’s book Drive devotes a great deal of space to this phenomenon. As a teacher, I know I could raise test scores in English by teaching grammar all day. However, my students would not learn to write. Like merit pay, teaching grammar sounds like a good idea, and it does allow students to do well on tests, but it doesn’t teach them how to write. In fact, like merit pay, studies since the 1930′s show it decreases writing ability because it takes time away from writing.

  • merit offered teacher

    The money was split evenly among the educators showing that each person in the school earned $3000 if they met the standards. The key is that merit pay cannot cause a person who is currently working to address the needs of the student meet the needs at a higher level. It is impossible to offer incentives that have a person think or design harder.

  • newman

    “Students are not widgets and teachers are not production managers”

  • rsaruse

    Well said. How can states expect more from teachers that are already giving 100%? Merit pay has been on the table in my state and is now off the table We have not had a pay increase in years. The teacher work load has increased significantly. Teachers that were giving 100% then are still giving 100% now.

  • Erecinos

    It’s not that teachers don’t want more money. It’s that more money is not going to change the fact that mom is a crack addict.

  • Svanschaack

    Hopefully most teachers do their job because they care deeply about the education of young people in this country. There’s not alot of greed in it because teaching by its very nature is a reward when done successfully and I frankly have never met a wealthy teacher.. Merit pay seeks to destroy good intentions and create wrong motives. It is divisive by its very nature and offers no real encouragement to unify the schools as a whole. As has been said time and time again, merit pay for teachers has never worked so why should a broken system be considered again?

  • Pingback: Study: New York City teacher pay initiative did not improve student achievement » Carol J. Carter

  • sbfps

    Teachers are already held to this standard. there is a way to fire teachers who are completely ineffective. What about the kids who arrive the day before the state test and are counted on a teacher’s list? What about the teacher who gets a large share of student with learning needs, because with scheduling it’s easier if they’re all in the same class. There are many thousands of teachers working hard every day under difficult conditions who are continually demeaned by people who have no idea how to get a child to pass a state test. If it was that easy, anyone could do it.

  • sbfls

    Absolutely. Children are not widgets. You can’t stamp them out every 2 minutes and throw away the ones who aren’t worthy. They all come and all must be educated.

  • sbfps

    Perhaps people haven’t really considered that most teachers give great effort. Perhaps there are other things that effect a child’s ability to learn.

  • sbfps

    Perhaps people haven’t really considered that most teachers give great effort. Perhaps there are other things that effect a child’s ability to learn.

  • Steveo35

    Teachers are not motivated by money. I took a $20k cut in base pay, plus I lost my bonus, stock options,401(k) match, expense report and long lunches to come back and teach. About $700k in the hole over the past 11 years.

    Tell me I’m over paid.

  • Cannonjlf

    PERHAPS?!

  • Cannonjlf

    However, when you have students in 9th grade who cannot think critically, speak responsibily or persuasively, write correctly or read with senstivity or understanding, how much incentive you provide to the teacher is a moot point.

  • Spiraltri
  • Dr D Jackson

    It happens every day to management people who cannot motivate their workers. If it is one’s job to motivate subordinates to higher productivity, he or she must find the means to do so or face the music. Likewise with teachers. Forget the decades old hackneyed excuse that they come from bad parents.

  • SESister

    Fryer and Rhee created a similar incentive program in DC that paid middle schooler to essentially come to school and be good. It faded away after one year. Fortunately the city refused to fund that preposterous scheme. Mr. Fryer fails to understand what most school people already know: It is not about the money!

  • Dr. D. Jackson

    This is a great point and suggests support for language and math across the curriculum. It also adds to my curiosity about the instructional and content elements in the classrooms within the study.

  • cp

    Keeping in mind that teaching writing while ignoring grammar is also negative, and a great deal depends on one’s approach. To truly write well, a student must also have some knowledge of grammar and mechanics.

  • Pingback: Study: $75M teacher pay initiative did not improve achievement « Lafayette Education Association : LEA-IN.org

  • Pcole

    I’m with you drjb. Most merit pay arrangements are poorly designed and thus are doomed to failure from the outset. Those that adopt a pay for contribution mindset and have several components that might yeild additional pay – including encouragement for mastering and adopting specific teaching approaches -

    generally are far more successful. Performance pay describes a generic concept whereas it is the type of model adopted that will produce ‘good’ and ‘bad’ outcomes – one model I have been involved in has 6 components including individual, team and achievement of targets and targets include student academic outcomes as well as student wellbeing, retention rates, transition patterns, etxc.

  • Susanrasmussen513

    Parental position and support of teachers is often vastly different in Americans than in those of other countries. I wonder if researchers can measure that.

  • Susanrasmussen513

    Parental position and support of teachers is often vastly different in Americans than in those of other countries. I wonder if researchers can measure that.

  • Eeppsrd

    Thus is the typical problem with the buzz word concept of “merit pay,” that is being used as a talking point towards the goal of ending collective bargaining. “Merit pay?” O.K. But how will the progam be run, and how will it work?

  • Pingback: Middle School Matters » Blog Archive » MSM 157 Call, Nuts, Failures, Tools and More!

  • Jennifer, first grade teacher

    Oh goody, now we can have all stick and no carrot.
    But the entire concept was an insult to dedicated teachers anyway.
    Bloomberg is the proverbial man with a hammer. He will never recognize that teaching is a calling, not a business transaction where teachers can be played for fools.

  • Sonia F., Librarian

    Throwing money into programs doesn’t make up for parental alienation. It doesn’t make up for nutritional deficiencies that negatively impact students’ cognitive and physiological development. It doesn’t make up for prenatal exposure to alcohol and other addictive substances.

    Parents and communities must embrace education and personal achievement as worthy goals; show their interest in their young peoples’ daily emotional health and educational progress; and acknowledge that their young people receive much education from the home and the neighborhood long before stepping into the world of formal education.

  • atb

    You can’t hold teachers fully responsible if the student does not meet the standard. Some of the blame needs to be on the student and parents. I am a teacher and I go out of my way to help my students learn, but there are always a couple of students who are lazy and nothing will motivate them. So you are saying I should get fired because they do poorly?

  • http://tom.mcdougal.myvidoop.com/ TomMcD

    The logic: Merit pay will help because most teachers are working below their capacity; they *could* do a better but don’t because it isn’t worth their while. Salary is their primary incentive for working hard or not.

    The reality: Seeing your students succeed is the best motivator for the day-in/day-out hard work teaching requires. Most teachers have tried everything they could think of to improve their students’ learning. The reason teachers aren’t achieving better results is because they don’t know how.

    Other countries have systems for helping teachers improve. We do not.

  • Pingback: School ‘reform’: Educational salvation or burning the village in order to save it? - Online Political Blog

  • Pingback: Incentives | EDPL Policy Forum

  • Pingback: For Merit Pay, It's Still "One Size Fits All" | Title I-derland

  • Pingback: Indiana merit-pay bill: Still waiting for details « School Matters

  • Pingback: Waiting for Superman: Some Links « EDF 3514 Class Weblog

  • Pingback: Indiana education reforms: research-based … or not? « School Matters

  • Pingback: The Dawn of the Evidence-Based Budget - NYTimes.com

  • Pingback: Teacher Merit Pay Has No Merit, Again – @ THE CHALK FACE knows SCHOOLS MATTER

← Older Comments

Tips, questions, feedback?

Contact us at .

Word from Our Sponsor

Chalk It Up

Recent Comments

1 comment so far today
View all comments »

Events Calendar

RSS Feeds

Newsroom posts Community posts All GothamSchools content

Archives

June 2013
M T W T F S S
« May  
 12
3456789
10111213141516
17181920212223
24252627282930

Copyright Notice