GothamSchools — daily independent reporting on NYC public schools

student voice

Student surveys seen as unlikely evaluations element, for now

Inspired by a 2010 study that found that students’ feedback about their teachers helped predict how well the teachers’ students performed on state tests, New York City asked some schools last year to test out a student survey that could become part of new teacher evaluations.

But if the city and its teachers union agree on a new evaluation system this year, student surveys are unlikely to play a role, according to people on both sides of the negotiating table.

The Gates Foundation-funded Measures of Effective Teaching study found that student feedback and teacher observations combined were more closely correlated with teacher effectiveness than observations alone, or any number of other attributes of teachers.

The city participated in that study and adapted the survey used in it, called Tripod, for use last year in 10 of the 108 schools in the Teacher Effectiveness Pilot, meant to test possible components of overhauled teacher evaluations.

Under the state’s new evaluation law, 60 percent of teachers’ ratings must come from subjective measures such as principal observations and peer reviews. The State Education Department has said student surveys can play a role, too, if districts and their unions agree.

The head of the state’s teachers union says student feedback could be a useful element of evaluations. But city union officials say they are staunchly opposed to incorporating student feedback in teacher evaluations.

UFT Secretary Michael Mendel said the union’s position is that it is inappropriate to ask students to make high-stakes decisions about their teachers, because it puts the students under pressure and also could encourage teachers to put student approval ahead of student learning.

“Could you imagine if you were a teacher and you were ineffective by a point or two because you were rated ineffective by the children?” Mendel asked.

Even though city Department of Education officials say they would like to see student surveys play a role in evaluations in the future, they dropped the surveys from the pilot program this year.

“I think it’s something that we have to introduce into the process, initially with low stakes, so that teachers can see what the data looks like and see what they think of it and begin to trust it,” said Chief Academic Officer Shael Polakow-Suransky.

The Tripod survey, which the state has approved for use in evaluations, asks students to mark how much they agree with statements about their teacher and classroom. Items are broken down under “seven C’s”: care, control (of the classroom), clarify, challenge, captivate, confer, and consolidate. Statements include “Student behavior in this class is under control”; “My teacher knows when the class understands, and when we do not”; and “My teacher really tries to understand how students feel about things.”

Issues like those are ones that only students can speak to, said Kara Kreisberg, a Spanish teacher at West Bronx Academy for the Future.

“They’re the ones that are in the room,” she said. “As many walkthroughs [by administrators] as you have, the students are the ones who see it all.”

But students don’t understand other important components of what it means to be a good teacher, such as planning lessons or using feedback to improve, according to Joseph Vincente, a chemistry teacher at East Side Community High School.

“Student feedback is important but it’s also limited,” he said. “They don’t get to see the behind-the-scenes work.”

So far, Syracuse is the only large district in New York State that has agreed to use surves in new evaluations. The district’s chief academic officer, Laura Kelley, said Syracuse schools will use surveys at all grade levels.

“We just felt the student perspective would be a valuable perspective,” Kelley said.

Dick Ianuzzi, the president of the state teachers union, said he supported evaluation plans that included multiple measures. Validated student surveys such as those used in the Tripod Project, he said, could be one measure.

“Student surveys, just like self-reflection, are all pieces that when you add them together you get the multiple measures that give you a sound evaluation,” Ianuzzi said.

  • Guest

    Agree with the Union on this one.  This would work well in some cases but not in all.  There are just too many factors involved in my opinion.  How is Syracuse specifically doing this in the lower grades?  What about self-contained classrooms?  I could see it possibly working with top notch kids as it does mirror the process students did/may still do on the college level in regard to assessing their teachers but really now.  Are you telling me that the teacher in room 319 who has some of the most difficult students who barely read on a 5th grade level and are disrespectful towards a hard working teacher is going to get the same results as the hardworking teacher in room 321 who teaches film study and shows nothing but “cool” movies for most of the year?  Come on – and that is just one example of how unfair this component could be.  Not buying and want nothing to do with it – and I know I would do well on this.

  • DisgustedNYCTeacher

    Seriously????  After 25 years of teaching, my professional future will dpend on the evaluations of 13 year olds?  The same ones who can’t remember to bring a pen to school every day and forget their books in class?
    I think there is a hot dog vendor on the corner outside my school.  Maybe we should survey him as to how I perform.  How about we ask the custodial staff to jump in here, too?  Let’s not forget the mailman, UPS delivery guy, the water delivery guy, and the guy who brings the new mop heads every week.
    Ironically, the staff at Tweed accepts no one’s evaluation of their peformance.
    The NYC DOE has truly become a sad little circus of self indulgence and ludicrousness.
    As for the critics who will chime in about me being too high and mighty to be eveluated, the fact is there are very few people left in the non teaching staff of the NYC DOE who are actually capable of comprehending and eveluating what I do on a daily basis.  I feel very secure saying there are very few of them who could DO what I do much less evaluate it.

  • DisgustedNYCTeacher

    Seriously????  After 25 years of teaching, my professional future will dpend on the evaluations of 13 year olds?  The same ones who can’t remember to bring a pen to school every day and forget their books in class?
    I think there is a hot dog vendor on the corner outside my school.  Maybe we should survey him as to how I perform.  How about we ask the custodial staff to jump in here, too?  Let’s not forget the mailman, UPS delivery guy, the water delivery guy, and the guy who brings the new mop heads every week.
    Ironically, the staff at Tweed accepts no one’s evaluation of their peformance.
    The NYC DOE has truly become a sad little circus of self indulgence and ludicrousness.
    As for the critics who will chime in about me being too high and mighty to be eveluated, the fact is there are very few people left in the non teaching staff of the NYC DOE who are actually capable of comprehending and eveluating what I do on a daily basis.  I feel very secure saying there are very few of them who could DO what I do much less evaluate it.

  • DisgustedNYCTeacher

    Seriously????  After 25 years of teaching, my professional future will dpend on the evaluations of 13 year olds?  The same ones who can’t remember to bring a pen to school every day and forget their books in class?
    I think there is a hot dog vendor on the corner outside my school.  Maybe we should survey him as to how I perform.  How about we ask the custodial staff to jump in here, too?  Let’s not forget the mailman, UPS delivery guy, the water delivery guy, and the guy who brings the new mop heads every week.
    Ironically, the staff at Tweed accepts no one’s evaluation of their peformance.
    The NYC DOE has truly become a sad little circus of self indulgence and ludicrousness.
    As for the critics who will chime in about me being too high and mighty to be eveluated, the fact is there are very few people left in the non teaching staff of the NYC DOE who are actually capable of comprehending and eveluating what I do on a daily basis.  I feel very secure saying there are very few of them who could DO what I do much less evaluate it.

  • Tim

    Too bad. Granted, I don’t have any close experience with high schools or schools with large numbers of at-risk kids, but the elementary and middle-school kids I know are not just startlingly perceptive about teacher quality, they are unbiased in a way that almost restores your faith in the human condition. Almost, but then you read something like this:

    “Could you imagine if you were a teacher and you were ineffective by a point or two because you were rated ineffective by the children?” Mendel asked.

    Could you imagine if you were a really highly rated teacher and your student surveys were off the charts, confirming that you are doing a whole lot of things right? Could you imagine if you were a teacher anywhere on the spectrum and you used your student surveys simply to hone your craft? Could you even imagine if you were a borderline ineffective teacher and your surveys — gasp! — pulled you over to the effective side?

    Ugh. 

  • A.S.Neill

    My anecdotal observation of students’ candid comments about teachers indicates that they often are remarkably accurate about some things. Ineffective teachers frequently are hostile to kids, oblivious to their concerns, or really are just plain ineffective. But there is still a significant danger to all teachers in using student evaluations nonetheless. Cliques of students and their friends could easily manipulate evaluations as a retaliation for some fantasized abuse or perhaps just because a teacher is pushing too much “rigor”.they don’t like, etc. Since a well established principle of justice calls for protecting the innocent sometimes at the expense of letting the guilty go free, student evaluations is a poor idea and should be firmly rejected.  

  • Tim

    I could be wrong about the timeline, but I believe Blackstone’s formulation predated compulsory education laws. I can’t help but wonder if he and Franklin and everyone else who has written that 10 (to the xth power) guilty should walk free before one innocent person is found guilty would reconsider in this context. After all, it’s hard to imagine anything more ‘innocent’ than a child born into poverty and social immobility, and when Blackstone and Franklin were talking about ‘guilt,’ they were talking about consequences like imprisonment and capital punishment, not someone losing a job.

    As far as your practical concerns, note that the proposal was to roll out the survey on a very small stakes, low-impact basis. I’m extremely skeptical that classes of kids would conspire to incorrectly rate a teacher. But gving the test on a random date within a 4-6 week window, having an adminstrator/DOE official administer the survey, selecting kids at random–these could negate any potential shenanigans (by students OR teachers).

    I don’t have any doubt that the surveys would have reflected poorly on that small but persistent number of chronically ineffective teachers, but I honestly believe its biggest value would have been as a feedback tool for everyone else. And if you feel that evaluating teachers needs to be a holistic process with multiple measures, well, then this was another measure. Alas. 

  • Jim

    Tim, I’m sure your experiences with your  K- 8 sample are valid but as you to your credit admit you can’t “know them all” when it comes to students. Higher achievers have no trouble engaging in conspiracies to to achieve their goals- just ask Stuy. Given the ludicrously low bar of college readiness metrics at least 71% of high school seniors should be judged at risk -that is not ready for further education.Many ,many students believe that school is an absurd exercise and will treat the survey accordingly. My take.

  • Alison

    I think student voice is critical and the potential drawbacks can be controlled.  (I’m not sure it’s any more likely for a student to be biased or conspiratorial than a principal or peer evaluator, anyway.)  Why not have the surveys but have them count for relatively little?  Or they could be something that peers or administrators use for additional context when making their own recommendations?

  • Farro

    Some general observations:
    A. A teacher has the power to evaluate a student and, unlike one student evaluation, their one evaluation of a student has incredible power.  It is, at the very least, unfair not to give students some of the same power. 
    B. Only students are there every day, can see what goes on every day, and not just when the teachers are putting on a show for parents or admins.
    C. Teachers often bully students with little to no recourse for the students.  Therefore, I am unsympathetic to the conspiracy argument.
    D. The best teachers I had in high school all had their own very extensive and totally anonymous evaluation forms that they had all students fill out, telling us not to hold any punches.

  • BloombergMustGo

    A. A teacher is a trained professional whose evaluation is supposed to have significance.  A students has NO training in evaluating a teacher, and in many cases, lacks the maturity and experience to do so effectively.  A student is qualified, as a child,  to give their IMPRESSION of a teacher.  Maybe, a junior or senior in high school can offer some valuable insight, but younger than that it’s primarily personal feelings.

    B.  I don’t know about you, but I don’t put on “shows”.  The only teachers I see putting on shows are the ones who can’t teach, and we don;t need a student to tell us who they are.

    C. “.. often bully…”?  Again, interesting comment on your perspective of teaching.  Actually, I have witnessed more incidents of students, parents, and administrators bullying teachers than the opoosite.

    D. High school is not middle and elementary school.  See comment A.

    Your comments are a perfect example of why student evaluations should not be used to determine teacher ratings.  Personal prejudices have no place in determining if someone keeps their job or not.  The primary evaluation ofr teachers should be peer evaluation, far and above supervisor and student evaluations.

  • Guest

    I would value the teacher who values the opinions of his 13 year old students.

  • Mook

    The Atlantic had an interesting article about this.  It was pretty convincing that these surveys are useful. 

    Limited use of these surveys might be the best way to help get rid of the worst teachers without endangering the rights of the rest of us.  It might also be the best way to protect us from abusive principals

    I would like to have the option of using student surveys as part of my evaluation.  Of course, I’d like to use it in place of whatever ridiculous measures of student progress we’ll eventually be forced to use.  Not only am I confident I would do well on the surveys, but they wouldn’t take an extra minute of my planning time.  No data collection, no seething about the lack of scientific rigor in the collection of the data.

  • A.S.Neill

    Tim, thanks for a thoughtful reply. I could be wrong about the timeline, but wasn’t Blackstone a contemporary of that famous Massachusetts case with the hysterical adolescents as character witnesses for the prosecution?  But even today, this is no idle occurrence, as incidents of cyber bullying attest, sometimes with tragic results. Individually kids are often very mature, but peer culture can radically alter that and often rather quickly.

    Also, to say that tenured teacher is “just losing a job” is a serious understatement of the facts. Losing a job as a public school teacher also takes away your state teaching license, so you are denied public teaching jobs anywhere in the state as well as reciprocal right neighbor states – no small matter. Not exactly equivalent to disbarment, but pretty close. And this is aside from the disgrace to one’s reputation which I think Franklin had a quote or two about.

    And yes, Blackstone was probably referring severe penalty situations. But more to the point, this was already early legal reasoning that eventually became expressed as modern human rights law. So unless you are some sort of constitutional fundamentalist, we are all better off individually and as a society with this expanded concept of rights and protections.

    Finally, concerning the “innocent child born into poverty”, I think you could eliminate all 2-4% most ineffective teachers tomorrow and its not going to alter educational outcomes that much if at all. The attention it receives is certainly out of proportion to the possible effects it will have. Real educational reform is somewhere else (pre-K, class sizes, etc). Funny how that kind of “news”never ends up in the NY Post or Daily News. I wonder why. Maybe look at where Klein is now, or Moscowitz, or the recent Daily News editorial
    about what happens next after the “heralded” Newark teachers contract fails to
    change education. Just follow the money
    as Deep Throat would say.

    And oh yea. I do use student feedback in my classes already. It really is very helpful. what works or doesn’t, likes-dislikes, and suggestions. And that’s exactly where I intend that feedback to stay. .

  • http://www.advocatesforchildren.org/ AdvocatesforChildrenofNewYork

    No one has a larger stake in a teacher’s effectiveness than students, and their voices should be heard. As the teacher quoted in the article notes, administrators conducting classroom observations witness only a tiny snapshot of a teacher’s performance, which may or may not be representative of that teacher’s practice over the course of an entire school year. Students, however, are with the teacher in the classroom every day, and have knowledge and insights that no one else does. For example, only a student can tell you if a teacher really made the material “click,” inspired them to become passionate about a subject, or provided the support that made the difference between staying in school or dropping out. Surveys will give teachers and principals a fuller picture of the learning experiences of their students and will let teachers know what’s working well in the classroom and what isn’t.
     
    Well-designed, research-based student surveys are not popularity contests. Rather, they ask students for feedback on specific, concrete elements of a teacher’s practice. Surveys are also designed to ascertain an overall pattern of response from all of a teacher’s students. When used for evaluation purposes, they should be administered multiple times over multiple years.  A handful of angry students with a grudge against a particular teacher will not be able to throw off the results. Further, research on the Tripod student survey has shown that there is very little difference between how an A student and a D student rate the same teacher. Students will not penalize teachers who give them a bad grade. 
     
    A number of urban districts, including Chicago, Boston, and Denver, have committed to including student feedback as one of multiple measures in teacher evaluation, and NYC should join them. Advocates for Children of New York released a report earlier this year making the case for student and parent input in teacher evaluation, describing efforts other states and districts are making to incorporate such input into their own systems, and providing recommendations to the DOE, available at http://www.advocatesforchildren.org/sites/default/files/library/essential_voices_june2012.pdf.

  • http://www.advocatesforchildren.org/ AdvocatesforChildrenofNewYork

    No one has a larger stake in a teacher’s effectiveness than students, and their voices should be heard. As the teacher quoted in the article notes, administrators conducting classroom observations witness only a tiny snapshot of a teacher’s performance, which may or may not be representative of that teacher’s practice over the course of an entire school year. Students, however, are with the teacher in the classroom every day, and have knowledge and insights that no one else does. For example, only a student can tell you if a teacher really made the material “click,” inspired them to become passionate about a subject, or provided the support that made the difference between staying in school or dropping out. Surveys will give teachers and principals a fuller picture of the learning experiences of their students and will let teachers know what’s working well in the classroom and what isn’t.
     
    Well-designed, research-based student surveys are not popularity contests. Rather, they ask students for feedback on specific, concrete elements of a teacher’s practice. Surveys are also designed to ascertain an overall pattern of response from all of a teacher’s students. When used for evaluation purposes, they should be administered multiple times over multiple years.  A handful of angry students with a grudge against a particular teacher will not be able to throw off the results. Further, research on the Tripod student survey has shown that there is very little difference between how an A student and a D student rate the same teacher. Students will not penalize teachers who give them a bad grade. 
     
    A number of urban districts, including Chicago, Boston, and Denver, have committed to including student feedback as one of multiple measures in teacher evaluation, and NYC should join them. Advocates for Children of New York released a report earlier this year making the case for student and parent input in teacher evaluation, describing efforts other states and districts are making to incorporate such input into their own systems, and providing recommendations to the DOE, available at http://www.advocatesforchildren.org/sites/default/files/library/essential_voices_june2012.pdf.

  • DisgustedNYCTeacher

    Who cares.

  • DisgustedNYCTeacher

    Yes, because students have never made false accusations against teachers for trivial reasons.
    It must be nice to live in a dream world where children are perfect little innocent angels.
    Sadly, most teachers live in a world where students and parents are dangerous weapons that can be aimed randomly and for no good reason. 
    Children are called children because they are not adults.  They lack the experience, maturity, and emotional stability that comes later in life.  Their efforts should be concentrated on learning and absorbing knowledge, not expressing their ill-informed opinions about the complex activities of the adults around them.
    School is a training ground for life, not an arena for self indulgence of the young.  If we reminded children a little more often that they have much to learn, instead of putting them in adult roles, they would be better developed as adults.
    Children don’t need advocates, they need parents, teachers, and role models.  Then, when they have developed as well balanced adults, they too can get involved in the adult activities of life.
    Any teacher or administrator that needs a child to tell them what is going on in the classroom is a poor example of an educator.
    Perhaps President Obama can recruit some of these insightful youngsters everyone refers to as advisors on economic matters, foreign affairs, and military affairs.  After all, their unique perspective should not be hoarded by schools.

  • Tim

    Yikes. 

  • NYR683

    WELL SAID MY FRIEND WELL SAID

Tips, questions, feedback?

Contact us at .

Word from Our Sponsor

Follow GothamSchools

RSS
Subscribe to the daily email digest:

Chalk It Up

Recent Comments

5 comments so far today

Events Calendar

Archives

May 2013
M T W T F S S
« Apr  
 12345
6789101112
13141516171819
20212223242526
2728293031