And of course, by market you’re talking about a much smaller market than you seemed to imply with that comment, right? We’re talking about a specifically educated, highly trained labor pool -a post graduate degree required for all three fully licensed professions. But you knew that when you made that comment, didn’t you?

Look, if you weren’t aware of the general topic then it was actually you who hadn’t thought it through.
 
And if you were, then you’re comment is probably guilty of cardstacking (giving too much weight to one set of facts while completely ignoring another set of facts that would otehrwise not support your claim). 

You know cardstacking is a tactic of car salesmen, right? Just checking.

Now, does APPR offend my ego as a professional? Yes. You bet. Absolutely. But before you go dismissing the assertion let me encourage you to read the other reason I asserted: We are just coming up with a new (post war!) evaluation system. Doesn’t it only make sense to reach out and look to see what other professions (not occupations) are doing with regard to their eval systems?

To Flerp,
I hate to admit it, but you have made an outstanding point. You have surmised correctly, and I am at the moment, unable to articulate what I would like to see with regard to an eval system. I have a bullet list in my head, but I won’t insult your awesome point with it until I can articulate it in a more rounded way. Very good form.

1. When discussing teacher evaluation systems, it’s useful to look at how other “professionals,” such as doctors and attorneys, are evaluated.

2. Doctors and attorneys are not evaluated by “hard-line metrics,” as if they were salespeople.

3. Because they, like doctors and attorneys, are professionals, teachers also should not be evaluated by “ hard-line metrics,” as if they were salespeople.

My question is simply, how should you be evaluated?  You say that because teachers are professionals like doctors and lawyers, they shouldn’t be measured by “hard-line metrics.”  But presumably you also don’t think teachers should be evaluated like doctors and lawyers are evaluated (i.e. according to their employers’ discretionary determinations about their performance).  It appears that you find it useful to look at other professions for purposes of identifying how “professionals” should *not* be evaluated, but not for identifying how they *should* be evaluated.

There are, I think, many ways to tie test results to a fair evaluation of teachers. The problems I have with NYS’ way are more related to how ‘high stakes’ the tests are, how effective they are in testing knowledge.

And, turning to what we’re discussing here, I have a huge problem with how important these test results are to the evaluation itself, including whether or not good results should be a required component for effective ratings.  I can’t stress that last point enough, because it (almost alone) means that students will (using the push-ups metaphor) HAVE to do a certain amount of push-ups before the teacher can be viewed as fair -and for me, that crosses the line into the absurd.

I have a deep ambivalence when the subject turns to ‘value-added’. On the one hand, I’d like to see value added targets for my high school students. On the other hand, it’s my understanding that the margin of error on the VAM model that the NCYDOE uses on grades 4 – 8 is higher than 65% -that’s higher than the grade needed to pass!!! Would that be used for the gym teacher? I don’t think that’s reliable. 

And yet,  I do know that even that model of VAM can be useful to me (or to Mr. Shoop), whether used by me or by my supervisor (the person who would be better equipped in discussing that measure with me against a particular child’s other (potential) natural gifts, should the standard not be met). So you and I have lots of room for agreement there. 

But it’s the misuse of good ideas that is the problem here. It’s not tests, it’s high stakes tests (that aren’t very reliable anyway). It’s not VAM, it’s the use of VAM in some math formula that trumps my judgement or the judgement of my supervisor. It’s not measuring, it’s using the wrong measures, and then giving poor weight to those wrong measures; weight that will ultimately remove a key component to the process of educating  -autonomy for those who are on the ground (teachers and school leaders (and ultimately, parents, although that may be a different conversation)). 

I don’t want to end this with a disagreement, but comparing evaluation system to other professions has a huge relevance. There are evaluations for lawyers and for doctors. But these do not include the use of making hard-line metrics (whether they’re applied correctly or incorrectly) . Evaluations for certain other occupations,  however, do require meeting a clear standard.  

I’m picking car salesmen as an example (I mean NO offense to people of other occupations). It doesn’t sound ridiculous to assert this: ‘If you don’t sell enough cars next week, you’re out of a job’. Does it?  

But It would be ridiculous to ask doctors to save a certain amount of at-risk patients before they can be allowed to continue to practice medicine. Likewise for lawyers who defend accused dangerous criminals (“get these clients off, or we’ll never allow you to practice law again -ignoring any facts the case may have”). My point was (and is) that using the same measurements that you use on a car salesman to rate a teacher is as ridiculous as, well, using the measurements that you use on a car salesman on a doctor, or a lawyer. The common thread, for me, is that there isn’t a clear line to reach when it comes to professionals. And, or course,  the point I’m trying to assert is that teaching is a profession , not an occupation. 

Comparisons also have relevance not related to my previous (selfish) point, We’re in the midst of developing the first ever post NCLB teacher evaluation systems across the whole country. That’s big! It just plain makes sense for us to reach out see what other professions are doing with regard to evaluation systems (versus what other occupations are doing with those evaluations). That’s natural. That’s the normal common sense thing to do.

Last thing. I don’t mean to sound like a high school teacher here, but smart people blog. Starting one takes about 5 minutes & I’d be happy to walk you through the process anytime (you can find my email through my Twitter profile). 

Just backing up a little: I don’t doubt for a second that there are a lot of bad-faith actors behind the push for VAM, both within and outside of the DOE. But I believe that if the correct measurements are chosen and weighted fairly, VAM can be a useful part of how we evaluate teachers. 

I agree with Flerp that the comparisons to evaluation systems in other professions are usually irrelevant and don’t do much to advance the discussion. You are probably right that there is a certain mania for measuring things, but that’s because so many of us are being measured and evaluated in our professional lives, and that cat is way out of the bag. I would have greater luck asking my employer to give me a pony to ride around Central Park at lunchtime than I would asking them to stop using metrics to evaluate worker performance.

As a parent and as a taxpayer, my interest in value-added is its potential–POTENTIAL– to help the entire system function under the stress of diminishing resources. If value-added can help building leaders make more effective staffing and tenure choices, great. If it can help purge the system of the genuinely bad apples, that’s also great. Let me stress that I do not want value-added to be the deciding factor in any evaluation–it’s not fair to the kids or the teachers. 

Hope this helps clear up my position. One last thing for you and Mr. Shoop: if you don’t think there is any fair way to tie some kind of test result to a measurement of your ability to teach your students, we’ll have to agree to disagree. If you do think there is a fair way and it’s not being captured in any of the proposals for VAM that you’ve seen, I would like your thoughts. 

I just hope that it would take into account how important it is to consider the worth of measuring when considering an evaluation system. If we can all agree that measuring teaching is hard -and that measuring learning is even more difficult than that- then we can collectively acknowledge that an eval. system that is based on measurements alone in education is just plain wrong. Even the reformers -with their support of a system that is (was?) 60%  based on observations admit this. 

But that the government has codified these observations (along the lines of Danielson’s framework for effective teaching) means that a “good” teacher now comes from a cookie cutter. This is one of the things principals object to. 

And to have measurements as part of an eval system is one thing, but to say that a teacher cannot keep his or her job if those measurements don’t meet a certain expectation (which is what the state has said with it’s ‘weighting the grades’ approach (the thing they NYSUT sued  them over) is problematic.

But to assume -the way APPR does- that 10% of a principal’s staff is bad and must, eventually, be fired (as Carroll Burris has written in the WaPo a few times now (which I think GS somehow consistently missed in their roundups)) undermines a principal’s autonomy in such a way that it actually becomes impossible to build a good, functioning staff for a school.

This means that 10% of the teachers that principals selected are bad? This means that a principal who takes care to build a school community based cooperation must now contend with competitiveness among his or her own teachers? 

These reformers are outsiders! They wouldn’t know the first thing about education and would probably not even pass a college level Ed class if they took one. But they’re embarrassed about years of flat test scores (measurements) and they would like to blame someone other than themselves. 

It is against all of this backdrop that devoted parents (like Tim) listen to the words “evaluation” and “value added” and immediately think that measuring everything we see in the learning process is going to be a good thing. 

But … years from now, when every student can make the bare minimum expectation for push ups, yet none of them can actually excel at anything (because there was no teacher there with the freedom to spot to spot and develop those gifts) will I read on Tim’s blog about how we helped this generation of children, or hurt them? 

I’m just trying to say evaluation is a good thing. A good evaluation? even better. But this?? I don’t think it’s 1)going to be fair to teachers or 2) going to help kids…and the reasons for that have all to do with the measurements. 
Anyway …

(Also, this discussion is now totally moot, because donuts’s post was about measurement, not evaluations.)

Two Points:

     - Your dismissal of the analogy between teachers and doctor’s/lawyers confuses some important distinctions. You say that clients, for all sorts of reasons or no reasons at all, fire their lawyers all the time, thus invalidating the analogy. Partially true, but teachers are employed by schools and school districts (employers), not clients.

Doctors and lawyers also work as employees, and can also can be union members. Witness unions for municipal hospital doctors (The Doctor’s Council, SEIU) and Legal Aid lawyers. In fact, with the shrinking of the traditional private practice and outsourcing and de-professionalization among lawyers, these fields are going to more and more resemble the traditional employer/employee relationship. 

It is perfectly fair for critics of the assessment/evaluation lobby to compare the discrepancy between the treatment of teachers and other professionals. Who would ever think of attacking OB-GYNs for this country’s high rates of infant mortality (www.oecd.org/social/family/database)? Yet it is demanded of teachers that they and the under-funded schools they work in be responsible for the effects of poverty, and for overcoming it.

     - Your description of the underlying dynamics of the teacher evaluation debate is apt. It is fundamentally a question of power, involving the balance of forces between management and labor. The people pushing the new evaluation regime could care less about its pedagogical validity; for them, it’s a politically potent vehicle for exerting more control over labor relations and the composition of the labor force (see young, white, transient).

Good stuff, though Flerp!