GothamSchools — daily independent reporting on NYC public schools

First, thank you for (again) explaining the insides of the intervention research within IES. (I hope that Easton is successful in institutionalizing that through the next half-decade.)

Looking at the connect-teacher-ed-to-student-test-data in a research text, of course you’re right. On the other hand, the idea of looking at student success in this way is something to address political needs, part of which are ideological and part of which are rooted in legitimate needs to hold K-12 and education schools accountable. The latter is what I have to pay attention to. One question that always accompanies criticisms of political policy tools is always going to be, “Okay, what do you suggest?” And “nothing” is not going to work as an answer!

Especially with public programs, there’s a pragmatic question of which evil is least. I don’t know about TC, but many of the larger teacher-ed institutions in the country have been repeatedly intervened in/pressured for political reasons, through either overt political attacks (as in the Ross Perot commission in Texas) or through semi-bureaucratic moves such as churning certification requirements applied to college-based programs (which usually don’t apply to ABCTE and other alt-cert programs).

I’m inclined to give programs such as Noell’s work in Louisiana much more leeway than you are, not because it’s been demonstrated to have an effect on student success but because it is considerably better than the alternatives I see in front of me. I know of one college of ed where the dean is hoping that post-graduation surveys of alumni and their supervisors will suffice to show that we care sufficiently about outcomes. I think that might work for one or two years, but that’s it.

Sherman: I don’t mind acknowledging pragmatic political realities. But you know as well as I how easily policymakers slip into “research shows…” mode to justify political choices that have no research underpinnings. The rhetoric surrounding data systems is as much about improving outcomes as it is about “legitimate” needs to hold public institutions accountable. Why else would there be so much talk about getting rid of the “weak” teachers/programs, or having the “ineffective” teachers/programs learn from the “effective” ones? In the absence of a research base connecting policies and practices to student outcomes, political claims that “it’s all about the kids” ring awfully hollow.

Ken: Setting aside the adjectives, which are irrelevant to the issue, I’d say that it’s always difficult to assess the efficacy of a “full-coverage” program or policy for which there is no comparative condition that represents the absence of the program. The same is true for evaluating the effectiveness of a host of American social, political and economic institutions that have represented “the way we do things here” for a long time. I’m certainly not arguing that this is the best of all possible education worlds. Rather, what I am calling for is building a base of evidence on the effects of well-designed innovations, rather than declaring on the basis of hope, faith or ideology that we should scale up untested innovations because we know that they’ll be good for kids.

I don’t see how such data systems could result in improved outcomes for students, nor do I think this is the real goal of Duncan’s idea. This is teacher education bashing, pure and simple. Duncan would like to use data in this way to argue for the closure of teacher and principal preparation programs that, he would argue, are not doing enough to raise student test scores. The reason valid researchers would have “laughed [it] out of the room” is that just as it is ridiculous to argue for merit pay based on a very simplistic and mediocre measure of student achievement (test scores) so too is it ridiculous to say that the purpose of teacher preparation programs is to raise pupils’ test scores. Aside from concerns for the morally defensible means to those ends, and all the logistical nightmares of states’ tests, their timing, and reporting, we want teachers to also be prepared to handle the social and emotional needs of children, to work in partnerships with parents and the community, and to be intellectuals and life-long learners. But RTT funds are a mighty big carrot, and the states are in dire need. I think we are in for some scary times.

Ken,
I can’t help but compare the “innovation’s always good” approach with… the FDA.

Frankly, I want to know something works before trying it on my kids.

Ken,
Philosophical difference alert: I believe that the guvmint’s job IS to filter out the speculative from the proven beneficial. That goes DOUBLE when it’s taxpayer DOLLARS being speculated with.

You state, “…government’s stranglehold on innovation is a big factor in the dreadful productivity gains in public education.”

I’m all for reduced waste and increased efficiency, but such are secondary to whether or not one believes government should even be in the business of ascertaining whether “innovation” is indeed “productive” when it comes to public DOLLARS in public education.

And yes, I’m talkin about charters. And so, I believe, are you.

It ain’t all about the mean guvmint won’t let the parents — or the investors — have more charters.

The FDA has a stranglehold. The Patent Office has a stranglehold. The FAA has a stranglehold.

I think America is coming around to the idea that the Free Market doesn’t self-regulate well, and when it does, it’s not for the protection and benefit of the consumers.

Is your objection to the FDA based on efficiency, or that government shouldn’t be in the business of telling consumers what is efficacious?

Our children — your guinnea pigs? Sorry, no.

GGW,

I love the way you put this and I have the same question. I would add one thing:

Policy B is, without argument, hugely expensive in terms of time and money.

Ken,

Nice to see you commenting, again. But, as usual, I disagree with you.

The issue you are missing here is scaling something up — what Aaron is writing about here — is about changing the status quo. We are not starting from zero, of a hobbsian state. We are starting from what we have, and trying to figure out what we should change.

You appear to be advocating an end to licensing requirements for teachers, which is itself an intervention. Where is the evidence that such a thing would help students? You would scale that up nationally without a single smaller experiment.

I suppose that we could look at private schools, which lack certification requirements, but — as you know — they have never welcomed the kinds of comparisons that might control for demographic differences between student bodies.

So, I have to ask why you would want to end teachers preparation/certification without ending the same for doctors, lawyers, architects and pilots? I would understand an objection to the quality of our preparation programs and the low bar that certification represents, but your are not arguing for increasing that quality (e.g. by requiring FAR more hours of supervised work in the classroom) or raising that bar.

Let me try to go through your reasons.

“extremely-expensive”: Compared to preparation in other professions? A 15 month program? I couldn’t disagree more. The problem is that we are so worried about cost that we do not add an entire year — or more — of highly supervised work in the classroom before granting certification. Where are the residencies and requirements for experience before certification? Why do YOU think we lack them? Everything I’ve ever read about say that are expensive and the challenge of programs with them is to find funding for them. The federal government support medical residencies, and there is no such support for teachers.

anti-competitive: that’s an ideological argument, I think. There’s no evidence anywhere that competition improves education. Charters are not better, as a group, than non-charter schools — except perhaps in states with strict caps, which CLEARLY suppress competition. Some fields simply are not completive (at least on quality), probably due to the high cost and high reluctance to change providers. We see that in medicine, we see that in architecture, we see that piloting. And its pretty clear that we see that in education. In fact, it is the absence of competitive market dynamics that makes government regulation so important, as there is no free-market driver on quality in those settings.

politically-protected: Well, that’s not actually an argument against. It’s just a complaint. I’m really not sure about about how preparation programs are politically protected, as you intend it. Preparation program accreditation has nothing to do with the government, and it is the accreditation process that shapes these programs to which you object so strenuously.

time-consuming: Actually, I find preparation programs to be far too short. No one really complains that law school is too time consuming. No one complains that their pilot had to spend too much time getting certified and trained. And it takes them a lot longer than 15 months.

talent-discouraging: I actually think that this is the least sensible of your objections. Does law school discourage the “talented” from going into the law? Does med school keep the “talented” from going into law? Does divinity school or other priestly training keep the “talented” from going into the clergy? If the rewards are great enough, the talented will invest to get to the rewards. Researchers like John Willette have found that those with higher paying alternatives are least likely to go into teaching, and more likely to leave the field. One of my wife’s new bosses used to be a teacher and then went into the law, despite the requirement that she spend three more years in school.

I get that YOU don’t like the programs. I get that YOU would prefer an unregulated market, simply because you prefer unregulated markets. Not because unregulated markets actually work better or anything, just because of your ideological beliefs. You don’t need to make up such objections to regulations and departures from free markets. Just state your preferences for their reasons.

As usual, I don’t agree with you or see strong reasoning behind your suggestions. But I am glad to see that you’re back.

There is to date no evidence that the implementation of longitudinal data systems linking teachers and teacher preparation programs to student achievement outcomes has actually improved student performance.

This is stacking the deck. Data tracking isn’t an “intervention” in any meaningful sense; it’s not itself altering anyone’s educational experience.  Rather, it’s just collecting data to find out whether ACTUAL interventions — such as, say, teacher education programs of differing types — are working or not. And now that you mention it, teacher education programs don’t meet the Goal 4 criteria right now.  So shouldn’t you be in favor of collecting that sort of data?  

Maintain High Standards, but Reallocate Resources

How much of an impact does evidence make at the Department of Education?

In the context of the Department of Education and the latest i3 grants– it’s a heavy weight. Recently Aaron Pallas made clear that an idea does not stand to be scaled up if there is not an immense amount of proof behind it.

The Investing in Innovation Fund, known as i3, shares this high standard. Successful applicants would receive funds to support local efforts to start or expand research-based innovative programs that help close the achievement gap and improve outcomes for students.

Developing education innovations that work is difficult and chancy. It is quite possible that the pool of applicants meeting the requirements for Scale-Up grants (superb programs with the potential to be brought to scale) will be quite small. Thus, one could expect that the Department will allocate the majority of the grant awards to go to proposals that meet the requirements for Development (for promising but still unproven initiatives) and Validation (somewhat proven initiatives ready to grow at the State and Regional levels) grants.

In particular, the Development grant category represents a tremendous opportunity for the Department to develop new ideas based on research that could foster continuous improvement to the lowest performing schools and Districts in our public education system.

http://knowledge-ablesource.blogspot.com/