GothamSchools — daily independent reporting on NYC public schools

You should ask the people behind the study if they are going to study the factor of class size. If not, why not? Especially as more than 90% of teachers respond in surveys that this is the most important factor in effective teaching.

But ever since the Gates foundation did a survey about seven years ago, and found that parents and teachers thought that class size was a far more important factor than school size, they have tried to bury this issue. When I asked the researchers in different cities why they didnt study the factor of class size in their evaluations of the small schools, they told me that the foundation had disallowed it; even after the issue had come up in their interviews with teachers and students over and over again. Instead, they were supposed to focus on the more amorphous factors of “rigor” “relevance” “relationships” (whatever that means) and “community partnerships.” In the recent Scholastic survey that Gates financed, 90% of teachers said that there was a critical need for ” teachingresources to help differentiate instruction.” When I asked the survey director what “teaching resources to differentiate instruction” meant exactly, she couldn’t answer; when I asked her why they didnt ask about class size, she said it had been asked before.

The Gates foundation cannot be trusted; and all NYC teachers should boycott this study until and unless the researchers agree to systematically include the factor of class size in their analyses.

I’m surprised at the lack of research fluency both on comments and in the original post. A few quick points: 1) Almost all evaluations of observations right now are being judged against their connection to producing student test-score gains. The question is to what extent are observation-based evaluations picking up the teacher’s value-added scores. 2) The skepticism over value-added seems to stem from union and policy-maker sound bites, both of whom have the whole debate wrong. If you want to see really good work on value-added, its power and its problems, look at the work Kane has been producing out of Harvard or even read Koretz on the issues with testing and test scores and valid inference making. 3) Pointing toward survey data of teachers and parents to prove small classes work as opposed to say, the Tennessee STAR experiment which is an absolutely seminal study in all education policy-evaluation, and certainly in the small class size arena, shows how out of touch practitioners are with what researchers are learning about “what works”. There is a tremendous body of literature on small class sizes– what effect they do or don’t have, when, and plenty of critiques, validations, and cost-benefit analysis. Just as important, there are many, many examples of the counterfactual on small class sizes. Many of the highest performing schools that have been examined do not have small classes. That’s not to say that they don’t help, it is to say that the overwhelming majority of research says it does help, it helps some kids in some situations a lot more than others, but it doesn’t seem to be a necessary precondition for success and while it is the most expensive kind of school reform, it is not the kind with the most benefit potential. 4) Almost every single one of the value-added measures I’ve seen for teachers has used class size as a covariate to lead to unbiased estimates of teacher effectiveness, though it does sometimes happen through some “fancy instruments” like a fixed-effects model which may not be transparent to non-researchers consuming the work. 5) For a great study on the Cincinnati peer review model, read Kane, Taylor, Tyler, and Wooten’s new paper.

Small-class size does go hand in hand with teacher effectiveness and student achievement. STAR concluded that a good teacher will not use the same methodology with students that are not meeting the standards. However, when we are forced to use a “one-size-fits-all model” and 24/7 test prep, real education takes a back seat. Having a large class makes it extremely difficult for the teacher to find one-on-one time with the student. A five minute “How’s it going?” conference doesn’t scratch the surface.

As for JB’s comment, I find it ironic he blames the union when in fact the UFT has been backing this study. But being anti-union seems to be the fade these days. Many years ago my district ran a program in conjunction with NYU where teachers would observe other teachers–either within their own school or another school in the district for 3 to 5 days. We would observe their best practices and take back what we thought would work with our students. It had to be one of the best programs in the city because it did what a film could not–be an active participant. The program was canceled after 5 years due to budgeting. Of course when a program works, it’s the first to go. This is the kind of program that should be funded. Instead some think tank that has never experienced a classroom, students and their real-life problems have the audacity to come into the public schools and use what could be a perfectly good program as a tool against public education. Administrators should not be a part of this program, and the fact that the student release made it clear administrators would be reviewing these tapes made it obvious that this was not the study that was presented to the teachers.

If you want to read a good study on improving student achievement, I suggest you read the works of Linda-Darling Hammond who actually worked with teachers. Alas, her work on creating good staff development models would never be accepted by the politicians and conservative think tanks because it serves to improve the schools without the benefit of starting charters or eliminating senior teachers. Those teachers actually taught without having to worry about test scores to measure their effectiveness. Yet test scores improved. Go figure.

Another study, and another study,…. What about the findings of Linda Darling Hammond more than 10 years ago??? This is a program that should be implemented around our country. The answer is simple…it supports teachers rather than an agenda.

I appreciate your looking at it from another angle–no pun intended. I observed a class a few days ago and produced two pages of notes in twenty minutes. I’m certain my teacher’s eye was more helpful than a camera would’ve been in identifying tips specifically designed for my young colleague. I understand that wasn’t the point of the study, of course.

Whatever the point of the study may be, it behooves them to be honest about it.

Can you clarify, Arthur?

My understanding was that the Gates study is upfront that they are going to videotape the teachers, and the teachers sign off on that.

Is your beef that you thought that meant using a normal video camera (pointed at the teacher), and because they’ve devised a better camera, they’ve misled the participants?

I saw a demonstration of that camera and it’s awesome! You might like it. My rookie teachers are pumped to try it.

But I was told it’s not for sale until maybe October, only Gates teachers get to use it now.

Imagine the teacher-friendly possibilities. I don’t know if you videotape yourself now, but doesn’t it drive you crazy that you basically have to try to reconstruct from memory what the kids were doing?

Or think about that pep talk with a struggling kid. “I need you to focus in class, etc.” How often do those pep talks work? Not enough, right? Now imagine showing that kid the video of herself. Looking hostile. Not taking notes. For some kids, they’ll say “OMG, that’s me?!” Or imagine a PARENT conference when you’re armed with the video.

Um, how can you measure if a method is effective without knowing the teacher is effective?? You observe everyone, then look at which teacher has the best value-added methods, then you go back and look at what the observations show is different about the successful teahers versus those who were not successful. That’s how you find out the methods.

Akademos, I’m not sure you get how this research works at all. It seems you’re suggesting:

1) Observe a teacher.
2) Describe what occurs.
3) ……
4) Say what works and what doesn’t.

Strong research actually has a third step that goes beyond describing what happens in the classroom. If this research is following the process the way this normally works, I would expect (3) is measure student learning outcomes, compare these outcomes to the rich description of what occurred, and determine if there are any common qualities in the observation which matched up with student-level outcomes. Then step 4 is say what works.

You can’t determine “what works” unless you define what “working” means. In most cases, it means that students saw considerable growth in learning outcomes, usually as measured by standardized tests. You can’t identify what practices were actually effective at better teaching students if you don’t look at which students learned best and under what conditions they were sitting.

This has the potential to help us improve on one of the many weaknesses of value-added. Right now it only tells us where things are working, but does nothing at all to describe why or how students are learning better in those classrooms. Adding this observational data provides a rich, thick description of just that (potentially).

The other reason this research is important is to determine if we’re even looking for the right things and if we can measure them to a high enough precision that we’re actually able to use observations to pick up on differences between one teacher/method or another. If we’re not, we need new, better observation protocols.

JB, this is about intentions and agendas, not research.

But, even if it is all on the up and up, I do have profound doubts about curricula in this era of standardized testing, the present standardized testing itself, and researchers’ abilities to amass meaningful information through observation of students learning, as if they already know how people learn, when that is part of what they are trying to figure out. Competent teachers are way ahead of this curve and can intuitively factor in and out cultural and community differences, personal styles, behavioral dynamics and school missions and themes.

If the research can truly give insight into thinking and learning, great. But so much of this stuff seems to be data-driven garbage in support of other data-driven garbage.

Should say, “I regret..” Sorry.

JB – point taken on research design. But this boondoggle still stinks from a mile away. Even with years of data covering entire districts or states, linkage of teachers to test score gains are imprecise and unstable. With much smaller (and self-selected!) samples of teachers participating in this study, it’s an exercise in trying to find out what practices correlate with fluff. Even if something “statistically significant” emerges from it (unlikely), it would be highly suspect. Reliance on annual state tests as a sole measure of student learning is a major problem with this study. A lot can happen to a kid in the course of a year. The few hours a week that a kid spends in a teacher’s classroom has relatively small influence.

If Gates really wanted to do something useful, it would have funded much smaller and targeted studies.

JB, I read the paper you mentioned above. Look, I don’t doubt the statistical models or the reproducibility of test score patterns based on teachers or styles. However, test scores, especially standardized multiple-choice test scores, do not equal depths of understanding, internalization or developmental milestones. (The 50% fade out does seem a bit suspicious, though. Why would retention work out like that? Or is it that there are so many other factors in the adjustment to a new teacher that it becomes sort of a wash, 50% adjust well, 50% don’t? The law of averages can trick you when you don’t know all of the variables.)

I’d like to see more research on the effects of things like software, online learning, SMART Boards, etc. And I want to see less research done that involves things we don’t understand and don’t know how to quantify yet, or that should not be quantified in the usual sense, UNLESS the goal of those studies is to actually gain a better understanding of those things. Suppose one of these studies dissuades a certain style or method due to how it pans out in the averages (once you’ve squeezed out the noisy variances), yet while on the average you get that slight uptick in scores or engagement, you completely lose and alienate a few students who otherwise would not have been lost. What good have you done? And to put the professionalism of teachers and staff below faulty tests with ludicrously low bars and gigantic swaths of random chance is reprehensible.

Thanks for the quick reply. I think you’ve understood my points.

I’m not hating on the studies, I’m playing devil’s advocate and pointing out the human elements involved in this.

Understood, standardized testing is one of those oxymoronic things: worst of all possible evils, yet better than everything else. Just keep in mind, it does not really test learning itself or teaching itself. As an alternative, one could ask students for explanations, definitions, demonstrations, etc.

To me, personally, the science is way too soft and there’s danger of mere dabbling here, when the stakes are extraordinarily high.

We all know the Knicks need to get LeBron.

All the research about magic bullets and best practices is a bunch of hogwash because what works in one school/community/classroom/for one teacher won’t work in another setting. There’s a very large universe of good ideas that can work depending on the objective, students, teacher, principal, etc. etc. context. If you want to know what works, read The Basic School by Ernest Boyer. There’s no magic recipe, just a community that’s all on the same page and rowing in the same direction.

Riffing with KS (my snowball bud) and Akademos,

Just as the Knicks could really use LeBron, one could similarly argue the flip side, that Cleveland might prefer to hang on to LeBron than hire someone fresh out of training camp from say…. Dunk for America.

But while LeBron might not have tenure, you only have to go back a few years to recall that Cavaliers fans didn’t measure his value by the team performance, let alone on the team’s standardized statistical metrics.

And the above AP could learn a lot from the best NBA coaches, some of whom are dogmatic, some of whom win with whatever players they have because while they may have a “system,” they know how to blend philosophy and personnel, and build a sense of team.

Contrast with our system: the Owner (Mike), gives an unconditional pass to the Manager (Joel) and coaches (principals), and blames the players (teachers), while turning a deaf ear to the erstwhile fans (kids and parents). That’s how the ball bounces at Tweed.

Careful Miss T,

It’s not PC to put kids or parents into the debate. (/sarcasm)

How would your TPS peers have done in that environment?
How would the charter teachers have done in yours?