GothamSchools — daily independent reporting on NYC public schools

I could not agreee more with that analysis.. Your first mention of parents selecting a school is accurate. As a public school teacher we would kill for more students who had parents that took an active role in their child’s education. It is by no mistkae the the few students we do have with active parents by in large graduate in 4 yrs. The very existence of charter schools at all is by it self a “creaming process” because we are drawing these students away from the public schools. I am not condeming the right of a parent to do what they feel is necessary for their child’s best interests but when we hear the screams for more charter schools based on results when measured against the public schools lets remember that we are not talking about an evenly stacked deck.

I’ll answer that question: I’m not opposed to your creating a school that realizes your unique vision for what it takes for students to succeed. In fact, you may be doing things that I wish I could be doing in my school. But when I, the regular public school teacher, get stuck with the leftovers – the kids who didn’t bother applying to your school or who dropped out because they didn’t measure up – please don’t use your superior statistics as justification for those who would take away my right to work under a union contract or deny your teachers this basic democratic right.

Aaron/Skoolboy makes some excellent point here.

I’d like to flesh them out, just a little bit.

1) We know that schools are being compared. Politicians compare schools. Newpapers compare schools. The DOE compares schools. Property owners/buyers and real estate agents compare schools.

2) Politicians, public figures and many in the education sector point to the apparent success of some schools to highlight the apparent failures of other schools. And then, they often claim that this “proves” the failure of particular methods or tools at those “failing” schools, or that this “proves” the importance of some particular methods or attributes of the “successful” schools.

If you have two schools — or two sets of schools — with different success rates and a bunch of known differences in methods or attributes, to which method(s) or attribute(s) ought you credit the differences in performance?

3) These claims/inferences are made without looking too closely at what is going on in each school or set of schools. In other words, many of the differences are not well known, or at least not known or understood by people making very public pronoucements.

4) Mobility is a huge issue, one that is not properly accounted for in quite a bit of the pro-charter school research.

There is good reason to only include kids who stayed in a charter school for the whole run. You want to be sure that you are getting the “full treatment.” But there are a couple pontential problems with doing this.

- If you compare the results of those kids with all public school kids, you are not comparing apples to apples. Even if you compare them to kids who have stayed in the same school system, that other group might have greater mobility — having moved from one regular public school to another. It is not always easy to find data of kids in public schools who have not changed schools in given period of time, especially if they stayed in the same system.

- As Aaron so ably points out, the kids who stay in a charter school with high expectations over many years might be substantially different in important ways that the kids who move or change schools. If you don’t look for those differences, you might assume that the overall attributes of original group should be taken for attributes of the later group(s).

Dirk,

Aaron/Skoolboy’s biggest point here is that we have to look really closely at the larger recruitment/admissions process, but we can’t JUST look at recruitment/admissions process. (And we have to look more closely at recruitment/admissions than we do.)

We also need to look at what happens to shape the student body after the recruitment/admissions process is over — especially if we are going to be comparing the results of the remaining students to other groups students (e.g. those at other schools).

This summary of the creaming effect is great! The widespread lack of comprehension is baffling (willful or not?).

Dirk, I’m a San Francisco parent/blogger and am apparently the instigator of the “comprehensive study on KIPP attrition in the Bay Area.” “It occurs” is an extreme understatement.

As I’ve already posted elsewhere on this blog, the numerous academic researchers and hordes of admiring press looking into KIPP schools had neglected to check into attrition at all. I got curious and looked up the year-to-year enrollment figures for all the California KIPP schools. (I checked the California schools because I know how to look them up on the California Department of Education website; I don’t know how to do this in other states.)

After I blogged those figures, they got attention because the attrition was so startlingly high — and was consistently much higher for the most academically challenged subgroup (statistically speaking) in the schools. At Oakland’s KIPP Bridge Academy, 77.2% of the African-American boys who started in grade 5 in fall 2002 did not make it to the FALL of 8th grade (the number of those remaining who COMPLETED 8th grade is not discernable from the publicly available figures).

A study of the Bay Area KIPP schools conducted by the research organization SRI International, undertaken after I had blogged those enrollment figures, did look into the attrition, and confirmed my layperson’s findings. That study showed that 60% (overall) of the students who start those KIPP schools don’t finish — and that those who leave are consistently the lower-achieving students.

Here’s a link to my post summarizing (and linking to) that SRI study:

http://tinyurl.com/42lc64

A crucial point is that unlike traditional public schools, those KIPP schools don’t replace the students who leave, as the figures confirm. Thus they end up with a streamlined class consisting, overall, of the 40% who were the higher achievers.

Jay Mathews has responded to this both in his new book about KIPP and here and elsewhere by claiming that I’m ignoring the fact that the attrition has dropped since. But actually, the SRI study was released in September 2008, and more recent enrollment figures are not yet available, so that criticism is not valid. (Plus, if the attrition DID drop once there was a sudden spotlight on the attrition, isn’t that a big asterisk on the claim that the attrition dropped? Would it have dropped if it hadn’t come under public scrutiny?)

Aaron,

This is a thoughtful and provocative post. I’m out of the country right now for February break and I’ll try to find time to write a more detailed point-by-point reply next week addressing your specific examples.

In the meantime, I think we actually agree on a key point, the practice of creaming is counter to improving the public education system and steps must be taken to acknowledge and counter-act the forces that promote creaming—whether they are intentional or de-facto. Creaming is a negative thing whether it is happening in public charters, private schools, or traditional public schools and should be called-out publically with unbiased statistical data wherever possible and efforts should be made to reduce selection bias in access to excellent quality schools like Democracy Prep.

I was surprised that your post didn’t offer an analysis on my larger point that traditional district schools–in Harlem in particular–do far more creaming than public charters schools, and that traditional public creaming should be acknowledged as a problem that is not exclusive to charter schools and that in fact, in Community School District 5(CSD5) in Central Harlem, most charters tend serve disproportionately MORE challenging students than the district based on the percent of IEP, 504 Plan, ELL, low-income, low- incoming academic performance, and students of color—if you look at our comparative data you’ll see that this is especially true at Democracy Prep. Moreover, as you correctly identify our sincere attempt to make admissions open and to counteract the effect of de-facto creaming means that Democracy Prep makes every effort to mitigate possible natural selection bias by reaching a wider audience of potential families than any other charter middle school in New York. We mail a bilingual 1-page lottery form to literally every 5th grade family in district 5 and send materials to many people within every traditional public school, ACS, Special Education coordinators, and community groups as well.

However, selection bias and post-enrollment attrition are two very different things and should be understood and discussed as such.

Let’s focus first on post-enrollment attrition which is the focus of your post. The assumption that departed students are somehow less academically successful and are weeded out is purely speculative–we neither have data to show that is true generally, and it is certainly not true at Democracy Prep. Much of our attrition is made up of students who move out of state, some students move out of commuting distance within New York, and still others are creamed from Democracy Prep to enter selective programs like Prep-For-Prep. But more importantly, to get a better understanding of the students we serve we must use comparative statistics on the transience/mobility within Community District Five traditional public schools, which I contend are higher than at public charters. The Chancellor’s Progress Report and Peer Index hold constant students’ incoming test scores and takes attrition into account, and Democracy Prep still performs incredibly well against the city and a group of our peers both in terms of absolute and especially value-added academic performance as well as parent/student/teacher satisfaction on the Learning Environment Survey.

Furthermore I would argue that the contention that “students who are stable differ from students who are mobile, and that stability promotes achievement” does not necessarily hold true for Central Harlem/CD5 in particular. As a sociologist who has written about the complexities of local situations, I’m sure you’d agree that it is important to note that in a community like Harlem with large housing projects and significant multi-generational poverty, stability may actually be an indicator of a higher likelihood for low achievement. Anecdotally, we have observed that our most engaged families are those who are recent immigrants (regardless of origin), economic strivers, and those who have left the traditional public schools because their students were failing. Thus, we have found that mobility/transience overall is more closely correlated with parental engagement and not parental disengagement. Conversely, many of those families that have had to attend the same zoned failing traditional schools in CSD5 generation after generation are least likely to succeed academically.

Your point that we had a higher level of attrition in our first year is true, but like all new schools, our first year attrition will be our highest. Incidentally, based on the national data that indicates that new schools in general have higher attrition rates in their first years, we intentionally over-enrolled that year for August orientation. Our normal program aims to have 100 students per year take the state tests in grades 6-8 and we’ll see how that pans out going forward, I’m optimistic.

Before I post a longer point by point reply to both the policy questions and the specific example of Democracy Prep next week, I hope you can find time to apply the same precise analytic framework you use here to the “creaming” of the traditional public schools in District 5 in Central Harlem. I’d suggest you start with the 28 page DOE middle school selection guide that can be found at: http://schools.nyc.gov/NR/rdonlyres/9ADB6AC4-14E7-4397-8B5E-E39B58496493/49090/_D5Directory_112009.pdf. It includes a list of all the 11 traditional public middle schools in district five including six that have stated admission criteria including “Review of Grades: Minimum GPA of 85 in Science, Math and Literacy classes. Review of Test Scores: Minimum of level 2 on both NYS Examinations, English Language Arts and Mathematics. Student Interview, Essay, Diagnostic Test.”

Then review the mandatory NYC DOE Discipline Code and Behavioral Contract at http://schools.nyc.gov/RulesPolicies/DisciplineCode/default.htm and decide which document requires more of students and parents, the Democracy Prep Handbook or the 32 page long required DOE Discipline Code and Behavioral Contract.

I’m glad to have started this dialogue on the true meaning of and definition of “creaming” but I don’t want us to miss the bigger point I was trying to raise. The core issue to me is ensuring that every single parent has a wide selection of excellent schools to choose from, regardless of the zip-code or CSD in which they happen to live. Public Charter Schools like Democracy Prep are providing an equal opportunity with totally open access to all parents in Harlem who, for decades, have never had decent choices unless their child scored high enough on state tests to be accepted into an elite Gifted and Talented public school or program.

Finally, I hope that going forward, passionate researchers, academics, parents, and teachers like yourself, will focus increased attention and provide new insights on how to deliver to those parents and teachers trapped at failing schools, in failing districts, and with limited resources the rigorous college-prep education we provide our students every day. There are exceptional teachers and school leaders at the traditional public schools and they should be applauded for their hard work but that doesn’t mean that all of us shouldn’t continue to push for reforms that discourage creaming and improve the quality of education for all children.

I’d also extend an invitation to anyone who reads this post to 1) spread the word to all families that our lottery form is currently available to help further reduce potential selection bias 2) come visit Democracy Prep to see the school in action, and 3) attend our public lottery on April 6th where more than 1,000 families will likely be waitlisted for their chance at an excellent education while only about 100 are given a chance at a free college-prep public education. Decide for yourself if we are trying to cream our students, or if in fact we are working with students and families who are only “different” because they were lucky enough to be selected in the lottery. If you notice a difference in parental attitude or motivation, I hope that you’ll ask those selected about whether they are grateful for the opportunity they have been given and what the effect has been on the parental motivation for so many other equally deserving families who have been devastatingly denied an opportunity to an excellent education at Democracy Prep.

This summary of the creaming effect is great! The widespread lack of comprehension is baffling (willful or not?).

I tried to post a response many hours ago, but it was “awaiting moderation” because it had a link in it, so I’m reposting a version without the link. Sorry that the near-duplicate may show up at some point too.

Dirk, I’m a San Francisco parent/blogger and am apparently the instigator of the “comprehensive study on KIPP attrition in the Bay Area.” “It occurs” is an extreme understatement.

As I’ve already posted elsewhere on this blog, the numerous academic researchers and hordes of admiring press looking into KIPP schools had neglected to check into attrition at all. I got curious and looked up the year-to-year enrollment figures for all the California KIPP schools. (I checked the California schools because I know how to look them up on the California Department of Education website; I don’t know how to do this in other states.)

After I blogged those figures, they got attention because the attrition was so startlingly high — and was consistently much higher for the most academically challenged subgroup (statistically speaking) in the schools. At Oakland’s KIPP Bridge Academy, 77.2% of the African-American boys who started in grade 5 in fall 2002 did not make it to the FALL of 8th grade (the number of those remaining who COMPLETED 8th grade is not discernable from the publicly available figures).

A study of the Bay Area KIPP schools conducted by the research organization SRI International, undertaken after I had blogged those enrollment figures, did look into the attrition, and confirmed my layperson’s findings. That study showed that 60% (overall) of the students who start those KIPP schools don’t finish — and that those who leave are consistently the lower-achieving students.

Seth Andrew posted: “The assumption that departed students are somehow less academically successful and are weeded out is purely speculative–we neither have data to show that is true generally, and it is certainly not true at Democracy Prep.” But Seth’s comment is inaccurate in the case of the Bay Area KIPP schools. The SRI study I mention DID find decisively that it’s consistently the lower-performing students who leave. It is not at all “purely speculative” but is confirmed by an academic study that was acknowledged and publicized by KIPP. (Obviously this does not address Democracy Prep.)

A crucial point is that unlike traditional public schools, those KIPP schools don’t replace the students who leave, as the figures confirm. Thus they end up with a streamlined class consisting, overall, of the 40% who were the higher achievers.

Jay Mathews has responded to this both in his new book about KIPP and here and elsewhere by claiming that I’m ignoring the fact that the attrition has dropped since. But actually, the SRI study was released in September 2008, and more recent enrollment figures are not yet available, so Jay’s criticism is not valid.

(Plus, if the attrition DID drop once there was a sudden spotlight on the attrition, isn’t that a big asterisk on the claim that the attrition dropped? Would it have dropped if it hadn’t come under public scrutiny?)

By the way, the notion that lower-achieving students are likely to be from LESS-mobile households is counter to everything I’ve ever read or heard. Not being a professional researcher, I’m not in a position to address that with data, but it definitely sounds like a contrarian view to me.

After Schoolboy’s concise summary, we have to then consider Caroline’s phrase about the baffling inability of some to understand and whether it is wilfull or not.

In my experience, many intelligent outsiders simply do not understand the definition of creaming and why it is so powerful. Others are reacting to educators who are resistant to so-called “reforms.” In those cases, we of the so-called “status quo” should ask whether we contributed to this wilfull refusal to acknowledge the obvious. Have we contributed to a fight that is so polarized that some try to deny the obvious?

On the other hand, others have a political point and they are sticking with it regardless of facts and logic. Its sad, because now is time to put the blame and shame of NCLB behond us and build on the opportunities we have with the stimulus and the Obama administration.