And another thing… Manhattan saw a spike in applicants to Kindergarten (not sure about other grades) from families who might in other years have afforded private. Sort of a subset of ceolaf’s just prior comment at 4:21 pm.

You are getting at a slightly different question than Sen. Huppenthal asked, though probably what he was trying to get at.

That is, are different families sending their kids to NYC public schools — or in different proportions — than eight years ago? (The difference is that the senator asked about the overall demographics, whereas this question asks about comparisons between individual families.)

Well, that’s hard to know. We’d have to be tracking those who left the system (i.e. transfers, dropout, graduations) and those coming in (i.e. transfers and matriculaters). We’d also want to compare cohorts, to see if there are changes in different grade levels — but that poses the difficulty I described above, in addition to many others.

My point is that either way it is a hard question to answer. Are most people better off than eight years ago, the recent market collapse notwithstanding? It depends on your approach to how you would answer the question.

How would we sort out the change in socioeconomics of any given family, or collection of families…. and the flat-out change in who lives here?

But it wouldn’t surprise me if the “Bloomberg bump” were to prove out to be a combination of rising tide floating all boats (i.e. statewide rising test scores, even in cities WITHOUT mayoral control let alone Mayor Mike), and, people filling up new luxo-towers have kids who do better on tests than the previous average New Yorker’s kids.

Isn’t that a political question? I mean, the Thompson campaign is claiming that the costs of living in the city have gone up far faster than wages. How do you want to correct for that?

I point this out to get an the important distinction between proxies and the real underlying constructs. We would expect to adjust income for inflation, but in the short or medium run that will be misleading in this kind of calculation.

Think about it this way: Your socioeconomic status (or the things we lump in with) has indirectly helped your kids to do well in school, right? You are able to give them certain advantages — including simply speaking to them with your large vocabulary, providing a quiet place for them to do their homework and demonstrating the importance of education without even having to lecture them about it. These are good things. But if you income dropped in half for five or eight years, you’d still provide virtually all of the same advantages.

However, someone whose income was always half of your would probably not have been providing all of the advantages.

You see, short term change in income does not really match long term socio-economic status (a vague, though important construct). Or, you can think of the Jane Austen novels, in which the highest SES families married the highest income families (i.e. the new rich) to restore some income to their recently impoverished families.

I think that you mean to ask if the composition of the city’s public schools has changed sufficiently in the last eight years to have an impact on expected test scores, right? I don’t know the answer to that question, and serious researchers would probably have trouble figuring out how to answer it in a truly valid and rigorous fashion.

This is you view as a parent, and whether or not it is unreasonable, it certainly is understandable.

But what about your view as a citizen and/or a member of a larger community? Do you have any responsibility there, or is it all only about what you and yours can get out of the system?

And what about kids who don’t have parents as active and involved as your own are lucky enough to have? As a citizen and/or member of the community, do you have any responsibility to look out for them at all?

While I am sympathetic to your meaning, I think that your words could hardly be more wrong.

The NAEP *does* help kids. It doesn’t help the kids who take it, I understand. But it *does* help kids that will follow them. The kids who took the NAEP in the past help today’s kids. It is an ongoing thing.

This gets at that important distinction between private good and public good to which I keep referring. I understand that your child taking part in NAEP testing does not contribute to that child’s private good, but it does contribute to the larger public good. I understand that you — when thinking about your child and your child’s time — need to be primarily concerned with your child’s private good, but you also need to consider the public good. I am not saying “either/or.” I am saying “both/and,” with room for individuals to balance them differently.

I believe — and I think that I’ve written about this on this site — that we need to think carefully about how much time/effort we demand of students for things that just make it easier for the adults. But that doesn’t mean that we don’t ask of the kids anything that does not benefit them directly. It means that we need to be careful about how often and how much.

However, it is not reasonable to expect that your child should be exempt from contributing to the betterment of our schools.

*********************************

Another problem with what you have written is how you equate state standards and state testing. Have you looked state standards and the state tests? Are you sure that the latter is good reflection of the former? I support your interest in seeing your child progress and advance on state standards, certainly. But how do you check the tests?

The NAEP is one way to do that. That is, the NAEP can help all of us tell whether the state’s tests are shifting in difficulty over time. It can tell us if gains on state tests are a reflective of teachers getting better at preparing students for the particular kinds of items on the state’s tests, rather than at teaching the underlying constructs that those item are supposed to test.

To do that, we need NAEP results over time, which means we need kids in the past, present and future take the NAEP — or another similar test. It is in your child’s interest that we be able to do this, even if s/he does not benefit directly from his/her own participation in the test.

**********************************

If you are happy with the state of our schools, we probably have nothing to talk about. But if you think that they should be better — not just your child’s school but schools generally — then we need ways to measure progress. Perhaps the NAEP is not the way to do it, but your objections would apply to any substitute for the NAEP.

**********************************

And last, what you mean by “educrats”?

Assuming that you are being serious….

* If the feds read the Constitution the way you do in this regard — and you are right, there is no mention of “education,” and “promote the general welfare” might not include a federal role for education — there wouldn’t be any NAEP at all. Doing it less often is no answer to unconstitutionality. So, clearly they do not read it the way you do. In fact, only a tiny minority reads it the way you do, and they are not in a position to make your reading binding.

* There are decades of jurisprudence that leverage the interstate commerce clause into a wide variety areas. Perhaps you think that it overreaches, but you must acknowledge that it has some applications.

* Your objection certainly is strongest when applied to federal mandates. However, the federal government does rather little with mandates in education. Usually, it is done with conditional moneys. That is, if states wants federal money for Title I (for example), they have to do certain things. But they can avoid doing those things if they simply forgo those funds. By this mechanism, the Feds are not exerting power over the states, and therefore — so the theory goes — not violating the Constitution, particularly not violating the 10th amendment.

* This does raise significant questions about national standards, no doubt. How would the feds enforce national standards? Would they continue to link them to Title I funding? Put too many strings on Title I funding, and a large enough coalition will form to turn it down.

* Penultimately, I think you are mistaken about the purpose of the NAEP. As a parent, the NAEP does not help you with the education of your current children. There is no way to get NAEP scores for a single child or even a single school. The reporting and analysis takes a long time — and continues for years. The kind of information you might want, as a parent, is not there there in the NAEP. Teachers concerned their students or trying to improve their own pedagogy is similarly out of luck, with regard to the NAEP.

On the other hand, policy-makers, researchers and other education professionals often look for different kinds of information. If you believe that *serious* reform (e.g. policy change, curricular change and even pedagogical change) takes years to implement, the relative infrequency is not so much a problem — quite different than a parent’s concern about his/her child’s individual progress. More importantly, having a common high quality test across states is critical. NCLB — which obviously you don’t support — assumes that different states will have slightly different approaches. Learning from this diversity requires some kind of common measure. The same can be said of charter school school legislation and any number of other state-level educational policy initiatives. Very few states have multiple large urban centers, so comparing different approaches in those contexts requires is much aided by a common national test.

* And last, whether or not you depend on the annual evaluations you receive from your local state should be informed by the validity and reliability of those measures. If the NAEP tells us that state testing is reporting inflated gains, that *does* apply to your children. This does not mean that the NAEP tells you about your child. Rather, it suggests that might instead look to other assessments. No doubt, your child’s school offers multiple reports a year about his/her progress. They might prove more useful to you than that state’s reports.

1) Everyone loves the idea of disaggregation of student data. Rather than looking at whole-school or whole-grade averages, NCLB requires that subgroups be independently reported (i.e. all students, black, hispanic, white, asian, american indian, english language learned, migrant student, low income and special ed, as I recall). Of course, there have been issues with this approach, and certain amounts of controversy. But the idea/goal has received, as best I can tell, universal acclaim.

Unfortunately, there is a technical problem with this approach. For statistical purposes, the more kids/subgroup, the more accurate the reporting. A subgroup with just 5 students, for example has a HUGE margin of error. The real statistical need for subgroups of at least a certain size is at odds with the moral goal of making sure that schools pay attention to their local minority groups. It’s a tension without an idea answer, leading to compromises that satisfy neither the testing/statistical expert nor the other advocates.

2) There are a aspects to the NAEP that make it such a high quality project. One of them is that the folks who run it know that they are looking at the status of very large numbers of students as a group, not as individuals. So, even in the smallest states, there are thousands of kids in each grade.

The NAEP does not give each kid the whole test, but rather — as Aaron explains above — just gives a faction of the kids a faction of the questions each. Because there are so many kids in the first place, each question is answered by many many children, even though no one child gets that many questions. This allows for a much larger set of questions, overall — and therefore better content coverage and greater reliability (in the statistical sense).

3) Well, this NAEP approach makes it even harder to do sub-group reporting. The NAEP requires many more students be available to take part in testing because no one kid does every question. That would blow the minimum subgroup size needed for valid reporting — which is already set to small in most states for it to be statistically reliably while simultaneously being so large that millions of minority students’ scores are not reported as part of subgroups. (i.e. If a school’s subgroup size does not meet a certain state-dictated threshold, it is not independently reported.) This is already a problem in small schools, even without NAEP style testing.

In other words, a NAEP style test would virtually eliminate subgroup reporting on the school level, and no amount of money can fix that.

What’s my stance on national standards and assessments? Ambivalence. The lack of national standards, and assessments of whether those standards have been met, results in an ambiguity about what a high school diploma means. Given the variability across states in standards and assessments, one can graduate from high school without knowing anything in particular. Requiring students to meet a common set of standards would remedy this.

But I’m not sure who I trust to come up with national standards. In a democracy, the content of what is to be taught and learned in public schools is something in which all citizens have a legitimate stake. I’m wary of delegating the authority to develop national standards to elected politicians, whose interests and experiences may not mirror the remarkable heterogeneity of American society.

Conservatives generally have been wary of federal standards, which is a big reason why NCLB has states each set their own standards and tests. Liberals are a bit more likely to go for federal standards, but they haven’t had a lot of faith the kind of testing regime that NCLB has mandated.

So, there was no natural constituency for linking a federal test to accountability provisions.

However, after all these years of NCLB, things are shifting around a little bit. There are more calls for national standards from the right, and more calls for better testing from the left. The left does not automatically reject the idea of testing as much as it used to, and the right is more accepting of bigger role for the DOE in Washington. What used to be inconceivable now just seems really unlikely.

This doesn’t mean that the NAEP can be used — but most people with bully pulpits don’t understand that the NAEP doesn’t work like other tests. So, there has been more of a call for its adaptation than is warranted.

I, for one, have no idea where this is going in the next 8 years.

Anytime some defends their own performance or analysis by attacking the NAEP, it is clear that either they generally don’t know what they are talking about or are hiding something. Either way, nothing else they say on the matter on which they are defending themself is credible. (The same is true for statistical sampling. Attacking both is a double-whammy to credibibility.)

This, of course, it not to say that sampling and the NAEP are invulnerable to criticism. But they are so well established and lauded that such criticism should stand on its own, so that it can get full hearing.