GothamSchools — daily independent reporting on NYC public schools

Here’s one more reason state tests might be getting easier to pass: a longstanding State Education Department practice of publicly releasing every question on each year’s exam.

The unusual practice makes it harder for test-makers to gauge how difficult a test is, said Howard Everson, chair of the state’s Technical Advisory Group, an oversight committee that monitors state testing.

Many states release some test questions but keep others private so they can be used again to compare one year’s test to another’s, said Daniel Koretz, a Harvard University education professor who studies testing. But New York has long had a practice of releasing every single test question to the public soon after students sit for the exams. (more…)

Mayor Michael Bloomberg is offering to open up. Photo via Wikimedia commons.

Responding to the national push for more transparent government, the Bloomberg administration is opening up some of its datasets for easier public consumption. The only question is what data the city will throw up on the new Web site.

The city is taking suggestions starting Monday, and the nonprofit that houses GothamSchools, The Open Planning Project, is part of the push to send those in. We will be helping TOPP fill out what are called RFEI’s, or requests for expressions of interest, this coming Monday.

With the deadline breathing down our necks, on our staycation no less!, we need your help. Our wish list includes information on outside contracts the Department of Education holds, school-by-school budget documents, and school accountability information organized in easy-to-search Excel spreadsheets rather than individual PDF’s.

What should we add? Please name names of specific documents, and please don’t be shy with ideas. Info on how to submit your own RFEI is here.

One of the questions given to a council member on Wednesday. (GothamSchools, Flickr)

Like Elizabeth, I was surprised to see a representative of the teachers union hand City Council members pre-printed cards with questions to ask during yesterday’s hearing on charter school expansion. Apparently, council members were taken aback by the move as well.

Organizations frequently provide information and suggestions for council members to use during hearings, according to a spokesman for Simcha Felder, the councilman who shared the cards with us. The spokesman, Eric Kuo, said Felder doesn’t think there is anything improper about the United Federation of Teachers and other groups suggesting questions to council members as one of their advocacy strategies.

But Kuo did say that the union took the strategy to a new level yesterday. “During the last hearing, it was more aggressive than before,” he said.

At the hearing, Felder drew uncomfortable looks from union officials sitting in the front row of the audience as he shared the cards with GothamSchools. Still, he even got up from his seat to collect the UFT’s question cards from his colleagues and pass them over to Elizabeth as well. Kuo said Felder shared the cards out of a commitment to transparency. “He doesn’t think it should be a secret” what council members have on their desks, Kuo said.

City Council Member Melinda Katz introduced a resolution asking the state to change the law so that the Department of Education is required to follow city contracting rules. (Via Azi's Flickr)

Comptroller Bill Thompson attracted lots of press Wednesday by accusing the Department of Education of “runaway spending” on contracts. But another, less sexy development could have a much greater impact.

That’s the fact that momentum is growing to force the department to follow the same contracting rules as other city agencies, in the form of endorsements from a list of advocates, including one office that rarely butts into policy debates, and a new City Council resolution calling on a change in the state law that allows the DOE to duck the usual regulations.

Agencies from the NYPD to the parks department cannot hand taxpayer dollars over to an outside contractor without first following the rules of a citywide office called the Procurement Policy Board. The DOE is the only city agency that does not have to follow the board’s rules, which do everything from forcing public hearings on contracts above a certain price to imposing strict guidelines on what contracts have to be bid competitively.

The DOE’s exception was born well before the 2002 mayoral control law gave the mayor authority over the schools, but it has gotten more attention under the new structure, which makes school contracts harder to track. While the old Board of Education reviewed all contracts above a certain size before they were signed and held public hearings where citizens could respond to the contracts, the Department of Education has presented only a small number of contracts before the new version of the board, the Panel for Educational Policy.

The result is that hundreds of contracts have been offered without competitive bidding — and without a public hearing to discuss what the contracts include.

A group of Columbia Journalism students has reported that the DOE also makes it difficult to find contracts once they’ve been signed. The department does not maintain reading rooms for the public to review contract documents, against the requirements of the Freedom of Information Law, and many contracts simply aren’t available for review, they reported. Asked about the concerns at a City Council hearing Wednesday, school officials said they would look into them.

A Public Airing

The lack of PEP hearings is despite language in the state law that gives the panel the power to “approve contracts that would significantly impact the provision of educational services or programming within the district.” (Read a PDF of the law here.)

Patrick Sullivan, a PEP representative from Manhattan who is a critic of the Bloomberg administration, told me that he has seen only labor contracts come before the PEP, never a goods-and-services contract. Sullivan said that he recently asked the department to submit a new $79 million contract with a firm called MAXIMUS to manage special education data for a PEP vote.

The department’s general counsel, Michael Best, denied Sullivan’s request in an e-mail message that I obtained, though he did offer to share some information about the contract — after the meeting had happened. Best wrote:

If you really want to see the contract, we do not have an electronic version to send around, but if you were willing to come down to tweed we can arrange to let you take a look at it.

Sullivan, who was appointed by the Manhattan borough president, Scott Stringer, said he was not satisfied. “If the PEP had to vote on the contracts, then there would be some accountability there. Then we would be holding Klein accountable for the spending,” he said. “Because they refuse to allow any of those, and they just spend whatever they want and whenever they want, they’re refusing to comply with the accountability requirements of the law.”

A spokeswoman for the department, Ann Forte, said of the contract, “We do not believe Panel approval was required.”

City Council members would urge state lawmakers to make that change under a resolution introduced this week by Council member (and comptroller candidate) Melinda Katz. “It is amazing to me that there would be allowed any exception to what any city agency must do,” Katz said at a hearing Wednesday, announcing the resolution.

School officials yesterday declined to follow an invitation from Katz to self-impose the restrictions other agencies follow. They said the department’s exception is important because it allows the system’s 1,400-odd schools to buy things like copy machines and textbooks on their own, without having to navigate a maze of regulations. “They need the flexibility, within accountability guidelines, to actually make the purchases necessary for their students,” the department’s chief operating officer, Photo Anagnostopoulos, said.

Best, the department’s general counsel, said other mayoral agencies must get every contract they write reviewed by a chief contracting officer. That would be very difficult in a system of 1,500 schools, he said.

Katz and other advocates said Wednesday that the exception means the department’s contracts fly under the radar of proper oversight.

George Sweeting, the deputy director of the city’s Independent Budget Office, added his endorsement to the resolution, in a move he said was unusual for the IBO, which usually stays out of policy debates.

“The PPB rules are intended to improve transparency, avoid excessive costs, and reduce the potential for favoritism that can result in the absence of competitive bidding,” Sweeting said in prepared testimony. “It is difficult to understand how those rules are considered useful when other city agencies procure goods and services, but unnecessary or too cumbersome for the DOE.”

The Speed Imperative

City Council members also pointed to the department’s $16 million contract with Alvarez & Marsal, the consulting firm that re-arranged the school system’s bureaucracy. The contract attracted attention because it was awarded without any bidding and because it led to the 2007 scandal where a midyear rerouting of school bus lines left many children stranded in the cold. The department has said the bus routing was a mistake but defends the rest of Alvarez & Marsal’s work, which it says saved the city $170 million.

David Ross, the department’s head of contracting, told City Council members Wednesday that Schools Chancellor Joel Klein awarded Alvarez & Marsal the contract without any competitive bidding because he felt a time crunch. “The chancellor had an interest in completely making extensive changes to the school system and operations,” Ross said. “It was felt that it was just not practical or possible to do an RFP or competitive process and make the reforms and changes that were needed in the schools.”

He said that Alvarez & Marsal “had the advantage” because they had already begun working with the school system under a contract with the Fund for Public Schools, which used private philanthropic donations to start off work with the firm. “They were already there. They had done a lot of the work,” Ross said. “So the inertia behind them was already very significant.”

School officials repeatedly called the Alvarez & Marsal contract unique. In an interview yesterday, Ross told me that the department handed out $28 million in no-bid contracts in 2008, a number he said is low compared to years past. In testimony to the City Council, Anagnostopoulos said the so-called “exceptions” contracts were all less than $5 million in value, and 85 percent of them were with community-based organizations that run pre-kindergarten classes.

Here’s something to sift through over the weekend: The State University of New York’s Charter School Institute this week decided to renew three New York City charter schools’ right to exist. Two Bronx schools, the Grand Concourse Academy Charter School and the Bronx Charter School for Excellence, and a school in Brooklyn, Excellence Charter School, won the renewal, which lasts five years.

The renewal news is important because it highlights a major way that charter schools are different from traditional public schools. In exchange for being free of Department of Education bureaucracy, they must prove every five years that they should continue to exist — or face extinction from one of the three “authorizers,” of which SUNY is the most respected.

Even more exciting, the renewal news is a reminder of the renewal reports, which SUNY publishes in full on its website — and which are worth a look, particularly as the debate over charter schools heats up. Each one includes not only a detailed description of a school’s plans, but also almost endless charts chronicling its test scores, demographics, enrollment patterns, and how much money it spent per student.

There are tons of miscellaneous tidbits, too, which I hope everyone posts in the comments. Here’s my contribution: According to the most recent report on KIPP STAR, KIPP New York’s plans to build a high school that their middle school graduates can attend are moving along — and slated to cost an initial $188,000 in private donations that initial per-pupil funding won’t cover.

Tania Shinkawa at her desk in the bull pen at the Department of Education's Office of Portfolio Development.

Last week I grumbled about a problem that was, at worst, a deliberate obfuscation and, at best, an annoying characteristic of Department of Education spreadsheets. The spreadsheets in question were supposed to convey two facts about every school in the city: 1) how much money the school had received from the state’s $150 million class-size reduction pot and 2) how much the school had actually reduced class sizes.

That would have been useful information, given that class sizes in the city got bigger, on average, despite the infusion. There was just one gigantic problem: I could not, for the life of me, extract the data from the spreadsheets — and even the press officer on the case, Will Havemann, couldn’t help me.

Today, I am delighted to report that the Department of Education has solved this problem for me, in the form of Tania Shinkawa, a staffer at the Office of Portfolio Development who manages class size reporting. Shinkawa just spent more than an hour with me, patiently re-jiggering the spreadsheets from this year and last year into a form that is much more understandable and analyze-able.

In the long term, this means please hold me accountable for drawing out some interesting facts about where the money went. In the short term, yay for transparency — and thank you, Tania!

First some good news: The Department of Education just released data on how students performed on Advanced Placement exams last year. Participation is up slightly from last year, and it’s up more among black and Hispanic students, contrary to a national trend. The total number of New York City students who passed is also slightly up.

Here’s the bad part: A cursory glance makes me think that while participation is up, and the number of students passing is up, the proportion of students passing is down. For instance, among black students, 3,825 took tests last year compared to 3,390 in 2007. But the passing rate dropped to 27% from 28%. That’s not necessarily a bad thing. If more students are taking the tests, it may be unfair to compare this year’s batch of test-takers to last year’s. But it’s not necessarily good, either.

Too bad only a very talented data analyst with no evening plans and no deadline would have time to write the full story. (Though the national numbers have been out for several hours, via the College Board, only the DOE can release New York City numbers. And the DOE only released the New York City numbers at 6:15. The press officer who did so compiled them into not an analyze-able Excel spreadsheet, but rather an upbeat list of talking points and a single Power Point slide.)

You can read more about national performance on AP’s here. I’ll try to have a better analysis by the morning.

UPDATE: I did call Andy Jacob at the press office to complain about the format of the release. He has not returned my call.

The rising tide of transparency seems to have infected a South Bronx schoolteacher. Since last August, the teacher has been skewering Department of Education policies on his blog, South Bronx School. He reserves the harshest words for his school administrators, whom he nicknamed “Numb Nuts” and “John Deacon,” and whom he recently accused of committing corporal punishment, in a complaint he says he sent to the Special Commissioner of Investigations.

Yesterday, for reasons that aren’t explained on his blog, the teacher revealed the name of his school, PS 154 in the Bronx, and his administrators’ real names. He did not disclose his own name. Many teachers have damning things to say about their schools, and while some criticisms are justified, others are not. We called 154′s principal today for comment and are waiting to hear back.

Not everyone has embraced the spirit of openness yet. The first comment, left just minutes after the post went up: “Whoo Hoo!!!!” Its author: Anonymous.

Yesterday, LA, Denver, and Houston. Today, Baltimore, DC, Chicago. The tour continues…

First stop, Baltimore. Maryland School Assessment test results – proficiency levels only – are available in a giant PDF report. But the state DOE saves the day with a data navigator that lets you check off groups you’re interested in and view graphs of proficiency data based on your choices. Two screenshots should give you a sense of the range of data available here.

Screenshot of the Maryland Report Card data tool.

Screenshot of county-level demographic data.

(more…)

test books by menlophoto

From the Washington Post, a glaring example of why it’s so important for educators, parents, and concerned citizens to turn a critical eye on education reporting, especially reporting about test scores:

Today, the paper ran a story about across-the-board improvements in test scores in Maryland’s Anne Arundel County, where reading test scores increased by 4 percentage points, to 86 percent proficient, and math test scores increased by 3 percentage points, to 84 percent proficient. Sounds pretty good, right? School leaders attribute the gains to the fact that they broke down test score data to focus on the particular skills individual students needed to improve and to teachers’ increased cultural sensitivity to how children learn. The Post reporter takes her sources at their words, writing, “Countywide results reflect that effort.”

But there is more to this story, and much of it appeared in the Post last week, in an article about statewide test results and the skepticism with which they have been received. First, Anne Arundel County’s scores may have increased, but they didn’t keep pace with the average gain in Maryland — statewide, reading scores increased by six percentage points, and math scores by four percentage points.

In addition, the Post reported just last weekend that Maryland test was shorter this year than in the past and that the state dropped its practice of drawing some test questions from a national exam. That important information didn’t make it into today’s article. A shorter test could have reduced fatigue that might have inhibited students from performing at their best in previous years. And because the test contained fewer items, the results should be considered less reliable than in past years. Finally, by including only questions devised by Maryland teachers in line with the state’s standards, test makers increased the likelihood that students would be able to answer more questions. Together, these changes could account for much, or even all, of the rise in test scores, but not surprisingly, state education officials are uninterested in examining whether that’s true.

Students in Anne Arundel County — home of the elementary school profiled in Tested, Linda Perlstein’s expert 2007 book about the effects of high-stakes testing on students, classrooms, and schools — might be doing better than they ever have before. But the county’s test scores don’t tell us that, and neither does the Washington Post.