Last week, the city announced that it would release a list of teachers’ names and their effectiveness ratings to reporters who had submitted freedom of information requests. The union has sued to stop the release, and the city agreed to postpone publicizing teachers’ names until a hearing is held in court next month.

The union asserts that the ratings should not be made public in part because they are non-finalized and often error-prone internal documents. To make that case, the union is asking teachers to comb their reports for mistakes and tell the union when they find them.

The union sent teachers a sample report showing teachers how to look for mistakes, and has set up a dedicated phone line and e-mail address for concerns about the accuracy of their ratings, according to a memo union President Michael Mulgrew sent teachers last week. A union spokesman said that, as of Friday, at least 200 teachers had called the union to report errors.

Department of Education spokesman Matthew Mittenthal said that the city had seen an increase in the number of calls since the union sent out its memo. But he said that the majority of calls were prompted by misunderstandings of the reports rather than inaccuracies.

Still, Mittenthal said, the city plans to check teachers’ complaints and fix problems it finds before releasing the reports publicly.

Nearly 45 percent of the teachers who received teacher data reports this year have not yet downloaded them, a Department of Education spokesman said today. But some teachers told GothamSchools today that there has been confusion about how they could access their reports.

The city originally told teachers that the reports would be kept confidential between themselves and their principals. But city officials said today they planned to publicly release teachers’ ratings in response to Freedom of Information Law requests from several news outlets. The city teachers union is suing to try to stop the release.

Teachers can see their ratings either by downloading their report online or by getting a copy from their principals. The city e-mailed teachers log-in information to download their reports, but several teachers reported that they never received that information.

One literacy teacher, for example, said that her school never distributed information about the reports, and she learned she could access the report online only after reading GothamSchools contributor Ruben Brosbe’s account of technical difficulties in obtaining his report. Since then, she said, she’s requested a password but has not received it.

A week before the city released this year’s reports to teachers in February, department officials shared them with principals and instructed them to share them with teachers.  ”Although teachers will be able to see their own reports this year, you are expected to have one-to-one conversations with each eligible teacher about his/her report,” said a DOE memo.

But the extent to which principals discussed the reports with their teachers is unclear. In 2009, department officials estimated that 80 percent of principals spoke with their teachers about the reports.

The new firm hired to produce the Teacher Data Initiative is reaching out to the teachers’ unions that bitterly opposed the program, and the firm’s researchers say they are committed to producing tools to help teachers learn, not to rank them.

The Value-Added Research Center at the Wisconsin Center for Education Research, the firm hired last month to produce the reports, held a summer workshop on their research methods for officials from school districts around the country. Two researchers from the United Federation of Teachers also attended.

Chris Thorn, associate director of the center, said that this school year’s round of teacher assessment reports will likely look much the same as last year’s.

But the long-term goal for the three-year, $840,000 program, he said, is to refine the way data is collected so it tells the most accurate stories about what’s going on in the classroom.

“You can’t connect students to teachers without data so clean you can eat off of it,” Thorn said.

Last school year, the first year of the program, the city used a different vendor. It issued the reports to about 12,000 fourth-through-eighth grade English and math teachers. The reports grade teachers using the so-called “value-added” model, which judges teachers based on progress their students made on tests from one year to another. The reports also factor in obstacles to progress such as large class size and students’ poverty levels.

The reports have been controversial since their inception. The teachers’ union responded to news of the program by fighting for a state law that banned using student test scores as a factor when making decisions about teacher tenure.

City education officials have been adamant that the data reports are confidential and will be used only for giving teachers feedback on their performance, not for any high-stakes personnel decisions. But some teachers and other observers have publicly worried that the reports may be “a Trojan horse” ushering in other forms of high-stakes teacher evaluations.

Thorn played down those concerns. He said that the center will pay close attention to elements that can unfairly influence the way a teacher’s “value” is measured. For example, he said that the center has teachers compare their students’ attendance records to the rosters of students sitting for tests. This ensures that teachers are measured by the performance of students they actually taught, he said.

The center also uses a 38-step diagnostic measure of state exams to make sure that it is fair to compare test results from year to year. Researchers plan to spend a significant amount of time learning more about New York’s exams and data-collection systems. Thorn stressed that the center is taking a cautious approach to learning about the complex New York system.

The project in New York will be the first time the center works with such a large urban district, Thorn said, and it will be one of its first forays into value-added research of individual teachers, rather than at the school level.

“One of the issues is that there isn’t another district that compares to New York,” Thorn said. “Even Chicago is less than half the size.”

The fundamentals of the New York project, however, will maintain the same approach that the center has been developing over more than 15 years of work in other school districts, including Minneapolis, Milwaukee and Chicago. In those cities, the program has researchers working directly with schools and districts to improve the way they gauge teachers’ performance. The center has posted a two-hour long presentation by its director Rob Meyer outlining its approach to value-added evaluation based on its work in Madison and Milwaukee.

Thorn also emphasized that the process the center uses to produce the reports is one thing; how the education department chooses to use the reports is another.

Education department spokeswoman Melody Meyer stressed that the teacher data reports are informational and will not be used to formally evaluate teachers. She also said that the department is still determining what kinds of improvements they seek in the reports.

“We’re in the process of defining ‘improvement,’” Meyer said. Some changes will likely be made to the methodology of the reports, Meyer said, and others might be as simple as cosmetic improvements to the reports’ look.

“We want to make them as informative as possible for people who don’t have a statistical background,” she said.

The Department of Education created videos to explain the reports. View them here.

The Department of Education is moving to extend a program that judges teachers based on their students’ test scores — and it plans to start paying for the project with taxpayer dollars, at a projected cost of $1.5 million over the next three years. A formal request for vendor proposals released today indicates officials are also mulling an expansion of the program to more teachers.

The program, called the Teacher Data Initiative, launched quietly this school year after causing a politically explosive fight between the DOE and the teachers union the year before. The reports allow principals to track the “value” teachers add to students by looking at student test scores from one year to the next. The teachers union here has gone along with programs to judge entire schools based on test scores, but it drew the line at measuring individual teachers’ performance, arguing that so-called “value-added” models risk unfairly misjudging teachers. (Many academic researchers make this claim as well.)

After news of the effort surfaced, the union fought back by ushering a bill into state law that made it illegal for the city to use test scores when making decisions about job security. Both Mayor Bloomberg and Schools Chancellor Joel Klein decried the bill (Bloomberg called it a “special interest protection”), which the legislature passed with no public debate, and the data reports went out as planned.

About 12,000 teachers received the reports this year, all of whom who teach fourth-through-eighth-graders and either English or math (the most-tested subjects in the city). The reports grade teachers based on how much progress their students made on tests last year and give extra credit to those who made progress despite limitations such as students’ race, poverty, and class size.

I’ve spoken to some educators who think the reports are a great first step toward helping teachers think carefully about how to improve their work. The executive director of Teaching Matters, Lynette Guastaferro, called New York “a thought leader” for creating the reports. Others have been wary, including a teacher who wrote about his experience anonymously at the union activist Norm Scott’s blog, reporting that his principal is threatening to use the reports to determine which teachers remain at the school when it phases out. (Asked about the teacher’s allegations, Forte said she hadn’t heard of them but that the city has clarified procedures for teachers to follow if reports are misused.)

The Carnegie Foundation has been financing the reports so far this year, but the grant is about to run out, so the DOE issued a request today seeking a vendor that would keep up the work on the taxpayer dime. The vendor would publish the reports and manage any future expansions. You can see the full Request for Proposals below.

A technology company called the Battelle Memorial Institute has been working on the project until now, Ann Forte, a school spokeswoman, said.

Read the full RFP (which an earlier version of this post said was not available, but now is):