GothamSchools — daily independent reporting on NYC public schools

Graph of principals' self-reported satisfaction with ARIS over time, from an audit of by Comptroller John Liu.

The Department of Education hasn’t gotten adequate bang for its buck from more than $80 million spent on ARIS, its data warehouse, concluded an audit released by Comptroller John Liu today.

Liu offered a solid clue to the audit’s conclusions last week, when he lambasted the city’s $10-million move to formally reassign its ARIS contract to Wireless Generation, which has managed the system for years.

The audit began in March 2011, shortly after Liu held a series of town hall meetings to solicit public input about what he should investigate. The data warehouse, launched in 2008 by IBM, has attracted no shortage of critics because of its steep price tag and early glitches.

Examining usage data, principals’ responses to a satisfaction survey the city administers, and the results of a survey that it distributed to educators in June, Liu’s office concluded principals’ satisfaction with ARIS has fallen, that many schools substitute other data programs in whole or in part, and that use among school staff has leveled off since the system’s first year, although use by department officials who work with schools has risen.

What’s more, the audit concludes, the city can’t show that ARIS is leading to higher student performance — something that former chancellor Joel Klein signaled would be a result when he rolled out the system in 2008.

“This costly tech program was much-touted by the DOE to help principals and teachers track progress and thereby improve student learning, even as long-time educators questioned its cost and effectiveness,” Liu said in a statement today. “$83 million later, there is little discernible improvement in learning and many principals and teachers have given up on the system.”

DOE officials disputed the audit’s methodology, conclusions, and very premise. (more…)

Three months after announcing that he would take a taxpayer’s suggestion and audit the Department of Education’s online data system, Comptroller John Liu is asking the system’s most frequent users for feedback.

Liu announced in March that he would audit the Achievement Reporting and Innovation System, the department’s data warehouse known as ARIS, which has attracted no shortage of critics for its $81 million price tag and early glitches. In June, Liu’s office distributed a satisfaction survey to some ARIS users, including teachers.

“As part of the audit, we are evaluating whether the system meets users’ needs,” read an email containing the survey sent June 14 by Vince Liquori, director of financial audit in the comptroller’s office. A high school teacher who received the survey sent it to GothamSchools after the deadline to complete it, June 24, had passed.

The 21-question survey asks respondents for details about how they use ARIS and whether they think they system is helping them boost student achievement. The survey also includes a free-response section for respondents to list what they like and dislike about the system and identify which of its features they would change.

The survey comes as ARIS continues to contend against lower-budget competitors for teachers’ attention. (more…)

A teacher who left the system to peddle a program he created to make up for shortcomings in the education department’s data system was fined this month by the city’s ethics board.

In 2009, Jesse Olsen was a teacher at Validus Preparatory Academy in the Bronx when he realized the school wasn’t accurately recording students’ attendance patterns. Realizing that the city’s school data clearinghouse, ARIS, couldn’t help, Olsen created a data system of his own, called Impact Solutions. The data system was one of several created around the same time by educators who were frustrated with ARIS.

By September 2010, Impact Solutions had already been picked up for use in 21 city schools, which were paying between $10 and $25 per student per year for the program, and Teach for America had started using it as well, we reported at the time.

Collecting those fees violated city ethics rules, the city’s Conflict of Interest Board ruled this month. The rules prohibit public employees from owning a business that contracts with their agencies and from using their positions to boost their business interests. Olsen violated both regulations when he began selling Impact Solutions to schools throughout the city, COIB ruled.

According to COIB’s report, Olsen didn’t learn that his business ran afoul of city rules until “in or around October 2010.” (more…)

City Comptroller John Liu announced today that he is launching audits of two of the Department of Education’s most ambitious technology programs developed under former Chancellor Joel Klein.

The comptroller’s office plans to examine the Innovation Zone, or iZone — a $50 million initiative the Department of Education is touting as a strategy to improve schools during budget-conscious times. Funded through a combination of Race to the Top winnings, private donations and $10 million in tax dollars, the iZone is paying for experiments in online learning, staffing, and school time in 80 schools this year.

Liu also plans to audit ARIS — the Achievement Reporting and Innovation System — an $81 million online data warehouse that debuted in 2008 and eventually overcame some of its early glitches. ARIS began as a contract with IBM, but soon became a project of Wireless Generation, a company that was recently purchased by News Corporation. The city plans to pilot a second phase of the database, known as ARIS Local, in some schools this spring.

Both projects have their skeptics and supporters, but it was mainly the former who attended Liu’s townhall meetings, where participants suggested that the comptroller investigate whether both of these programs were accomplishing their goals. (more…)

Acknowledging that asking teachers to analyze student data has not fundamentally changed how they teach, Department of Education officials are beginning to change their approach. And other states just beginning to build their own student databases can learn from the city’s pivot, according to a report out today from Education Sector, a D.C.-based think tank.

The report recounts the brief history of New York City’s adventures with its own database known as ARIS, for Achievement Reporting and Innovation System. It also looks at schools’ gradual adoption of “data inquiry teams,” which about 65 percent of teachers were using at the end of last year.

Inquiry teams are groups of four or five teachers that select a small number of low-performing students to focus on. With information culled from ARIS, the teachers try and alter curriculum and teaching methods to improve the students’ performance. These teams are the DOE’s largest-scale reform that directly targets the instructional process. Officials hope to bring the participation rate up to 90 percent by the end of this year. (more…)

Created by teachers at the High School for Telecommunication, DataCation collects and analyzes student data, rivaling the city's own database.

When he began teaching at a Bronx high school, Jesse Olsen found the school had a large blind spot when it came to taking attendance.

If a student came to class for the first half of the school day and then skipped out, she’d go down in the official record as being present for the full day. The information holes made it impossible for teachers to know what their students’ true attendance was like, Olsen said.

A new, sophisticated database known as ARIS, for Achievement Reporting and Innovation System, might have been just the thing to solve the problem. But the system only let schools see how many days a student had missed, not how many classes they were skipping.

So Olsen took matters into his own hands, drawing on his computer science training to build an attendance system for his school, Validus Preparatory Academy.  In doing so he joined a growing number of teachers who don’t rely on the city’s data tools to track student information.

Brought into the city’s public schools in 2008 as a major initiative of Chancellor Joel Klein, ARIS cost $80 million to make. It debuted at the same time that Klein began to ask teachers to keep close track of student data and use it to adjust their instruction. (more…)

Nearly three weeks after state officials announced a dramatic reduction in test scores, city teachers and parents can find out how their students fared.

State test scores for students in grades 3-8 quietly went live today on ARIS, the city’s online school data system, and parents and teachers in district schools can log in to check the scores. (Charter school parents have to call their schools for the information.)

Charter schools have their scores and can decide individually how to share them.

We heard from an Upper East Side middle school teacher who said she eagerly awaited this moment but doesn’t know what she’ll do next. The teacher wrote:

I’m sort of the unofficial news watcher for my grade team, so I put this date on my calendar a few weeks ago when the DOE announced that they’d finally be posting the scores on August 16th. I didn’t get any email from the DOE today, so I’ll bet that a lot of people don’t know about it. I was looking forward to seeing [the scores], especially because of the new grading system and all the news surrounding it of late.

To be honest, though, I’m not sure how I’ll use this information just yet. (more…)

The BetterLesson profile for sixth-grade Roxbury Prep Charter School teacher and BetterLesson celebrity Jason Armstrong

The most popular member of a new social network is neither Lady Gaga nor Ashton Kutcher, though Kutcher is a fan of the website.

The distinction goes to Jason Armstrong, a sixth-grade teacher in Roxbury, Mass., who has more than 6,500 total views and more than 1,100 downloads on a new website for teachers called BetterLesson.

BetterLesson’s circle of about 7,000 teachers are downloading Armstrong’s math lessons, grouped into six units: whole numbers, decimals, fractions, percents, geometry, and a year-ender called extensions and review. They can also download his quizzes and tests and become his “colleague” (the equivalent of a Facebook friend).

Armstrong’s former colleague and roommate, Alex Grodd, created the site — which Kutcher recently promoted in a Tweet, a stroke of generosity devised by a BetterLesson staffer. Grodd first came up with the idea for the site when he joined Teach for America in 2004.

Assigned to teach third grade science during his summer institute training at a Houston elementary school, Grodd went online to hunt for ideas. Surely one of the other hundreds of third grade science teachers in the world had come up with a smart way to explain his assigned topic, the solar system. Why should he have to reinvent the pedagogical wheel? The last remotely relevant class he’d taken was Harvard’s notoriously science-light “Natural Disasters.”

Hours of Googling later, Grodd came up with nothing. “This was 2004, it wasn’t, like, 1994,” Grodd told me today. “The Internet had been around for a while.”

BetterLesson is not the first attempt to solve the problem of teacher isolation, but it’s already catching on more quickly than many efforts. Those 7,000 users are up from just 200 in June 2009, when the site launched to a small group, and Grodd won backing from NewSchools Venture Fund, the philanthropically financed new-idea incubator. (more…)

Nearly two thirds of principals say the Department of Education’s $81 million online data warehouse could help improve teaching and learning at their schools. 

The finding is among the results of a survey conducted by Public Advocate Betsy Gotbaum’s office, which released a statement today emphasizing that more than a third of principals did not think the system was helping their schools. In its coverage of Gotbaum’s report, the New York Times billed the system as being “supported by most principals,” And the city has said that its internal survey results show that most principals see benefits to the system.

ARIS’s solid approval rating doesn’t mean all of its kinks have been worked out. The Manhattan School for Children’s parent coordinator sent the following e-mail to parents last week:

ARIS and Classroom Assignments
It has come to my attention that the classroom teacher assignments have been posted on ARIS and I have been trying to unravel the mystery as to how these assignments came to be posted. I have also discovered that there are many mistakes. The official letters from MSC will be sent at the end of August. I am also out of town and cannot access the ID numbers that many parents are now requesting. Please double check the letters that you received from your classroom teacher. Both numbers were given out at the same time. Again, you will be notified about your official class by mail. Please do not rely on the ARIS site for this information.

The parent who forwarded me the e-mail said the incorrect information has been removed from the system but new information hasn’t yet been uploaded. (The system opened to parents in May.) (more…)

Remember how, in 2001, when he was first running for mayor, Michael Bloomberg vowed to require all public school students to wear uniforms, to bring in private companies to take over long-failing schools, and to re-evaluate tenured teachers every two years?

These are among the fun facts included in a self-evaluation Bloomberg released today, running through all the promises he made in his 2001 and 2005 campaigns, and reporting that he’s followed through with most of them (97% in 2005, the report says).

The list of education promises Bloomberg terms stick-a-fork-in-it “Done” (as opposed to those he “reconsidered”) includes many that did obviously happen, but it also includes claims that could inspire challenge. Four promises that caught my eye:

• Improve access to selective schools for students in under-served communities. (2005 campaign promise) The mayor’s report notes that the city now offers summer workshops for parents to encourage them to consider having their children take the entrance exam for selective high schools like Stuyvesant and Bronx Science. The city has also offered summer test-prep institutes for low-income students. Still, The New York Times reported last year that proportionately fewer racial minorities were taking the admissions exam, and a lower percentage were passing. There was little change when the paper reexamined the figures this year. Gifted and talented programs for primary school students, meanwhile, have also gotten less racially diverse under Bloomberg’s watch, The Times reported.
• Give teachers more control over how they teach. (2001 promise) The report explains that this “done” stems from the new availability of “a series of tools for teachers that highlight students needs and provides teachers the information to focus on helping students master their subjects.” I assume that refers to projects like ARIS, the data warehouse, and the periodic assessments known as Acuity, meant to give teachers an ongoing portrait of what students do and don’t know throughout the school year. While some teachers embrace these tools, others say the tools limit the way they teach, forcing them to focus too much time on test preparation. (more…)