Posts tagged "test security"
April 17, 2013
Most students taking this week’s state reading test are doing so under the watchful eyes of their regular classroom teacher. Teachers proctor their own students’ exams in most schools, in an arrangement that is logistically simple and keeps students calm — but also represents a soft spot in the state’s efforts to prevent cheating.
As part of its recent efforts to safeguard against fraud, New York State has reduced educators’ access to tests before they are administered and increased scrutiny on tests after they are returned to see whether answers were changed unusually often. The latter measure, known as erasure analysis, helped investigators uncover adult cheating in Atlanta, Philadelphia, and Washington, D.C., in recent years.
But even as the state has taken steps to prevent improprieties at a time when ensuring that scores accurately reflect student performance is increasingly important, it has left proctoring relatively unregulated. Erasure analysis and pre-test security can’t reveal whether students were advised to check their work on specific questions or, more egregiously, were actually given the answers while they took the tests.
“Test administration with educators proctoring their own students is one of the weak links in the testing process,” said Greg Cizek, a professor at the University of North Carolina who specializes in educational measurement and test security. (more…)
April 11, 2013
An effort to root out possible cheating that the State Education Department billed as stopgap actually included every single test that elementary and middle school students in New York took last year.
But the state is not yet saying how many red flags turned up when three million students’ answer sheets were scrutinized using a test security procedure known as “erasure analysis.”
For the last two years, under increasing pressure to show that the state’s test scores are meaningful, state education officials have asked the legislature for funding to carry out erasure analysis on students’ answer sheets. The process detects how often answers have been switched from wrong to right, a key barometer of cheating. Erasure analysis helped uncover cheating in other districts, including Atlanta, where 35 educators were recently indicted for their roles in a sweeping cheating scandal.
But in both years, legislators turned down the education officials’ requests. Last year, after legislators rejected a $1 million request to perform erasure analysis on 10 percent of tests, the officials said they would free up funds from their budget for limited testing. (more…)
January 28, 2013
City educators gave out answers to state test questions, inflated Regents exam scores, and coached students to change incorrect responses dozens of times in recent years, according to reports from a slew of investigations into test improprieties.
Responding to a Freedom of Information Law request by GothamSchools for information about complaints about test security, the Department of Education released 97 reports from investigations that concluded violations had taken place. The reports were completed between 2006 and 2012 by the Department of Education’s Office of Special Investigations and the independent Special Commissioner of Investigation.
Thirty-eight of the reports documented relatively minor violations of administrative protocol. In multiple cases, for example, investigators found that teachers had photocopied exam books when there were too few before getting official permission.
But 59 of the reports substantiated allegations about cheating, some of them serious.
One of the people found to have participated in cheating in a newly released report told GothamSchools today that an administrative trial ultimately concluded that no misconduct had taken place. The department did not immediately provide details about what happened in the cases after the investigations were over. (more…)
January 15, 2013
Starting today, school staffers can report their cheating suspicions online.
It’s one of the first concrete moves by the State Education Department’s new test security unit, created last year after a self-imposed audit of the department’s test security policies found them severely lacking.
The audit came after U.S. Education Secretary Arne Duncan urged states to scrutinize their test integrity practices following a spate of high-profile cheating scandals. The scandals threatened to undermine the real and perceived value of test scores even as New York State was attaching higher stakes to its scores.
The audit concluded that the state’s paper-based system for receiving allegations of improprieties was disorganized and outdated, creating the potential for “underreporting and underestimation of information.” Plus, the Office of State Assessment did not have anyone assigned exclusively to investigate allegations that did come in.
Now, four investigators — all former state and federal law enforcement officers — are ready to look into cheating allegations received online, according to Tina Sciocchetti, who heads SED’s Test Security and Educator Integrity office. The investigators are also working on piles of years-old cold cases absorbed from the assessment office. (more…)
November 9, 2012
After slashing its test monitoring program in the face of budget cuts this year, the Department of Education is now making plans to build it back up.
The department is looking to hire a new test security chief and triple the number of schools that it sends monitors into, according to a job ad that appeared online this week. The person who formerly occupied the position is retiring this fall, department officials said.
In 2011, the city monitored 97 elementary and middle schools during state testing days as part of a program that was meant to deter staff from violating test security guidelines. In 2012, the program shrank and monitors visited just 37 schools, most of which were already under investigation for cheating.
Chancellor Dennis Walcott blamed the reduction on budget cuts. Now for the 2013 exams, the city is putting renewed attention on test security, according to details provided in the job ad.
Key responsibilities for the new test security chief include creating a “unique” list of at least 100 schools every year that would be monitored by about 50 people, the ad says. The manager would also recruit and train monitors, then disperse them to schools during the testing period. (more…)
October 22, 2012
The Department of Education crunches state test scores in dozens of ways to measure the performance of schools, principals, teachers, and students. But it does not perform a statistical analysis that can reveal whether an elementary school’s graduates have received test scores that far outstrip their actual skills.
Researchers say it would be relatively easy for the department to calculate “swing rates” to find the proportion of students from each school whose scores rise or fall by a statistically unlikely margin when they move to another school. Such an analysis could take some of the burden off of individual educators to report suspicions of cheating.
The city used to conduct swing rate analysis prior to the Bloomberg administration, according to a former testing official, and the state is poised to launch the measure as part of an overhaul of its own approach to test security.
But department officials say the analysis would be too onerous. They also say that they never launch investigations into cheating based on data anomalies alone. Instead, they say they will dispatch investigators only when they receive formal allegations of test improprieties.
The policy means that some top-rated schools whose students’ scores plummet at far higher than the average rate never have their testing practices scrutinized.
For all of the criticism of state tests as being arbitrary and imperfect measures of student performance, they are remarkably stable. In 2011, students who saw their scores fall by more than two standard deviations from the previous year made up just 0.6 percent of the sixth grade test-taking population in English, and 0.4 percent in math. That degree of decline is highly improbable under normal circumstances and is more likely to reflect externalities than real changes in academic proficiency. (more…)
October 10, 2012
All but four of the three dozen schools that monitors visited in April as part of the city’s test security program had previously been the subject of cheating allegations.
Last spring, the Department of Education sent test monitors into 37 schools during a six-day period when students take standardized state tests, the results of which weigh heavily in how schools and teacher performances are measured.
Officials had previously billed the visits as a randomized tool to deter school staff from violating test security guidelines.
“Even schools that don’t actually get a visit … know that they could get a visit at any moment,” spokeswoman Connie Pankratz said of the program in August.
But it turns out that 33 of the 37 schools were not randomly selected at all, according to officials. Instead, the department was taking a hard look at the test administration practices of schools where it had already dispatched investigators to look into allegations of cheating. (more…)
August 22, 2012
Schools Chancellor Dennis Walcott blamed budget cuts on the reduction of monitoring visits that the education department made to schools on testing days earlier this year. He added that he hoped to restore the program to full capacity in time for next year’s standardized tests, but that he couldn’t promise it.
“I can’t give you a guarantee,” Walcott said this morning at a charity event in Brooklyn. “In a number of areas we’ve had to make some very difficult decisions around how the budget realities have an impact.”
Department officials weren’t immediately able to be more specific than the explanation Walcott offered. They did not know how much the monitoring program cost or how much was cut from it this year.
Update: A spokeswoman said that the cuts came from a $5 million reduction in the department’s testing budget, which was roughly $20 million last year. The cuts were made “in order to protect school programs from the citywide budget cuts.”
In April, monitors made 41 unannounced visits to 37 schools over a six-day testing period as part of a program that was devised to both deter schools from violating test security guidelines and check up on schools that were already suspected of misconduct and other improprieties. Many of the schools that received the surprise visits saw their test scores plummet this year.
But the total was down sharply from 2011, when the city paid 99 visits to 97 schools. The reduction comes at a time when federal and state officials are pushing districts to ramp up their test monitoring presence as part of a larger goal to ensure that the credibility of standardized tests are not compromised. (more…)
August 22, 2012
Each spring, as part of its test monitoring program, the Department of Education disperses a small team to schools on testing days to scrutinize and enforce security guidelines. Some schools are picked randomly, but others were flagged by the department because allegations were lodged by school staff and test score data showed “anomalous” results in recent years, officials said.
During this year’s six-day elementary and middle school testing period in April, education department employees paid 41 visits to 37 schools, according to records obtained by GothamSchools in a Freedom of Information Law request.
The city would not specify which schools were the subject of a targeted monitoring visit, as opposed to a random one. But an analysis of test score data for the schools that had monitors visit showed that many had large increases in 2011, a year when the citywide pass rate barely budged. When monitors visited the schools for the 2012 tests, some of them saw sharp drops on its scores — even while the citywide average increased.
Not all monitored schools saw declines this year and, in fact, some saw large gains. But of the schools that made significant gains on either English or math in 2011, more than half regressed to some degree in 2012. One school’s math proficiency rate dropped by more than 40 percentage points.
The previously undisclosed details about the monitoring program comes at a time when state and federal education officials are increasingly focused on devising policies to improve the integrity of tests in the wake of cheating scandals that have erupted in other cities. The number of schools listed in the monitoring program also provides a limited glimpse into the scope of cheating allegations that the city education department receives and is able to deal with. (more…)
June 21, 2012
A renewed effort to boost test security means the math and reading exams that city schoolchildren took in April will undergo a process designed to catch cheating.
The process, known as erasure analysis, is considered a fundamental security measure for detecting evidence of cheating, but the state has never used it in significant volume. As cheating scandals erupted in other states last year, New York education officials penciled in $1 million for a pilot program that would subject 10 percent of this year’s tests to erasure analysis. But legislators scrubbed the funding, along with another $1.1 million meant for other test security measures, when they passed this year’s $132 billion budget.
State education officials have aggressively sought public and private alternatives to fund it ever since.
Now, the funding is getting restored, and erasure analysis will be conducted on some of this years’s elementary and middle school tests, city and state officials have confirmed.
Details about which districts would participate in the pilot, what proportion of tests would be screened, how much money would be spent, and where exactly the funding would come from are still up in the air, a state spokesman said. (more…)