The state will cancel its contract with standardized testing company Questar Assessment — a strong but not wholly unexpected response after computer glitches delayed thousands of students in taking their year-end exams this spring and left administrators questioning whether the results will be valid enough to use.
The decision came after an hourlong deliberation by the Utah State Board of Education that started late Thursday in an executive session closed to the public. When the meeting reopened just after midnight Friday, members voted 12-1 to terminate the $44 million, 10-year deal it had with the provider.
“There are performance standards and expectations built into the contract and they weren’t met,” said board chairman Mark Huntsman.
The state will immediately seek bids for a short-term contract with another company to provide the exams, called RISE, for the 2019-2020 school year. It will then try to find a permanent provider.
This is the first year the state has tested with Questar. But the company has had issues with testing in other states, too.
In some, the tests were marked by cyberattacks. One school district threw out its results because the software was so unreliable. In another, all of the students had to start over when the programming shut down and didn’t save their responses.
Many of those issues came up before Utah signed its contract with the company.
The board responded with a statement to that: “Of the known issues other states had encountered with Questar prior to and after the contract was signed last year, Utah requested and received explanations and assurances that those issues had been resolved.”
The outages here delayed more than 18,000 public school students in completing their assessments this April and May. For one day, no one was able to take a science exam. On at least four others, testing was stopped entirely for some school districts. The window to complete the assessments had to be expanded into June.
Because the computers froze when students were in the middle of tests, no other student could log on to the same device. And Questar couldn’t fix the problem without rebooting the systems — something that took 24 hours. Each glitch, then, pushed back testing another day.
Some were not able to recover their work. Others were not able to finish.
It’s still unclear just what the overall impact was and what data will be valid.
“That is always a concern,” Huntsman said. “The integrity of the data is always a concern.”
Questar, which is based in Minnesota, told The Salt Lake Tribune: "While we regret this decision, Questar Assessment Inc. is going to do everything possible to ensure a smooth transition. We have committed to the Utah State Board of Education that we will maintain services on behalf of Utah students, teachers, and districts across the state until such time that an alternative vendor is selected.”
The board said it will do that “rather than risk continued interruptions." Meanwhile, it can still seek damages from Questar for the glitches. According to the contract, the state can charge the company up to $50,000 each day there was a major disruption. That would amount to about $250,000.
Huntsman called that “an option and an opportunity” to recollect some lost money from the deal. It’s unclear how much the state will have to pay for the one year it used the service. But the board isn’t scheduled to meet again until August.
“We will be conducting studies to determine what effect — if any — this year’s glitches had on our ability to use the data for statewide accountability purposes, including school grading and identification of schools in need of support and improvement,” said State Superintendent Sydnee Dickson.
Annual testing is required by federal law in grades three through eight (as well as at least once in high school). Those scores are then used to determine what schools get funding, how they’re graded statewide and which ones are struggling enough to possibly merit closure.
The exams focus on language arts, writing, science and math and are used to assess how well students are improving year to year. Results can tell teachers, most importantly, who is falling below grade level. Many educators had worried the scores would be used to judge their work.
“Teachers take these tests very seriously,” Yvonne Speckman, a teacher in Davis School District, previously told The Tribune. “We use this data. It’s important to us to guide our instruction.”
The state switched vendors last year after previously contracting with American Institutes for Research to conduct what was then called the SAGE test.
SAGE had failed to gain traction in Utah since it was implemented in 2013. More and more parents each year opted their students out of the test. That is allowed under state law, though the school board has said it undermines the accuracy of using the exams for accountability rankings.
Most board members had hoped the new RISE tests would encourage more parents to have their kids take the assessment. Roughly a million tests are completed in Utah each year (with many students taking more than one based on subjects).
