Editor’s note: The Salt Lake Tribune is providing free access to critical stories about the coronavirus. Sign up for our Top Stories newsletter, sent to your inbox every weekday morning. To support journalism like this, please donate or become a subscriber.
Amid questions about the accuracy of TestUtah.com’s coronavirus test results, the statewide testing operation declined to join other major Utah labs in a joint experiment to confirm one another’s quality.
Instead, the TestUtah companies agreed to a less-sophisticated “compromise” experiment to compare results with the state lab, said state epidemiologist Dr. Angela Dunn. “This is a quick and dirty way to learn whether there is a difference,” she said.
But the results of the more rudimentary accuracy check may not be made public.
“[State] attorneys have asked to review it to determine if it should be a public document or not,” said Utah Department of Health (UDOH) spokesman Tom Hudachko.
As infectious disease experts questioned whether TestUtah’s tests were sensitive enough to be reliably accurate, state data in April showed that the testing operation’s rate of positive results was far below that of the state’s other labs and health care systems.
And in Nebraska, where the same Utah tech companies have contracts to run testing, the disparity between their results and the rest of the state’s results is even bigger, according to new data there.
Meanwhile, patients in Nebraska as well as Iowa, where the companies have a third contract, are complaining of long delays in getting test results. But Nomi Health, one of the companies, said in a statement late Wednesday, “There is currently no backlog in either Nebraska or Iowa. Any previous backlog is 100% resolved in both states.”
In Utah, ARUP Laboratories, Intermountain Healthcare and the state’s public health lab agreed about two weeks ago to test samples with various concentrations of the virus, to learn how sensitive their tests and processing were — especially in samples with small amounts of the virus, said David Hillyard, a medical director at ARUP.
“Since we use many different testing platforms in the state of Utah ... what is the possibility that some of these tests are missing positives?” said Hillyard, who designed the survey and is preparing the results for publication, likely later this week.
TestUtah was invited to join the “proficiency challenge” — but the group declined, Dunn confirmed. Nomi Health has two Utah contracts, totaling at least $5 million, related to testing, and with its partners in the TestUtah effort runs seven testing sites using test kits produced by Salt Lake City biotech firm Co-Diagnostics.
So, Dunn said, the Utah Department of Health proposed this alternative quality check: TestUtah last week sent 50 samples to the state lab, and the state lab sent 40 samples to TestUtah, which processes its tests at Timpanogos Regional Hospital in Orem. The state lab is now comparing and analyzing the results, which should be complete later this week, Dunn said.
“It's something that we're still evaluating and working towards a resolution this week,” Dunn said. “We're hoping we come to the point where we can have confidence in the results that all of our laboratories are returning.”
The sample comparison is not as formal or detailed as the proficiency challenge conducted by the other labs, Dunn acknowledged. Depending on the concentration of the virus in the samples shared by UDOH and TestUtah, it may not show whether the tests are detecting the virus in patients who might be producing scant amounts of the virus, or whose swabs didn’t capture much of the virus.
The tests have a higher “limit of detection” — that is, they require more of the virus to trigger a positive result — than most other coronavirus tests approved for sale in the U.S., according to an analysis by the life sciences publication BioCentury.
By comparison, the tests used by Intermountain Healthcare labs are listed as requiring between one-fifth and one-seventeenth the viral concentration to produce a positive result, according to Food and Drug Administration documents.
Co-Diagnostics’ founder and chief science officer, Brent Satterfield, has said that the tests scored between 99.52% and 100% in evaluations conducted by the FDA and in Europe — comparable to other available test kits, he said.
The company also has said that samples from actual patients have thousands, or even hundreds of thousands of particles in their samples — plenty to trigger a positive result in its test.
That may be true for most patients, Hillyard said — but with a new virus that affects some people very differently from others, labs should anticipate a big range.
"A test that might detect the majority of positive cases could miss a clinically significant percentage of cases with low concentrations of virus,” Hillyard said.
In the most common type of test for coronavirus, machines identify the presence of the virus by amplifying its RNA again and again, Hillyard said; if there’s a lot of virus in a sample, the machine recognizes it early.
But, he said, “some of the samples that are positive come up at the very, very end of the process."
“There are patients we’ve seen that are in the lowest concentration range, that are barely picked up by the most sensitive" technology, Hillyard said.
TestUtah’s pending experiment with Utah’s public health lab won’t confirm the test’s sensitivity, Dunn said.
“All it does is say the number of times [the labs] found a positive. We’re looking for concordance, using [the Utah Public Health Lab] as the gold standard,” Dunn said. “Ideally, we would look for a one-to-one match. Anything beyond that would cause us to investigate a little further.”
The more detailed sensitivity experiment in which Utah’s major testing providers participated is the standard method for verifying the claims of a test manufacturer, Dunn said.
Nomi Health, which holds the contract for TestUtah’s testing, referred The Salt Lake Tribune’s inquiries to MountainStar Healthcare, which owns Timpanogos Regional Hospital. MountainStar defended the existing due diligence verification of test results from its lab.
“Our COVID-19 testing results have been validated on multiple occasions,” said Mike Graul, MountainStar spokesman. "We have been more than happy to participate in proficiency testing and will be happy to participate in further proficiency testing with the state as long as all other testing locations are included and the process for such a validation is agreed upon by all parties as well as an independent reviewer.
“We have communicated this to the Utah Department of Health and have yet to hear back or be invited to participate in further discussion about proficiency testing criteria."
Although TestUtah began as a philanthropic effort by Utah’s tech sector, the companies involved have received more than $60 million in no-bid contracts to operate online symptom assessments and testing sites in Utah, Iowa and Nebraska.
The contracts in Utah and Iowa don’t address the expected accuracy of the tests; however, Nebraska’s contract calls for accurate results “a majority of the time,” according to the Omaha World-Herald. Nebraska Gov. Pete Ricketts on Monday said the hospital lab that is processing TestNebraska’s tests has confirmed an accuracy rate of 95 percent, the Lincoln Journal Star reported.
But in Nebraska, as in Utah, the tech companies’ testing sites have returned a far lower rate of positive results than the rest of the state’s sites. Only about 3% of its Nebraska tests have been positive, compared with 18% statewide since the epidemic began, the Journal Star reported Monday.
Ricketts blamed the disparity on an elevated rate of positive results early in the epidemic, when limited testing supplies forced many states to restrict testing to seriously ill patients, the Journal Star reported.
But screengrabs of Nebraska’s case updates collected by the data website CovidTracking.com show the statewide positive rate since April 30, when TestNebraska was to begin testing, has been more than 20%.
It’s not clear how many of TestNebraska’s patients have been asymptomatic. Data from Utah show the majority of patients tested by TestUtah were asymptomatic — and representatives from Co-Diagnostics and Nomi have said the test sites are less likely to attract seriously ill patients and more likely to attract those with very mild symptoms.
However, Utah’s numbers separate TestUtah’s results by asymptomatic and symptomatic patients — and even when asymptomatic patients are filtered out, the rate of positive results is still less than half that of patients tested elsewhere in Utah.
Ricketts also has noted that Nebraska’s other testing efforts have focused on hotspots, which likely elevates the rate of positives in contrast to TestNebraska sites. It’s not clear whether that accounts for a seven-fold difference in the rate of positive results; one of the TestNebraska sites was in Grand Island, where an outbreak at a meat packing plant has led to an explosion in cases. However, the site there stopped administering tests on Wednesday and it disappeared from TestNebraska.com’s list of locations.
In its statement, Nomi said that site closure was “a strategic decision made in conjunction with the Nebraska Department of Health. Tents will be erected and re-positioned as needed in order to best serve the evolving needs of residents statewide.”
In stating that the backlogs in both Nebraska and Iowa had been cleared, it added: “We commend the local labs in each state for working tirelessly to serve the needs of its community.”
This week, four state senators sent a letter to Nebraska’s governor, urging him to cancel the $27 million in TestNebraska contracts because Nomi has not performed the agreed-upon 3,000 tests per day, and has taken more than the promised 48 hours to deliver results.
In Grand Island, for instance, where the outbreak has been one of the worst in the country, local health officials on Tuesday said they could no longer report daily case numbers because of lags in results.
“Nebraskans do not have the luxury to wait for Nomi leadership to learn on the job,” the senators wrote.
Delays also have dogged the operation in Iowa, where Nomi won its first major no-bid contract outside of Utah after actor Ashton Kutcher — a tech investor and friend of Qualtrics executive Ryan Smith — recommended the Utah companies to the governor. Nomi secured a $26 million contract for the TestIowa operation.
Some patients have waited as long as 17 days for results after being swabbed at TestIowa sites, The Des Moines Register reported Tuesday. In one county, health officials said nearly ten percent of TestIowa’s results were “inconclusive,” The Gazette in Cedar Rapids reported.
Iowa Gov. Kim Reynolds confirmed last week that some of the TestIowa samples were damaged and could not be processed. And as of this week, the state lab there still was not using the test processing equipment supplied by Nomi, as lab staff tried to confirm the accuracy of its test results, Iowa Public Radio reported. Instead, the state lab is using its own equipment, according to The Gazette.
