Coronavirus tracking metrics that regional health officials use to levy social restrictions were flawed throughout July because of delays in test processing, according to data analysis and interviews by the Post Register this week.
The flaws mean the number of active cases tends to be understated.
In late July, test results took about 10 days to come back, Eastern Idaho Public Health said. That delay has since decreased, but the test turnaround delays last month resulted in incomplete daily information, which left regional health officials to rely on old metrics to decide on coronavirus response measures.
Delays in testing and other "errors" with coronavirus data can "build up, and pretty soon, you're on a runaway train, and you're not getting to your trigger points (to decide on new social restrictions) as rapidly as you need to to avoid overwhelming your health care system," University of Idaho professor Benjamin Ridenhour told the Post Register on Wednesday. He designed the COVID-19 model that the state uses.
Eastern Idaho Public Health officials actively monitor coronavirus cases from the time they get the test results back from the state lab until the person has been asymptomatic for 10 days. By the time coronavirus cases have been publicly reported by the health district recently, most people hit that benchmark within a few days. That time window — between the case being publicly reported and the person recovering enough to be released from monitoring — narrowed throughout the month of July, the Post Register found.
During the first week of July, the average patient in eastern Idaho was monitored for nearly a full week by public health officials after testing positive for the virus. That dropped to just four days of monitoring during the last week of July. That means a person who tests positive for COVID-19 is counted as an active case for a shorter time period.
Some cases are even listed as recovered, or released from the health district's monitoring, the same day they are first reported. Of the 297 cases reported during the last week of July, more than a quarter were immediately released from monitoring after the health district reported the case.
The health district's regional coronavirus response plan focuses on active case counts sustained for three days, along with hospitalization metrics, to determine what social restrictions to pursue.
Two people who study the coronavirus in Idaho told the Post Register that with significant delays in testing and case reporting, daily case counts don't provide a real-time picture of how widespread the virus is locally.
"I don't put much stock into the daily case reports," Ridenhour said. "Over time, and as the dust settles and you look back at the data a bit, then I start looking back on it and trusting it."
Testing delays complicate responses
James Corbett, community health administrator at the health district, said active case rates account for two concerns with tracking COVID-19 cases: Making sure mitigation strategies are based on current cases and accounting for the delays in symptom onset.
"If we receive a report that someone was sick 20 days ago, that's really not demonstrating or reliably stating what's happening right now," he said. "So, we're trying to be as closely aligned as we can get."
But some who track the coronavirus say that active cases can only catch blips of cases for a few days when testing is delayed and limited. Ridenhour said those who use active case rates must account for testing delays in their models.
For example, health officials could zoom out and look to weekly averages of new cases, as many national coronavirus trackers are doing.
Because someone with coronavirus typically doesn't experience symptoms for a few days, there's a natural delay in when people get tested. “That compounds with the delay in testing," Ridenhour said.
"Using active case counts is a decent strategy, you just have to go about it carefully, and you have to use those trigger points very carefully," he said. "And those trigger points need to incorporate cognizance of those delays and the sort of changing nature of the data."
Dr. Ronald Solbrig, who serves on Idaho State University’s coronavirus task force, said concerns about data reliability make already difficult decisions about coronavirus response measures even more onerous.
On one hand, hospitalization data is “clearly an indicator when things are really, really bad” because the medical system could become overwhelmed, Solbrig said.
Ridenhour said he and some others look to hospitalization data to create projections for how widespread coronavirus will be in four weeks. The data is somewhat less current, he said, but it's more complete.
"It's a bigger delay, but the tradeoff is that you can maybe ... have a better forecast using those data than using the case rates because of the consistency of those data. ... You're eliminating the sort of vagaries that go along with the diagnostic testing, but you do give up a bit of timeliness in there," Ridenhour said.
More immediate data, like case reports, can help to better capture how severe the pandemic is. Both Ridenhour and Solbrig said reported cases themselves are only a reliable metric if testing is truly random. In most places, testing is dependent on who seeks tests and how severe their symptoms are.
"If testing is bad,” Solbrig said, “then we have no confidence that we know what’s happening in our community.”
When asked who gets tested in eastern Idaho, health district spokeswoman Mimi Taylor declined to say. "Testing varies from provider to provider as to who they decide to test," she said.
Corbett acknowledged that health officials need to balance accuracy and speed in their metrics. He said when the health district receives a positive test result, health officials work quickly to confirm the person lives in eastern Idaho before reporting the case or hospitalization.
He said the district could make exceptions to that practice as cases increase "to make sure we accurately put information up" about daily cases. Asked about how reliable daily active case counts are as a metric when testing is delayed, Corbett said, "It's still representative. I still think it's valuable."
'Skewing our active case' count
The daily active case rate fluctuates rapidly with eastern Idaho's small population. Only a handful of cases can make the difference in designation for some smaller counties.
Clark County's population of less than 1,000 is so small that the health district chose to handle restrictions on a case-by-case basis. Larger counties like Jefferson have, in recent weeks, constantly teetered between being above or below the 10 active cases per 10,000 residents threshold that would put them in moderate risk, where the board's plan calls for mask mandates and large event bans.
Over the past two weekends, Jefferson County has fluctuated above and below the threshold for moderate risk level. Its active case rate on each Friday was slightly below the 10 active cases per 10,000 resident metric.Each Saturday, it jumped above that metric, staying there Sunday. Each Monday, it dropped just below the threshold.
Jefferson County's representative on the board, Shayne Young, raised concerns about whether active case rates were reliable at the past two health district meetings.
"If those test results take that long, that is skewing our active case number," he said at Thursday's meeting. "… I think we need to take that into consideration as well, because, how accurate is that metric when our test results — many of them are being delayed for so long."
Eastern Idaho had mostly escaped major outbreaks of the coronavirus until near the end of June. As the number of cases and tests grew, the region encountered the same backlog for test results that had affected Idaho when the virus first arrived in the spring.
Express Lab, which provides private coronavirus tests and has worked with Mountain View Hospital on drive-through testing, saw a backlog of nearly 14 days for test results around the Fourth of July. Lab manager Lori Leask said turnaround time has since come down to around three days.
"I will say, that can change any day when you see spikes in demand. It overloads the system, and it pushes them all back for a few more days," Leask said.
In late July, Corbett said test results were taking 10 days to come back from national and state reference labs, after which the health district confirms a patient lives in the district before publicly reporting the case. Test results are now taking four days to come back, he said.
"I'm confident that testing is going to improve," Corbett said. "That doesn't mean we won't look to other metrics that can help us understand what the current situation is."