COVID-19: Risk Assessment, Model Uncertainty, Unknown Prevalence, Unclear Mortality, Alternative Paths, and State Anomalies

The COVID-19 battle is trending favorably. There is reason to be grateful for lower death rates, less hospitalization, less ICU use, and less ventilator need than was predicted.  Those that directly engaged at great personal risk, from first responders to health care workers, deserve our thanks.  Moving forward vigilance and rigorous critical analysis is needed to understand the risk COVID-19 posed.  There is much we do not know to include:

How prevalent is it in the population?  
What is the mortality rate?
Did uncertain models overly inform and influence narrative and government policy?
What role, and to what degree, did social distancing influence cases and deaths? 

Were there alternative strategies that could have been as effective?

It is important that the nation undertake an investigation to fully understand the risks and realities of COVID-19 and assess whether or not the actions that local, state, and federal governments were in fact necessary and effective.  Shutting down the U.S. economy was an extreme action.  It is an unsustainable action that cannot become part of the standard play book for each new virus threat.  Understanding what really happened in during the pandemic is essential to ensuring future preparedness and viable strategies.

My concern is that harmful policies were taken by federal, state, and local governments that threaten our economic well being and played loose with Constitutional rights based on uncertain predictive models.  Did the governments actions stop nearly 2.2. million deaths?  Or, were the original projections simply wrong?  We do not know. We need to find out.  Our future well being may depend on it.

As the COVID-19 pandemic settles down and election season draws near some politicians are going to be patting themselves on the back for having saved thousands if not millions of lives.   If the total number of deaths by fall is approximately 60,000 President Trump can say his Administration's  actions saved between 1.44 and 2.14 million lives.  This assumes the models were correct in their predictions.  Governors can extrapolate their great success from the same numbers.  A handful of governors will have to explain to their citizens (particularly NY, NJ, MI, MA, LA, IL) why their states were negatively impacted disproportionately.  

The stories of success will rely on accepting the original model estimates of high side death estimates of between 1.5 and 2.2 million deaths if no mitigating actions were taken.  Modelling is uncertain estimating of the future – not fact.   Drastic policy action was taken that relied on such modeling to assess risk.  A great deal more skepticism of public health modelling is warranted and each of the models relied upon require rigorous scrutiny.

Understanding Risk

New York Governor Andrew Cuomo said of the lock down restrictions he ordered on economic and social engagement in response to the COVID-19 pandemic, “"if everything we do saves just one life, I'll be happy."  Did he really believe that?  Unlikely.  He is an experienced politician who knows that leadership and governing is about making hard choices and trade-offs.   

His statement begs the question how many deaths are acceptable?  As President Trump frequently says in various ways the death of, “one  person is too many.”  But he too knows that is sentimentality.  Leaders of states and countries make relative judgments to achieve the best outcome.  Sixty-thousand potential deaths are a serious risk, but leaders must ask, “relative to what?” The cure or mitigation cannot be more harmful than the disease.

There is risk all around us in daily life.   As individuals and as a society we face risk every day.   We make choices personally or through government or businesses or organizations to mitigate that risk.   As individuals we walk out our doors every day and drive our cars to various locations for various reasons.  Transport is an essential aspect of modern economic and social engagement.   Offset against that activity is risk. 

We know that nearly 37,000 people are killed and 3 million seriously injured in cars every year.  Personal mitigation actions of safe driving technique and wearing a seat belt reduce the probability we will fall victim to this risk.  Government establishes regulations and auto manufacturers install technology that can and has reduced the number of deaths and injuries.  But there is only so much mitigation that we can do.  The risk will never be reduced to zero.   So, we make a relative choice that the benefit of driving in the car exceeds the risk of being killed or injured in each trip that we take.

We endure influenza season because we cannot eliminate it - only mitigate its impact. Despite our technological and scientific prowess only one human virus has been eliminated - small pox in 1979.  Last influenza season approximately 60,000 people died.   The risk of influenza is known with some seasonal variation.  We mitigate that risk with the development of a vaccine thought to be most effective against the dominant strain of the season (sometimes correctly, sometimes not).  

According to the CDC 64% of children and 68% of those over 65 and only 40% of those 18-64 were vaccinated last year.  Sixty-thousand died despite this massive vaccination.  COVID-19 is projected to kill about the same number without a vaccine.  Does that mean COVID-19 is lower risk than seasonal influenza? Maybe social distancing substituted for and was as effective as influenza vaccination?  We do not know but need to find out.

Assessing risk is extremely difficult when there is little information upon which to assess the risk.  In the COVID-19 pandemic we lacked the knowledge to assess true risk.  Therefore, governmental leaders relied upon epidemiological modelling to assess risk and acted upon it in ways that caused other serious harms.

 

Modelling Narrative and Public Policy

The accuracy of modeling upon which government officials relied is uncertain due to a lack of data, flawed assumptions, unspecific definitions, and varied algorithm choices.  John Allen Paulos, Professor of Mathematics at Temple University said in an interview with WKMG Orlando, “No model is perfect, but most models are somewhat useful… But we can’t confuse the model with reality.”

Early modeling of the potential progression of COVID-19 indicated a grave risk to public health.  This modeling informed government decisions to take aggressive action to slow the spread of the virus.   The United States of America shut down much of the world’s largest economy and states restricted civil liberties in reaction to this modeling.

Early models estimated as many as 2.2 million U.S. deaths from COVID-19 if no action were taken to mitigate its course. The lowest projections were a range of between 100,000 and 240,000 with aggressive mitigation.  The model currently favored by U.S. authorities projects a death toll of approximately 60,308 by August 4^th within a range from 34,000 to 140,000.   Was this decrease the result of government imposed social distancing and millions were saved? Or were the model projections simply wrong and the lower death rate is because COVID-19 is not as deadly as projected?  We do not know but need to find out.

The U.S. risk assessment of the SARS-CoV-2 virus that causes Coronavirus Disease 2019 (COVID-19) was initially based on slow, sparse, and sometimes politically suppressed reporting from the People’s Republic of China.  The implementation in late December, 2019 of a quarantine in three cities in Hubei Province, including Wuhan where the outbreak began, focused world attention on the virus.

 

The U.S. officially became aware of the virus on January 3, 2020 when the director of the Centers for Disease Control and Prevention had a conversation with Chinese colleagues.   From that point forward there was a struggle to understand the dimensions of the problem, its overall risk, and an appropriate response.   This began to change dramatically with the formal release of epidemiological modelling using very limited data of Chinese origin.

On March 13^th the CDC issued a report based on its modeling of the virus progression in China, “Estimating Risk for Death from 2019 Novel Coronavirus Disease, China, January–February 2020.”  The CDC estimated the risk of death in Wuhan reached “12% in the epicenter of the epidemic and ≈1% in other, more mildly affected areas.”  The CDC concluded, “Because the risk for death from COVID-19 is probably associated with a breakdown of the healthcare system in the absence of pharmaceutical interventions (i.e., vaccination and antiviral drugs), enhanced public health interventions (including social distancing measures, quarantine, enhanced infection control in healthcare settings, and movement restrictions), as well as enhanced hygienic measures in the general population and an increase in healthcare system capacity, should be implemented to rapidly contain the epidemic.”

Within two days of the release of the CDC report President Trump declared a national emergency and allocated $50 billion to aid state and local governments; the CDC recommended no gatherings of more than 50 people; President Trump discouraged congregating in groups of more than 10; and New York City closed the largest school system in the country.  The March 13^th CDC modelling report began driving the narrative and prescribing response to slow the virus and mitigate impact.

On March 16, 2020 the Imperial College of London released a report based on epidemiological modelling, “Impact of non-pharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcare demand.”   The New York Times said, “Imperial is treated as a sort of gold standard, its mathematical models feeding directly into government policies.”

The report linked COVID-19 to the worst pandemic in recent history stating, “the public health threat it represents is the most serious seen in a respiratory virus since the 1918 H1N1 influenza pandemic.”  The authors’ models estimated for the UK (Great Britain) and U.S., “in an unmitigated epidemic, we would predict approximately 510,000 deaths in GB and 2.2 million in the US.”  

This model played a major role in governmental response and public perception in the UK and U.S. 

The London model began driving the media narrative and U.S. policy.  On March 29^th, President Trump said he heard the numbers from the London model for the first time earlier in the day, and said, “I think we got our most accurate study today, or certainly most comprehensive… think of the number, 2.2 million if we did nothing.” 

The University of Oxford’s Evolutionary Ecology of Infectious Disease group released a study shortly after the Imperial report titled, “Fundamental principles of epidemic spread highlight the immediate need for large-scale serological surveys to assess the stage of the SARS-CoV-2 epidemic.”  The Oxford report contradicted the Imperial estimates, stating the UK and Italy current wave in “the absence of interventions should have an approximate duration of 2-3 months.”  The researchers contend the infection rate is much higher than many estimates and the first infections may have occurred a month before the first reported deaths in the UK and Italy providing significant levels of herd immunity.  The lead author, Professor Sunetra Gupta said, “I am surprised that there has been such unqualified acceptance of the Imperial model.”

Both the Imperial and Oxford studies were criticized for different reasons.  But the Imperial report’s frightening number of 2.2 million deaths in the U.S. captured media attention and propagated.  Anyone that took exception to the threat the model indicated came under criticism.  Politicians who expressed doubt or did not take fast and severe action were accused of ignoring scientific advisors. Some in the media calling those who questioned models “Covid Deniers.”   The Oxford study was treated in this manner but may well prove in the end to have been more informative.

On March 17^th, John Ioannidis, Professor of Medicine, Health Research and Policy, Biomedical Data Science at Stanford University published, “A fiasco in the making? As the coronavirus pandemic takes hold, we are making decisions without reliable data.”  He further stated, “The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable… We don’t know if we are failing to capture infections by a factor of three or 300.”

The Santa Fe Institute recently published a series of articles surrounding the lack of clarity in models and projections across the world. In one report they state, “There is no shortage of data on the unfolding coronavirus epidemic….The problem with this massive quantity of data is its quality— data sets from different countries are not really compatible with each other, are often internally inconsistent, and in some cases could be politically manipulated.”

They point to specific issues in the data stating, “Test density and methodology vary greatly; not all virus carriers also show symptoms; not all infected people are identified; hospitals do not necessarily report releases to the authorities; those who have recovered at home will not always report; and the death toll is unclear, because it is difficult to distinguish between people who die from corona versus with corona.”

On March 31^st, Dr. Deborah Birx, the White House Coronavirus Response Coordinator, provided her “Goals of Community Mitigation” presentation at the daily White House Coronavirus Task Force briefing.  Her first slide was the now familiar “Flattening the Curve” graphic.   She began thanking “domestic modelers from Harvard, from Columbia, from Northeastern, from Imperial [College of London] who helped us tremendously. It was their models that created the ability to see what these mitigations could do.”

She explained that there would be “between 1.5 million and 2.2 million people in the United States succumbing to this virus without mitigation.”  The Imperial College of London high estimate evident at 2.2 million. The other institutions apparently estimated no less than 1.5 million on the high end of their modeling.

Dr. Birx described the lower mitigated range as between 100,000 and 200,000 if actions such as social distancing and other actions were taken.  She then extolled the work of the University of Washington Institute for Health Metrics and Evaluation (IMHE) and its modeling.   IMHE used data from Italy, Spain, South Korea and China to “give insight into the hospital needs, the ventilator needs, and really the number of people who potentially could succumb to this illness.”  She was impressed by the model as it was updated daily with on the ground data from the U.S. and could create projections by state.

The IMHE model became the preferred model of the federal government supplanting the Imperial College of London report.  But almost immediately the IMHE estimates of projected deaths and hospital requirements began to decrease in the model with tremendous reductions in some states.  “Forecasted fatalities have fallen in North Carolina (-80%), Pennsylvania (-75%), California (-70%), Texas (-65%) and Washington (-55%),” reported the Wall Street Journal.

On April 16^th the IMHE model projected 68,841 deaths by August 4^th.   Two days later on April 18^th it had reduced that estimate to 60,308.   On April 16^th the model said Florida would peak on May 2 and by August 4^th would suffer 4748 deaths.  By April 18^th IMHE reported that Florida had its peak on April 14^th and the death projection for August 4^th had dropped to 1363.

The point is that models are filled with uncertainty.  Skepticism is warranted.   Robust debate about the quality of models, how the media portrays them, and how government policy is informed by they is sorely needed.

There is a divide beginning in social media that produces memes that try to make social distancing actions an either or choice of saving lives or lowering social distancing strategies.  This is an illogical "false dilemma" argument.  Two extreme points from a spectrum of potential choices give the impression that the options are mutually exclusive.  They are not.   

Alternative Paths

Was there an alternative path to the national economic shutdown?    Could we instead have taken a very aggressive approach to protecting vulnerable populations in institutions such as nursing homes, assisted living facilities, prisons, high population density communities, poorer communities with known underlying comorbidities, and encouraged those over 70 with underlying comorbidities to isolate themselves as much as possible?  

Could we have kept schools open and encouraged families to avoid contact with grandparents, elders, and those with comorbidities?   All others would be encouraged to follow hygiene recommendations. Mass entertainment and sports events would be cancelled, and travel would be discouraged.   

The virus would be allowed to spread at a slower pace under these restrictions relying on building immunity in the younger and healthy population while aggressively protecting the vulnerable and treating the sick.

Sweden has taken the alternative path similar to that described in the preceding few paragraphs.  The IMHE model projects 5,890 deaths in Sweden by August 4^th.   That is about .0006% of the 10.2 million population of Sweden.   This compares unfavorably with the U.S. at 60,308 deaths and .0002% of the population projected to die with COVID-19 by August 4^th.   But in the long run will the Swedes gain herd immunity much faster than other countries and avoid any second or seasonal relapse from the virus?

Contrasted against Sweden is New Zealand where the government has put a cordon around the small island of about 5 million.  Only 11 deaths have been recorded in New Zealand and only about 1400 confirmed or probable cases.  New Zealand will have to sustain this isolation, with all of its economic repercussions, until well after it has vaccinated its population and it opens its borders.

Accurate Prevalence Data

To understand the true risk that COVID-19 represents requires accurate data.  The first data point needed is the number of people that have or have had Covid-19 or have antibodies indicating their immune systems have confronted the SARS-CoV-2 virus.  Scientific sampling for infection and antibodies may be the best way to determine virus prevalence.

Researchers at Stanford University released a sampling study this week titled, "COVID-19 Antibody Seroprevalence in Santa Clara County, California."  That study tested 3,300 for antibodies and revealed 2.4 to 4.16 percent of the participants had antibodies.  This indicates the prevalence in Santa Clara County is between 41,000 - 81,000 people - not the 1,000 the County was reporting on April 1. If as this study estimates the true prevalence is "50-85-fold more than the number of confirmed cases" it has tremendous implications for determining the true risk of COVID-19 morbidity.

In a recent Boston Globe OpEd Harvard researchers said,  “In Massachusetts, a random sample of 5,000 residents out of the total population of nearly 7 million would be large enough to determine the prevalence of COVID-19 infection within a margin of error of 1.5 percentage points.” 

A robust scientific national sampling for COVID-19 disease and antibody reaction to the SARS-CoV-2 virus is needed.

Defining COVID-19 Deaths

Virus prevalence is only one part of the equation in determining COVID-19 risk.   The second part is defining accurately what represents a COVID-19 death.  We do not know that number now because there is still not adequate refinement in describing cases of those who die and their associated demographics.

Dr. Deborah Birx said at the April 7 White House Corona Task Force briefing, “So, I think, in this country, we've taken a very liberal approach to mortality… - the intent is, right now, that … if someone dies with COVID-19, we are counting that as a COVID-19 death.”   

We are reporting deaths of people WITH COVID-19 not those that died FROM COVID-19.  Some states have started estimating and adding to their counts people who died before tracking began and were suspected of dying from COVID-19.  A few examples to illustrate the distortion:

If a person testing positive for COVID-19 is involved in an auto accident and killed they will be recorded as a COVID-19 death. 

If a person testing positive for COVID-19 was in the hospital for an organ transplant and died in the operating room they would be recorded as a COVID-19 death.  

If a 95 year old man testing positive for Covid-19 died while in hospice with metastasized cancer in his lungs, pancreas, and liver he would be recorded as a COVID-19 death.

There is a tremendous difference between dying “with” and dying “from” or even “dying with complications that may have hastened near term inevitable death from underlying comorbidities.”  Much more refinement is needed to determine the true mortality of COVID-19.

Anomalies of Impact

There are major geographic COVID-19 anomalies to be investigated.   Six states that combined make up less than 20% of the national population represent 69% of the deaths thus far. New York alone represents 39% of the nationwide deaths, New Jersey 12%, Michigan 7%, Massachusetts 5%, Louisiana 4%, and Illinois 4%.  

What is happening in these states that makes them so different from other states?  Are there variables at play such as comorbidities, population density, weather, preparedness, policy choices, that make these states strikingly different from others?  Understanding those variables that contribute to this extreme anomaly may prove essential to understanding risk far better and help to develop better strategies going forward.