Separate Realities

Being trained as a scientist, I have always believed that there is such a thing as truth and a single reality. But people in the USA seem to be living in completely different realities, as a recent poll shows. It asked Americans if they found the current number of deaths from COVID-19 in the US acceptable. The results vary dramatically by political orientation:

The majority of Republicans (57%) think that the current number of deaths is acceptably, but only 1 out of 10 Democrats thinks this way.

A key to understanding these differences is that "About two-thirds of Republicans (64%) think the number of US fatalities from coronavirus is actually lower than what is being reported".  In sharp contrast, only 12% of Democrats think that the number of COVID-19 deaths is lower. But in both parties, the percentage of people who believe that COVID-19 deaths are overstated closely mimics the percentage of people who deem the current fatalities acceptable. This raises the question:

Are the reported COVID-19 deaths numbers accurate, too low, or too high?

This is a reasonable question. There is no doubt that not all COVID-19 deaths are reported accurately.  If someone dies of COVID-19 without ever being in a hospital and without a positive COVID-19 test, the death will often not be reported as a COVID-19 death, leading to under-reporting. But at other times, COVID-19 may be listed on the death certificate even if the death clearly was not caused by COVID-19, causing over-reporting of death. Many people who believe that actual deaths are lower than reported numbers will have stories of someone without COVID-19 symptoms who died in a motorcycle accident or similar. Some believe that hospitals cheat by listing COVID-19 as the cause of death so they get higher reimbursements.

There is no doubt that both under- and over-reporting of COVID-19 deaths happens. Theoretically, both could happen at a similar rate so over- and under-reporting cancel each other out, but this seems unlikely. So how can we get an idea how many more deaths are really caused by the coronavirus epidemic in the US?

The most straightforward way is to compare the number of people who died since the start of the epidemic to the number of people who died during the same time period in previous years. If COVID-19 has caused a large number of additional death, that should show up in the number of reported death. This approach, named "excess death analysis" is standard when trying to estimate the impact of epidemics; for example, it is generally used to estimate how many people die of influenza. 

Before we get started, let's have a quick look at the number of COVID-19 deaths reported in the US as of today (August 27, 2020, 8:15 pm):

•  The CDC web site reports 178,998 COVID-19 deaths.
• Worldometer.info reports 184,764 COVID-19 deaths.
• Johns Hopkins University reports 180,527  COVID-19 deaths.

Exact numbers differ a bit depending on when and how data are collected, but we can say the reported number is close to 180,000.

Next, we can look at the data from the National Center for Health Statistics, were all states send reports of deaths to. The web page provides a download link for "National and State Estimates of Excess Deaths", so you can download a file in .csv format that you can import into Excel or OpenOffice.

The file contains state-by-state data for weekly reported and expected deaths since 2017, and totals for the entire US. There are three data sets in the table, and we'll look at each in turn.

The first data set we'll examine is the "unweighted, all causes" set. These are the numbers for the death reports that the CDC had received by the time the file was generated. For recent weeks, these numbers will be incomplete, since not all states and counties have reported their numbers yet; typically, it takes about 8 weeks until about 99% of the death certificates have been submitted. Therefore, this data set will give an underestimate of the number of excess deaths.

The latest data included are for the week that ended 8/15/2020. The "unweighted" (incomplete) data set reports an excess of 203,840 deaths for 2020. This number is significantly higher than the number of reported COVID-19 deaths. This is a clear indication that the reported COVID-19 deaths understate the actual number of deaths cause by the epidemic.

Since many recent death certificates are missing from the "unweighted" data set, we need to look at the other data sets. The next one we can look at is the "weighted, all causes" data set. For this set, the CDC has estimated how complete the submission of deaths certificates was for each week and jurisdiction, and adjusted the totals to account for missing reports. The "weighted" (predicted) data set reports an excess of 245,305 deaths for 2020. 

The third data set in the file ("weighted, all causes excluding COVID-10") calculates excess deaths after subtracting reported COVID-19 deaths. This gives the number of excess deaths that are not classified as COVID-19 on the death certificate. This results in 82,049 excess deaths, in addition to 163,256 COVID-19 deaths. In other words, for every 2 reported COVID-19 deaths, there is another additional death that does not list COVID-19 as the cause of death. 

One way of interpreting these results is that only about two thirds of COVID-19 linked deaths are reported. In other words, the actual death toll from the corona virus epidemic in the US is about 50% higher than reported in the official death counts.

There are a few different ways to calculate these numbers, but they all end up with pretty similar results: actual excess deaths are about 40-50% higher than reported deaths.

The final exercise is to provide a "best estimate" of the death toll as of today. The CDC spreadsheet only contains data until 8/15/2020; since then, and additional 10,536 deaths have been reported. Using the under-reporting factor of 50.3% described above means we expect more than 5,000 additional excess deaths, for a total of 261,136 excess deaths in the US linked to the COVID-19 epidemic.

The analysis is based on data submitted by the states to the CDC, a government organization that has been under the control of the current administration for the last 3 1/2 years. The data are publicly available, and anyone can download them and do their own analysis. But even the incomplete "unweighed" data, which does not include many deaths from the most recent weeks, clearly show:

Actual deaths linked to COVID-19 are significantly higher than reported numbers

Evaluating the US Response to COVID-19

 How well did the US response to the COVID-19 pandemic work?  This post examines both COVID-19 tests and deaths, and compares them to data from other countries - mostly European countries, since their culture and political system are similar to the US.

Let's start with a look at COVID-19 deaths. We'll use 7-day rolling averages to smooth out day-to-day variations, and use death per million population so that we can compare countries of different sizes. 

During the initial phase, death rates at the peak  were lower in the US than in Italy and France, two of the hardest-hit European countries. But US death rates were about 3-fold higher than in Germany.

But if we extend the graph until August, the picture changes:

In the three European countries, death rates dropped dramatically. In contrast, the drop in the US was less pronounced. We can see the differences better if we use a log scale for the y axis:

The graph below  shows a comparison of how much COVID-19 deaths were reduced from the maximum to August 1:

 Whereas France, Germany, and Italy reduced daily COVID-19 deaths by between 55-fold and 135-fold,  the US only achieved a 2.4-fold drop. In other words, the US efforts to contain COVID-19 were between 22-fold and 55-fold less effective than the efforts in France, Germany, and Italy.

But what about COVID-19 testing?

There have been some repeated claims that the US has done "the most COVID-19 testing in the world". However, these tests are both false and very misleading.

In terms of absolute numbers, the country that has performed the most COVID-19 tests is China, with 90.4 million tests compared to 60 million to 64 million tests in the US. But absolute numbers mean little, since countries differ vastly in population size and COVID-19 cases.

When population size is taken into account, the US drops down to the 19th spot on the Worldometers ranking:

But while population size is one factor to consider, the more important factor is the number of actual COVID-19 cases in a country: a country with more infections also needs to test more. If we look at the number of tests relative to the number of confirmed COVID-19 cases, the US does poorly:

It is illuminating to examine how the ratio of tests to cases developed over time in different countries:

Note that the y-scale is logarithmic. For the last few weeks, the tests/cases ratio for the US has been about 12. For the European countries shown, the ratio has been above 100 - typically 10 to 20 times higher than in for the US.

If we look at the ratios in April in the graph above, and know how severe the COVID-19 pandemic was in the different countries, a clear relation becomes visible: the countries with the lowest amount of testing had the worst epidemics. Of the European countries shown, Greece had the lowest number of COVID-19 deaths in April (relative to population size), followed by Germany. Greece also had the highest ratio of tests per case, again followed by Germany. The countries with the worst epidemics in April were Italy, the UK, and the United States - all countries with substantially less COVID-19 testing.

The same pattern was present again and again during the COVID-19 pandemic: less testing meant more COVID-19 deaths, and often out-of-control growth. Currently, two countries with very rapidly growing COVID-19 case numbers illustrate this: Brazil and South Africa.

With adequate COVID-19 testing, people can see rapidly raising infections, and adjust their behavior, often even before mandatory public health measures are initiated. But without sufficient testing, the warning flags go up too late. The real scope of the epidemic remains hidden from view; infections keep multiplying rapidly; and the end results is a large death toll, as was seen in Italy, Spain, France, New York City, and many other places.

The bottom line

The efforts to contain COVID-19 in the US have been substantially less effective than in other countries, including several of the European countries that were hit hardest by the COVID-19 pandemic. As a result, the current number of new COVID-19 infections and deaths is an order of magnitude higher in the US.

Even though the US has performed the second-highest number of COVID-19 tests in the world, the high level of active infections means that the current COVID-19 testing capacities are insufficient. This is evident in the poor performance of the US when comparing the test-to-case ratio with other countries. Average wait times for test results of more than 4 days, and wait times of 10 days or more for 10% of tested individuals, also underscore the shortcomings of COVID-19 testing in the US, and severely limit the potential effectiveness of contact tracing and related measures to contain the spread of COVID-19.