Since the COVID-19 pandemic was declared in March 2020, a recurring topic of debate has been whether official COVID-19 death statistics in the U.S. accurately capture the fatalities associated with SARS-CoV-2, the virus that causes COVID-19.

Some politicians and a few public health practitioners have argued that COVID-19 deaths are overcounted. For instance, a January 2023 opinion piece in The Washington Post claims that COVID-19 death tallies include not only those who died from COVID-19 but those who died from other causes but happened to have COVID-19.

Most scientists, however, have suggested that COVID-19 death tallies represent underestimates because they fail to capture COVID-19 deaths that were misclassified to other causes of death.

We are part of a team of researchers at Boston University, University of Minnesota, University of California San Francisco and other institutions who have been tracking COVID-19 deaths since the beginning of the pandemic. A major goal for our team has been to assess whether the undercounting of COVID-19 deaths has occurred, and if so in which parts of the country.

Examining excess deaths

One way to examine the issue is to look at what population health researchers call excess mortality. It’s a measure which, in this case, compares the number of deaths that occurred during the pandemic to the number of deaths that would have been expected based on pre-pandemic trends.

Excess mortality captures deaths that arose from COVID-19 directly or through indirect pathways such as patients avoiding hospitals during COVID-19 surges. While determining a cause of death can be a complex process, recording whether or not someone died is more straightforward. For this reason, calculations of excess deaths are viewed as the least biased estimate of the pandemic’s death toll.

As a general rule of thumb – with some important caveats that we explain below – if there are more COVID-19 deaths than excess deaths, COVID-19 deaths were likely overestimated. If there are more excess deaths than COVID-19 deaths, COVID-19 deaths were likely underestimated.

In a newly released study that has not yet been peer-reviewed, our team found that during the first two years of the pandemic – from March 2020 to February 2022 – there were between 996,869 and 1,278,540 excess deaths in the U.S. Among these, 866,187 were recognized as COVID-19 on death certificates. This means that there were between 130,682 and 412,353 more excess deaths than COVID-19 deaths. The gap between excess deaths and COVID-19 deaths was large in both the first and second years of the pandemic. This suggests that COVID-19 deaths were undercounted even after the pandemic’s chaotic early months.

Major studies have also concluded that excess deaths exceeded COVID-19 deaths at the national level during the first two years of the pandemic. And preliminary analyses by our team have found that the gap between excess deaths and COVID-19 deaths has persisted into the third year of the pandemic. This suggests that COVID-19 deaths are still being undercounted.

Making sense of the discrepancy

Explaining the discrepancy between excess deaths and reported COVID-19 deaths is a more challenging task. But several threads of evidence support the idea that the difference largely reflects uncounted COVID-19 deaths.

In a recent study, we found that excess deaths peaked immediately before spikes in reported COVID-19 deaths. This was the case even for excess deaths associated with causes like Alzheimer’s disease that are unlikely to rapidly change due to patients avoiding hospitals or other changes in behavior during the pandemic.

This finding aligns with the observation that COVID-19 deaths may go unrecognized – and be misclassified to other causes of death – at the beginning of COVID-19 surges. At this time, COVID-19 testing may be less frequent in the community, among medical providers and among death investigators. If excess deaths were not caused by COVID-19, they would instead either remain relatively constant during COVID-19 surges or they would peak afterwards when hospitals were overcrowded and deaths may have resulted from health care interruptions.

Excess deaths related to external causes of death such as drug overdose also increased during the pandemic. However, a preliminary study found that the scale of this increase was small relative to the overall increase in excess deaths. So deaths from external factors alone cannot explain the gap between excess and COVID-19 deaths.

This evidence is worth considering in light of the prominent opinion piece in the Washington Post mentioned earlier, which suggests that the U.S.‘s tally of COVID-19 deaths is a substantial overcount. The author argues that in some hospitals, widespread COVID-19 testing has led patients with COVID-19 who died of other causes to still have COVID-19 included as a cause on their death certificate. There is a fundamental misunderstanding, however, in generalizing these hospital deaths to the entire country.

One reason this overgeneralization is flawed is because hospital deaths are distinct from out-of-hospital deaths. In out-of-hospital settings, COVID-19 testing is often lacking and death investigators have less training and less information about the deceased. In fact, our research suggests that COVID-19 deaths are largely undercounted in out-of-hospital settings.

Investigative reporting among coroners in rural areas has also revealed significant variability in out-of-hospital cause of death assignment. Some coroners have even gone on record to state that they do not include COVID-19 on death records if it contradicts their own political beliefs or if families wish for it to be omitted.

The other problem with the overgeneralization is geographic. Our preliminary study demonstrates that excess deaths exceeded COVID-19 deaths in the vast majority of counties across the U.S. In particular, counties in the South, the Rocky Mountain states and rural areas had many more excess deaths than COVID-19 deaths. This suggests that COVID-19 deaths were likely undercounted in these areas.

The idea that COVID-19 deaths are sometimes overreported is, to a very limited extent, supported by our analyses. A select number of large and medium-sized metro areas in New England and the mid-Atlantic states have had more COVID-19 deaths than excess deaths. But most of the country has not followed the patterns of this small group of counties.

While it is possible that some deaths assigned to COVID-19 in New England and the mid-Atlantic states were not actually caused by COVID-19, other explanations are also possible. First, COVID-19 mitigation efforts could have prevented deaths in these areas via pathways unrelated to COVID-19, reducing excess deaths. For example, some people living in wealthy, urban counties had the privilege to work from home and avoid household crowding, which may have reduced their risk of dying from flu. Flu is typically responsible for as many as 50,000 deaths each year.

In fact, the 2020-2021 flu season was minimal, likely because of social distancing. Another possible explanation is that later in the first two years of the pandemic, there may have also been fewer deaths than expected in some areas because some of the least healthy people in the area had already died of COVID-19. These alternative explanations imply that, even in those New England and mid-Atlantic counties where more COVID-19 deaths were recorded than estimated excess deaths, many COVID-19 deaths may still have occurred even as other kinds of deaths decreased.

Why it matters

Ultimately, figuring out how many people have died as a result of the COVID-19 pandemic is a major scientific undertaking that has significant political importance. Knowing how many people died and where these deaths occurred has widespread implications for informing how current pandemic response resources are allocated and for preparing for future public health emergencies.

As a result, in our view, it is critical that the scientific community carefully reviews the rigor of the science behind the counting of COVID-19 deaths. Given the intense politicization of the pandemic, claims of overcounting or undercounting need to be made cautiously.

Finally, research by our team and investigative reporting conducted in partnership with our team has found that the undercounting of COVID-19 deaths is significantly more common in Black, Hispanic and Native American communities as well as low-income areas. Claims that COVID-19 deaths have been overcounted undermine efforts to reconcile the undercounts in these communities and to ensure resources are being allocated to those most affected. For example, if a person does not have COVID-19 assigned as a cause on their death certificate, their family is ineligible for pandemic social programs such as the FEMA funeral assistance program.

To understand where the U.S. public health system has succeeded and fallen short during the pandemic, a full accounting of deaths caused by COVID-19 is needed. More than that, families, friends and loved ones of those who have died so far also deserve to know the true toll that COVID-19 has taken.

Andrew Stokes is Assistant Professor of Global Health, Boston University; Dielle Lundberg is Research Assistant in the Department of Global Health, Boston University; Elizabeth Wrigley-Field is Assistant Professor of Sociology, University of Minnesota, and Yea-Hung Chen is Research Data Specialist in Epidemiology and Biostatistics, University of California, San Francisco.

This article is republished from The Conversation under a Creative Commons license.