Earlier we discussed the issues in COVID-19 research related to observational studies that were used as the study design for the research, especially the early time frame of COVID-19 in the U.S. like the spring of 2020. We discussed some of the overarching issues that had plagued some of these designs for this research. The other study design that was thought to be better than observational studies for interventions have been clinical trials. After the push back last spring from major journals of not accepting observational studies, more people designed randomized trials to test interventions for COVID-19. Many researchers worked in these designs and these were thought to improve answers to key questions, like did hydroxy chloroquine really work against COVID-19, etc.. However, issues still existed amongst these designs.
First, there is no standard way of designing these trials and the less rigorous these designs, similar problems as seen in observational studies can appear. Also, the way the investigators decide on their exclusion and inclusion criteria can make a big difference in terms of ultimately which patients become part of the study sample. This then affects the generalizability of these studies, because the more different types of patients get excluded, the less generalizable the studies became. Also, if these studies lack a control group, then this could also lead to problems. In a May 4, 2020 JAMA article, Randomized Clinical Trials and COVID-19: Managing Expectations, a lot of these issues were discussed and nicely summarized.
Second, no matter what these problems end up being, they ultimately affect the statistical analyses. Once the inherent biases exist in the study designs then they are difficult to correct in the analyses. Regression models can attempt to control for confounding and effect modification, but controlling for biases, like selection bias, would not be possible. The selection bias that can occur in randomized trials by excluding or including certain subsets of people could greatly affect the conclusions of the analyses. For example, with all the clinical trials that have been conducted on vaccines, some major concerns were that they excluded certain groups, like pregnant woman or children, so therefore there was no knowledge of knowing how they affected these groups from the actual trials.
While the clinical trials for COVID-19 have continued for the vaccines, researchers then circled back to observational studies and sought them for generalizability with perhaps better designs to try to control for potential biases. Though in the spring of 2020, rushing to get results was imperative, researchers realized that good study design is ultimately needed to avoid inherit flaws that could not be corrected through the statistical analyses.
Written by Usha Govindarajulu
March 30, 2021
https://marlin-prod.literatumonline.com/cms/asset/aba7feaa-9d99-428f-a99b-c18da3a4d128/gr1.jpg