Doing research in COVID-19 presents many types of problems for having statistical analyses free from biases. In studies that were organized as observational studies to compare treatments or differences on some outcome between COVID-19 patients and either some type of control or less sick COVID-19 patients, many inherit biases came to play. In general, these studies tend to suffer from baseline differences between comparison and control groups which lead to confounding and other biases, which can also be present in these study designs. I will discuss some of these issues focusing on observational studies.
Observational studies around COVID-19 were largely designed early on, especially in the spring of 2019 when answers were needed early on. However, the studies were often fraught with selection bias due to selection of cases in a given time period or particular geographic area and potential sample size problems for doing high level analyses. Also, depending upon the way the data was collected, the data may have had measurement error issues in study values collected on the patients, even with BMI and other laboratory measures. Reliability of real data collected were also questionable as being able to assess the validity after the fact was an issue. Also, parsing out an appropriate control group for comparison could again be an issue due to selection bias
In addition, at some point in the spring, major journals like NEJM and JAMA set rules that they were no longer accepting observational COVID-19 studies, due to some published articles on hydroxychloroquine that were later found to have erroneous results and had to be retracted. They then put rules to only accept randomized trial studies of COVID-19. This then caused the potential for observational studies to have less likely of a chance to be published in high impact journals. The downsides of observational studies have already been mentioned, but the positive angle has been the opportunity for more generalizability than a randomized trial. Therefore, should observational studies really have been excluded on the basis of some published studies that were dishonest? Fortunately, observational studies on COVID-19 continue to be published but there is more thorough review now of these than before, which is important as these studies can continue to provide useful and timely information in the treatment and prevention of COVID-19.
Written by Usha Govindarajulu
March 17, 2021
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSWaZFru56pSCJKDsZP7KLuJDpmC58jbYBbRg&usqp=CAU