COVID-19 challenges of publishing statistically relevant studies
COVID-19 is a tricky period in which to attempt to publish new studies. Criticism came from reviewers and those reading the articles whether or not one showed negative results or positive results from the study. In fact, due to the ongoing situation with the pandemic, much criticism has been given to any potential positive results since other articles had been published with results that were not accurate. It has been a very difficult time period then for perhaps well designed studies even to get published due to the bias in the media against new results and the skepticism around them. Studies were then torn apart. These were cases where studies suffered statistically. Many of them were pilot in nature. Often these studies were not powered for sample size and were using available cases. Also, many of the studies suffered in selection bias. Generally there has been a bias in the literature toward studies that published statistically significant results, but in this situation, studies that published negative findings, say of a potential treatment, had less issues or potentially less criticism, even though they showed statistically non-significant results. Any studies showing positive results for a potential treatment of COVID-19 met with a lot of criticism.
Another way around this was to publish a study that was a systematic review of other studies and then try to send a message about the information generated from the average amongst these. Systematic review and meta-analyses are not exactly the same and sometimes the terms were confused, even by policy makers trying to use these types of articles to make decisions. A systematic review collects information available on a specific topic of articles available in the literature and attempts to make summary amongst this information while a meta-analyses goes a step further with statistically analyzing the information on a specific topic and summarizing this statistically. Gleaning information from a systematic review can be tricky, especially if there are outliers amongst the results of the articles, and using this for public opinion could bias decisions, like the ones made about potential COVID-19 transmission outdoors and the risk. Most studies showed relatively low risk but one particular study showed a larger risk though the persons in that study were not always outdoors. Therefore, whether using information from articles that show positive results, negative results, or summary of results like systematic reviews or meta-analyses, one always has to be very careful with the interpretation of these results.
https://media.nature.com/lw800/magazine-assets/d41586-020-03564-y/d41586-020-03564-y_18696342.png
