Preprint, non–peer-reviewed studies rose in visibility dramatically during the COVID-19 pandemic, as scientists around the global tried to decipher and describe the novel coronavirus. An analysis yesterday in JAMA shows that preprint study abstracts were more incomplete and more prone to "spin"—or deceptive persuasion—in the preprint stage compared with published study that followed.
As previous research has noted, clinicians made treatment decisions during the first 2 years of the pandemic in part based on findings in preprints. Preprints also become a tool on social media for scientists to disseminate information quickly about new findings pertaining to COVID-19.
The current work assessed how different the preliminary abstracts were from their published versions, and it found that the former often lacked completeness, with only 52% of preprint abstracts presenting a clear objective or hypothesis.
Blind analysis used
Researchers presented reviewers with blinded abstracts formatted in identical Word documents of all COVID-19 randomized control trial (RCT) preprints posted to medRxiv from January 1, 2022, to December 31, 2021, and those of matching published studies.
Of 236 RCT preprints, 182 (77.1%) had published counterparts, with a median time to publication of 134 days.
Completeness scores were assigned based on the Standards of Reporting Trials criteria, and reviewers developed scores to measure spin, which included items such as "statistically significant secondary outcomes emphasized in a negative study" and "claims extend beyond target population of the trial."
Only 30% of preprints included primary results
The greatest difference between preprints and published studies was in the inclusion of results for the primary outcome. That information was present in only 30.7% of unpublished preprint abstracts, 53.4% of published preprint abstracts, and 57.8% of published counterparts.
Two thirds (65.3%) of preprints had spin in the conclusion section, compared with 41.3% in published preprint abstracts and 34.7% in published abstracts.
Finally 20%, or one in five preprints, remained unpublished within 12 months, and these preprint abstracts were the most incomplete and had the most spin.
The data demonstrate that a gap exists between preprint abstracts eventually published and those that remain unpublished in terms of completeness and spin, potentially indicating selection for desirable qualities in the journal review process
"The data demonstrate that a gap exists between preprint abstracts eventually published and those that remain unpublished in terms of completeness and spin, potentially indicating selection for desirable qualities in the journal review process," the authors concluded.
