The Impact on Systematic Reviews of Risk of Bias Assessment Changes From Conference Abstracts to Full Text.
This study compared Cochrane risk-of-bias ratings assigned to conference abstracts versus subsequent full-text reports across 52 RCTs and found many abstracts were rated 'unclear' and that full texts more often led to higher or lower risk judgments—notably increased odds of higher risk for…
What the AI sees
This study compared Cochrane risk-of-bias ratings assigned to conference abstracts versus subsequent full-text reports across 52 RCTs and found many abstracts were rated 'unclear' and that full texts more often led to higher or lower risk judgments—notably increased odds of higher risk for…
Research significance
For Parkinson's therapeutic research, reliance on conference abstracts can mischaracterize trial quality and certainty of evidence, potentially skewing interpretation of intervention efficacy and priorities for follow-up studies or drug development.
Source abstract
Conference abstracts are commonly included in systematic reviews of evidence. Due to limitations in word count, conference abstracts often lack data or information. This causes issues for the assessment of risk of bias (RoB). We therefore aimed to compare the RoB rating, using the Cochrane RoB tool, for abstracts and full texts. This was accomplished using previously published Cochrane reviews and comparing RoB ratings for included studies originally included as an abstract and later as full text. To accomplish this, we searched the Cochrane Database of Systematic Reviews for reviews with updates across numerous disciplines (depression, anxiety, surgical, Parkinson's disease, Alzheimer's disease, multiple sclerosis, motor neuron disease, cancer, cardiovascular disease, and musculoskeletal disease). We identified 29 reviews, with 52 randomized controlled trials included, which had an abstract and subsequent full text available. If abstracts and full texts were not assessed using the Cochrane RoB tool, we obtained the texts and performed the assessment (n = 32). To assess the likelihood of changing the domain assessment rating (low, unclear, or high) from conference abstract to full text, we performed a Bayesian categorical multinomial model for each domain (i.e., signaling question) of the Cochrane tool. At the abstract assessment stage, the most common decision was unclear. Using unclear as the reference level in the model led to increased odds of being rated high at full text, compared to abstract assessment, for domains 2 (allocation concealment: odds ratio [OR] = 3.09, 95% credible intervals (CrI) 1.01 to 9.84) and 3 (blinding: OR = 5.09, 95% CrI 1.67 to 16.20). Domain 2 also had odds of being rated low (OR: 2.93, 95% CrI: 1.13 to 7.87). This suggests an impact of changing conference abstract to full text assessments on RoB. The numerous unclear ratings observed at the abstract assessment were usually due to a lack of reporting. While the findings of this study should be interpreted within the context of small numbers, the evidence still suggests that, in some instances, such as allocation concealment and blinding, it is likely that the decision could change based on full-text assessment. This also has implications for the certainty of the evidence, which is impacted by the RoB assessments, with having abstracts only or full texts available potentially changing the overall certainty. Current RoB tools may not be suitable for assessing conference abstracts.