Early Trial Results Often Report Exaggerated Treatment Effects
On average, the effect size in the early trials was 2.67 times larger than the overall pooled effect size.
HealthDay News — Trials to evaluate drugs or devices used to treat chronic medical conditions that are published early in the chain of evidence often show an exaggerated treatment effect compared with subsequent trials, according to research published online in the Mayo Clinical Proceedings.
Fares Alahdab, MD, from the Mayo Clinic in Rochester, Minn., and colleagues assessed meta-analyses (MAs), published between Jan. 1, 2007, and June 23, 2015, in the 10 general medical journals with highest impact factor to identify randomized controlled trials (RCTs) evaluating a drug or device in patients with chronic medical conditions.
The researchers identified 70 MAs that had included a total of 930 RCTs with an average follow-up of 24 months. An exaggerated early effect (defined as the proportion of MAs with largest effect or heterogeneity in the first 2 trials) was seen in 37% of MAs.
On average, the effect size in the early trials was 2.67 times larger than the overall pooled effect size (ratio of relative effects, 2.67). There were no significant associations between the presence of exaggerated effect and trial size; number of events; length of follow-up; intervention duration; number of study sites; inpatient versus outpatient setting; funding source; stopping a trial early; adequacy of random sequence generation, allocation concealment, or blinding; loss to follow-up; or the test for publication bias.
"Considering the increasing morbidity and mortality of chronic medical conditions, decision makers should act on early evidence with caution," the authors write.
Alahdab F, Farah W, Almasri J, et al. Treatment effect in earlier trials of patients with chronic medical conditions: a meta-epidemiologic study [published online February 6, 2018]. Mayo Clin Proc. doi: 10.1016/j.mayocp.2017.10.020