Antti Malmivaara
Centre for Health and Social Economics / Scientific Unit, Centre for Health and Social Economics / Scientific Unit, FI-00270 Helsinki, Finland. E-mail: antti.malmivaara@thl.fi
DOI: 10.2340/16501977-2659

Objective: To assess how items relevant for the assessment of the generalizability of findings from randomized controlled trials were recorded in systematic reviews published in leading general medical journals.
Methods: All systematic reviews and meta-analyses published in the Annals of Internal Medicine, BMJ, JAMA (The Journal of the American Medical Association) and Lancet from 1 January 2016 to 28 February 2019 were searched via PubMed. Reporting of the characteristics of randomized controlled trials in the systematic reviews was documented by the benchmarking method.
Results: A total of 115 systematic reviews were found. Of these, 71% included pharmacological interventions, 35% included other conservative treatments, 13% included surgical interventions, and 0% included rehabilitation interventions. None of the systematic reviews assessed patient selection, 35% reported disorder-specific clinical features, 25 % reported comorbid conditions, and 21% reported patients’ behavioural factors in randomized controlled trials. Functioning, environmental factors and inequity-related factors were recorded in 3%, 0% and 9%, respectively, of the systematic reviews; and adherence to interventions, crossovers, and co-interventions in 7%, 0% and 2%, respectively; follow-up percentages in 8%; and adequacy of statistical analyses in 3%.
Conclusion: In all systematic reviews the recording of characteristics of patients, adherence to interventions, follow-up, and statistical analyses in the RCTs was insufficient. The data did not allow assessment of the clinical homogeneity of the randomized controlled trials, or provide justification for meta-analysis, or allow generalizability of the findings.

In a systematic review it is important that all characteristics of randomized controlled trials are reported, so that clinicians can determine to which patients the results of a systematic review can be applied. This study assessed how comprehensively these characteristics of randomized controlled trials were recorded in the systematic reviews published in leading general medical journals. A search of the literature found a total of 115 systematic reviews. Of these, 71% were on pharmacological interventions, 35% were on other conservative treatments, 13% were on surgical interventions, and 0% were on rehabilitation interventions. None of the systematic reviews assessed how patients were selected to the study; 35% reported relevant clinical features; 25% comorbid conditions; and 21% patients’ behavioural factors. Functioning, environmental factors, inequity-related factors; how interventions were carried out; how well the patients were followed-up; and the adequacy of statistical analyses were reported in only 0–9% of the systematic reviews. In conclusion, the reporting of study characteristics in the systematic reviews does not make it possible to assess how similar the different studies had been, or to which patients these study findings could be generalized. In future, randomized controlled trials should be described better in systematic reviews. Further studies are needed on this subject.

Table SI