OBJECTIVE: The aim of this study was to quantify and assess the levels of evidence of research and review articles published in professional Physical and Rehabilitation Medicine journals with international circulations.

DESIGN: Quantitative analysis of articles published in Physical and Rehabilitation Medicine journals.

METHODS: Selected articles from 7 Physical and Rehabilitation Medicine journals, published over a period of 6 years, were classified according to their level of evidence. Differences in the mean number of each type of article among journals and among years were analysed.

RESULTS: A total of 5,451 articles were included in this study, of which 636 (11.7%) were randomized controlled trials. Articles published with the highest frequency were those classified as Level III (n = 2,424, 44.5%), whereas “systematic reviews” were the least frequent (n = 164, 3.0%). Heterogeneity was found only in terms of distribution over journals.

CONCLUSION: The distribution of different types of article in Physical and Rehabilitation Medicine journals is similar to that in other disciplines. No increase in articles with a high level of evidence was found in the selected journals over the period of study.

Key words: rehabilitation; research design; evidence-based practice; periodicals.

J Rehabil Med 2011; 43: 264–267

Correspondence address: Matteo Paci, Via Vittorio Bottego, 4, IT-50127 Firenze, Italy. E-mail: matteo.paci@applicazione.it

Submitted June 3, 2010; accepted October 29, 2010

INTRODUCTION

Evidence-based practice (EBP) has been defined as integrating the “best research evidence with clinical expertise and patient values” (1). Journals are generally the most important instrument in the dissemination of research results and the promotion of EBP (2).

A number of studies have assessed the types of article in many fields of medicine, such as Sport Medicine (3), Orthopaedics (4) and Physiotherapy (5). Despite the fact that several articles, in particular editorials and letters to the editor (2, 6–9), have discussed the publication of research in Physical and Rehabilitation Medicine (PRM), to our knowledge no research has yet been published analysing the literature provided by journals in this area.

The aim of this study was to quantify and assess levels of evidence of research and review articles published in 7 PRM journals with international circulations over a period of 6 complete years.

METHODS

Journals studied

A group of 7 international PRM journals were considered, as follows: American Journal of Physical Medicine and Rehabilitation (AJPMR), Archives of Physical Medicine and Rehabilitation (APMR), Clinical Rehabilitation (CR), Disability and Rehabilitation (DR), European Journal of Physical and Rehabilitation Medicine (EJPRM), International Journal of Rehabilitation Research (IJRR) and Journal of Rehabilitation Medicine (JRM).

All of these journals are official publications of scientific associations or societies, published in English with free online access, at least for abstracts.

Article selection and classification

All of the articles published in the selected PRM journals between January 2004 and December 2009 were reviewed by two independent raters (MP and GB).

All research or review articles were considered for analysis, while historical articles, studies of animals, studies of cadavers, editorials, expert opinions and proceedings of congresses were excluded. Cochrane Reviews and clinical guidelines were also excluded. Articles published in supplements or special issues were not reviewed. Letters to the editor were included if they explicitly reported the results of a study (e.g. case reports). The selection and assessment of articles were based on analysis of the abstract.

Selected research articles were grouped and graded for hierarchy of evidence according to a modification of Sackett’s system (5): randomized controlled trials (RCT) (Level I) and non-randomized controlled trials (CCT) (Level II) at the top, observational studies in the middle (Level III), and uncontrolled studies at the bottom (Level IV). Expert opinion was not included in this study.

Studies with historical controls were included within the CCT category. Non-controlled “before-after” studies with more than 10 subjects were included in the Level III group, while similar studies that included ≤ 10 subjects were classified as Level IV. Cross-sectional studies, case-control studies, cohort studies and studies without interventions were included in the Level III category.

Further article categories were validation studies and literature reviews. The former classification included studies of psychometric features of measure instruments, including reliability, diagnostic accuracy, validity and responsiveness. Literature reviews were differentiated into two categories of decreasing levels of evidence: systematic reviews and reviews.

Reviews were classified as “systematic reviews” (also including “meta-analyses”) only when this was stated in the abstract, otherwise they were classified simply as “reviews”. Surveys of professional categories or students, studies of education and economics, and historical articles, were classified as “others”.

Statistical analysis

Since there was a high variability among journals with respect to the number of issues per year, which for two journals also varied among years, the mean number of each type of article per issue was analysed, instead of the absolute number. A one-way analysis of variance (ANOVA) was used to compare differences in: (i) the mean number of each type of article per issue published over the whole period of the study (6 years) among journals; and (ii) the mean number of each type of article per issue published globally in all the journals among years. When differences were significant, a Bonferroni post-hoc analysis was used. The level of statistical significance was set at 0.05. Data analyses were performed using the SPSS statistical package 17.0 for Windows.

RESULTS

Journals

Two journals published 4 issues per year and 2 journals published issues monthly. CR published 8 issues per year until 2006 and 12 issues/year later, whereas in 2008 DR increased the number of issues per year from 24 to 26. JRM published 6 issues per year until 2006 with a variable range of issues per year subsequently, with a high number of supplements. Between 2004 and 2009 DR and CR published 9 issues and 1 double issue, respectively. During the period of the study Europa Medicophysica changed its name to EJPMR. The characteristics of the journals are summarized in Table I.

Articles

A total of 6044 articles were analysed, for a period of 6 whole years. In addition to proceedings of congresses, 593 (9.8%) articles were excluded because they did not meet the inclusion criteria. A final number of 5451 articles were included in the study.

APMR published the highest number of articles that met the inclusion criteria (n = 1821), followed by DR (n = 1,152), while the EJPRM published only 251 eligible articles.

The frequency of each type of article is summarized in Table II. A total of 636 RCTs (11.7% of the articles reviewed) were published. Articles published with the highest frequency were included in Level III (n = 2424, 44.5%), followed by validation studies (n = 847, 15.5%). Conversely, systematic reviews were published very rarely (n = 164, 3.0%).

The highest number of eligible articles were published during 2009 (n = 1,852, 34.0%), when RCTs were the second most frequent type of article (n = 125), after Level III articles (n = 507). The highest number of systematic reviews was also published in 2009 (n = 50).

However, differences in the mean number of articles among years were found only for systematic reviews (F = 4.347, p = 0.001), with an increase in 2009 compared with 2004 (p = 0.001) and 2005 (p = 0.005). The distribution over years of the different types of article is summarized in Table III.

The distribution of types of article was heterogeneous among journals (p < 0.001). Comparisons between journals showed that the highest mean number of RCTs was published in CR and APMR, followed by JRM, while Level IV articles were more frequent in the AJPMR and APMR and review articles in EJPRM. Systematic reviews were more frequent in CR, EJPRM and APMR. Over the 6 years studied, the highest number of articles classified in Level II and III and validation studies were published in APMR. The mean number of studies classified as “others” was highest in IJRR, EJPRM and APMR. The distribution of the different types of article over journals is summarized in Table IV.

DISCUSSION

Journals reviewed in this study represent a group of well-established international PRM journals, with large circulation. The relevance of each journal to the PRM field is still debated (2, 7–9). There are many journals that are potentially relevant to PRM, due to the interdisciplinary orientation of rehabilitation, and this variety makes it difficult to provide a complete and, at the same time, specific, overview of journals in the whole area of rehabilitation (2, 7). However, reviewed journals are generally considered relevant to the field of rehabilitation (2, 8).

Although an increase in the number of articles at higher levels of evidence was expected over the years, no significant differences over time were found in the selected journals.

However, differences in the distribution over journals of each type of article were found in our sample. In some cases, this may be due to differences in the editorial policy between journals. For example, CR gives the highest priority to RCTs and systematic reviews and low priority to validation studies and case reports. This journal published the lowest number of case report/case studies, largely published in other journals, such as the AJPMR and the APMR, which welcomes submission of these types of studies.

Moreover, IJRR and DR are multidisciplinary journals with a focus on integrative rehabilitation sciences rather than the professional sciences (7).

In terms of absolute number, Level III articles are the most frequent, followed by validation studies and RCTs. RCTs are traditionally the gold standard for judging the benefits of treatments, because they are more able to attribute effects to causes. Despite RCT remaining the gold standard for evidence of efficacy of treatments, Concato (10) highlights that observational studies also provide useful information for EBP. In addition, the use of RCTs may be limited by ethical, medico-legal or practical factors (11). In fact, RCTs generally require more resources than other studies, sometimes there may be problems with randomization or recruitment, and the treatment of patients with an intervention believed to be ineffective is often considered to be unethical.

The high number of validation studies confirmed the relevance of this type of study in the field of PRM, because the psychometric proprieties of measurements tools represent the basis for evidence and clinical decisions.

The strength of our study was that we reviewed a large number of articles (5,451) from a diversity of PRM journals (n = 7). However, the study also had some limitations.

The classification of trials by reading only the title and/or the abstract might have introduced some bias, because the study design is not always described in detail in these parts of an article. Moreover, the study design may sometimes be misclassified in the abstract and/or the title, especially for Level III articles (12).

Unfortunately, not all journals required structured abstracts or the description of the study design in this section, even though this is essential information for readers (13), and improvements in accuracy are needed. In addition, the reviewers were not blinded to the journal in which the article had appeared, and this is a potential source of detection bias. Single-case studies have been classified as “case reports” even if they are well-conducted research experiments and not simply case descriptions. Finally, in some cases, expert opinion and review articles were not easy to distinguish, and in unclear cases papers were classified as reviews. This choice may have introduced intake or classification biases.

Our study shows that the levels of evidence published in PRM journals are comparable to the standards in other areas of medicine. For example, Level III studies were the most commonly published study design in physiotherapy (5) and sports journals (3). The percentage of RCTs found in our analysis is comparable to that observed for other clinical specialties (3–5). Other types of studies, such as validation studies, systematic and non-systematic reviews are comparable to those published in physiotherapy journals (5), while articles classified as “others” or Level IV were less frequent in our sample.

In conclusion, the distribution of different types of articles in PRM journals is similar to that in other disciplines. There has been no increase in articles with a high level of evidence published in the selected journals over the time-period analysed.

REFERENCES