Accepted Sep 25, 2018; Epub ahead of print Oct 9, 2018

J Rehabil Med 2018; 50: 00–00

The purpose of all activities in medicine (education, research, administration, management, clinical work etc.) is to advance the health and wellbeing of patients and populations in ordinary healthcare settings (1). The real-effectiveness medicine (REM) framework recommends instituting actions on 4 levels to advance effectiveness in routine clinical work (Fig. 1). The basis of all effectiveness is the competence of the staff; without this, any benefit for the patient is unimaginable. Since the beginning of 1990s the Royal Society for Physicians and Surgeons of Canada has used a competence framework, CanMeds, which has been successfully implemented also for healthcare professionals other than physicians (2). In addition to medical expertise, the CanMeds framework assesses competence related to communication, co-operation, management, health advocacy, scientific skills and professionalism, including relevant ethical issues.

The second level of the REM framework is evidence-based medicine as a means to convey the current scientific evidence into everyday treatment decisions. Randomized controlled trials (RCTs), systematic reviews and meta-analyses, and clinical guidelines constitute the core of evidence-based medicine. The third level of the REM is to attain valid data on patients, interventions and outcomes in the real-world setting, and the fourth level is to use this data in benchmarking with peers treating similar patients (Fig. 1).

Medicine and rehabilitation is very much about intervention: how we are able to help patients beyond what would be their state if we were not there. Knowledge of the effectiveness of our interventions and of the most effective interventions is therefore important.

There are 2 ways to obtain scientific information about effectiveness: experiment or observation. The question is how to decide which of these is the most appropriate option for a particular study. If the study question is on effectiveness of a single intervention, then an RCT, i.e. experimental design, is usually the best option. To be valid, the RCT should include a concealed randomization protocol in order to reach maximal comparability between the intervention groups. Consequently, the between-group differences in the outcomes can be attributed to the differences in interventions in the treatment arms. The most important validity items in RCTs can be grouped into 3 categories: baseline comparability (aim of randomization), difference in the interventions between the treatment arms (the cause for outcome), and validity of the outcome (valid outcome measures, high follow-up rate) (Fig. 2).

However, even when one is studying a single intervention, there may be ethical reasons, or issues related to the study question or the feasibility of the study, which may render observational effectiveness study, the benchmarking controlled trial (BCT) an alternative option, or even the only option (3) (Table I). If the aim is to assess the effectiveness of a clinical pathway, BCT is, in most cases, the design of choice, since cluster randomized trials are rarely feasible. If the study question is about comparing the performance of different healthcare providers treating and rehabilitating a particular group of patients, the only suitable study design, by definition, is the BCT.

There are recommendations on how to plan, conduct, report and assess observational effectiveness studies, the BCTs (4, 5). The term benchmarking is derived from the necessity to make between-peer comparisons, which makes it possible to learn from the best practices. The validity criteria on BCTs are rather similar to those of RCTs, except that there are 3 further main validity requirements for BCTs due to lack of randomization: the selection of patients must be reported, as well as the staff competence and relevant healthcare system features (4, 5).

As always in science, one starts with a study question, and then proceeds to find the methods that provide a valid answer. Thus, whether to blind the patients and healthcare professionals treating and rehabilitating them is dependent on the study question. If the question is on the biological (or physical) effectiveness of an intervention (intervention effect per se), one has to use a double-blinded study design. However, if the study question is to assess the effectiveness of an intervention in the non-blinded circumstances of everyday healthcare, one should neither blind the patient nor the therapist. In most cases rehabilitation interventions fall into this category, where blinding is contra-indicated.

The idea of using double-blinding only when blinding is needed to answer to the study question has important implications. As double-blinded RCTs assess only the intervention effect per se, excluding the placebo effect, it produces abstract effectiveness estimates that do not occur in ordinary health care. The true effectiveness of, for example, a drug for depression is composed of the biological effect shown by a double-blind RCT plus the placebo effect of the drug to the patient (6). Consequently, the effectiveness estimates from double-blind RCTs are not valid for calculating number needed to treat figures, or cost-effectiveness (or cost-utility) estimates for real-world circumstances.

Appropriate description of patient, index and control interventions and outcome (PICO) characteristics that have actualized in the original study are necessary for the clinician to be able to judge whether the results are generalizable. In addition to PICO characteristics,  also those of co-morbid conditions, and patients behavioural, environmental and equity-related items are often important for making generalizations.

For a treatment to be considered effective the results must be statistically significant. An additional question is the size of the treatment effect. Often a clinically important treatment effect is determined based on the mean difference in perceived outcome between the treatment and control groups. However, it is more appropriate to assess the differences in percentages of those patients in the treatment and control arms who exceed the effect size considered clinically important. It is suggested here that the statistical significance of between-group proportions of patients receiving small, moderate and large treatment effects will be assessed, and the difference in proportions would be used and offered to clinicians for informing patients. Similarly, it is suggested that, in systematic reviews, instead of assessing the magnitude of effectiveness from standardized mean differences in the individual RCTs, the between-group differences in proportions of patients receiving, small, moderate and large treatment effects would be assessed.

Healthcare systems may have a strong modifying effect on patient outcomes. For example, building up a well-functioning clinical pathway for fragile elderly people with a hip fracture improves the patients’ probability of returning home (7). The design of choice for assessing the impact of healthcare systems on patient outcomes is the BCT, as cluster RCTs are usually unfeasible (8).

Systematic reviews aim to provide a summary of all relevant effectiveness studies for a particular clinical question. Comprehensive description of the characteristics of patients, intended interventions (and how they actually happened during the trial) and outcomes are the cornerstones of a systematic review, and make it possible for the clinicians to assess whether the results are generalizable to their own practice. The first  of the 3 most important validity items in synthesizing evidence from original studies is, that only those trials which are clinically homogeneous are used for evidence syntheses, and only these clinically homogeneous trials will be subjected to a meta-analysis. The second essential validity criterion is that the interpretations are based on the most valid original studies. Thus, primary meta-analyses should be based on clinically homogeneous studies having best internal validity (9). The third validity item relates to the whether there may be a publication bias.

Overviews of systematic reviews aim to summarize evidence from different systematic reviews covering a particular area of interest. As the study objects are the systematic reviews, not the original publications, the overviews should aim to provide a thorough description of the patients, interventions, and outcomes in each systematic review; to assess the internal validity of the reviews, and to present the results and authors’ conclusions. The overviews should not engage in statistical assessment of the evidence itself.

The recently established Cochrane Rehabilitation provides summaries of those Cochrane systematic reviews that have assessed the effectiveness of rehabilitation, and helps rehabilitation professionals in their efforts to apply the best current evidence in their clinical praxis (10).

The aim of medicine and rehabilitation is to help the patient. The starting point is to find out what the diagnosis is, and what are the factors contributing to the patient’s condition. The first clinical intervention category is to convey this information to the patients, advise and support the patients in their own efforts to alleviate symptoms and increase probability of recovery (Fig. 3). The second intervention category is to assess the functioning of the patient and take actions to increase the patient’s abilities and participation (11). The third intervention category is to consider effective biological (or physical) ways of treating the patient. The way to increase the placebo effect of the biological treatments can be used to further increase the effectiveness of treatment and rehabilitation.

The author has no conflicts of interest to declare.