Sir,

A recent paper by Lundgren-Nilsson & Tennant (1) and accompanying commentary (2) reviewed the development of Rasch analysis in rehabilitation. The authors of the two papers acknowledge the advances that have been made in terms of analysis. However, they did not highlight an important issue with regards to the use of Rasch measures once developed. We believe this is a fundamental issue that needs to be addressed to ensure the potential of Rasch is fulfilled.

There has been a recognised increase in Rasch analysis in rehabilitation (3, 4), with Lundgren-Nilsson & Tennant (1) reporting over 50 papers applying Rasch to the Functional Independence Measure. Given the importance of measurement in clinical trials and clinical practice, it would seem logical that a technique that enabled ‘better’ measures and improved interpretation (5) would have been welcomed by both the research and clinical communities. Rasch analysis has the advantage of interval level measurement which has potential to allow parametric analysis in clinical trials, improve synthesis in systematic reviews and map individual and service outcomes over time in a more robust manner. However, the extent to which Rasch developed and refined measures have been used in research or practice has not been investigated. It has been supposed that Rasch analysis is rarely applied to practical implementation of outcome measures (3). We undertook a review to explore how often Rasch developed scales have been used in research.

Review on use of Rasch developed measures

The review methodology as detailed by Tesio et al. (3) was repeated, searching for Rasch as free text. Published articles that specified they applied Rasch developed or refined measures to a study were included. The entire electronic databases Medline, EMBASE, Cinahl, PEDro, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews, PsychINFO and Web of Science searched from the earliest publication to June 2011.

Initially the number of studies using Rasch analysis were documented to compare with Tesio et al. (3) to document the increase in Rasch analysis over the last 4 years. The title and/or abstract of articles identified from the initial Medline search were then examined to identify those papers which had applied a Rasch developed or derived scale to clinical trials, case studies or clinical practice.

From the initial search of the electronic databases, it was clear that the use of Rasch analysis has continued to increase across all of the databases (Table I).

From the initial Medline search which retrieved 1,458 articles, only 25 studies were found that applied Rasch developed or refined measures to clinical trials, case studies or clinical practice. Four of these were randomized controlled trials, with the remainder split between correlation or regression studies, case studies, cohort studies or retrospective analysis. The earliest study found applying Rasch measures was in 2002 (6). The main field which has implemented the application of Rasch analysis in studies is in vision and rehabilitation, with 10 out of the 25 studies on this area (4 by Lamoureux and colleagues (7–10). Most of the excluded studies had used Rasch in the development or refining of measures but not the application and there were a number of reviews and methodological reports.

It is recognized that there are obvious limitations in the search procedure and it may not be comprehensive. The review method may have some elements of bias and have underestimated the number of studies which applied Rasch. There may be an unknown number of papers where Rasch-derived scales are used, but the word Rasch is not mentioned in a way identified by the search. However, 25 studies equates to approximately 1.8% of the number of studies published on Rasch analysis actually applying it so it is hypothesized that the use of Rasch scales in clinical outcome research does not correspond to the use of Rasch in outcome measure development.

These results are consistent with those found be Tesio et al. (3) and Belvedere & de Morton (4). Bond (11) recognized this discrepancy, questioning why Rasch measurement is well-used in the development and refinement of measures but not in measuring patients. There is a gap between psychometricians and applied researchers, with researchers themselves slow to apply Rasch-derived measures. Even researchers who are co-authors on Rasch measure development/interpretation papers have then done further studies and not used the Rasch versions but the original non-Rasch scales.

There has been no research specifically investigating factors affecting uptake, but several reasons have been postulated. A letter by Granger (12) stated 10 obstacles to Rasch analysis. A major recognized barrier is the need for specialist knowledge in terms of mathematical understanding, and the need for specialist software (5). Few researchers or clinicians are trained in its use and interpretation. This makes it essential that Rasch scales are accessible in a format that eases interpretation. It should be encouraged that researchers publish conversion tables, ideally through a website. Franchignoni et al. (13) recognizes that often papers do not include conversion tables showing the relationship between ordinal raw scores and the corresponding Rasch interval measures and mentions that Abilhand website with an online Rasch analysis routine enables the conversion.

The challenge to Rasch researchers to demonstrate the usefulness of Rasch and facilitate application of Rasch measures in research and practice was given by Tesio (14) in 2008, yet in 2011 little has changed. The danger of Rasch being restricted to being used by a small community of outcome measure researchers (3) remains a reality. The measure development process should not end once psychometric properties been demonstrated, there is a need to look at implementation issues too. As identified by Krumholz (15), outcome measure researchers must build partnerships with those who will use the measures developed. The suggestion by Heinemann & Deutsch (2) that journal editors request the use of contemporary psychometric measures or explicitly state this limitation may be one way forward.

The use of Rasch analysis in rehabilitation continues to grow but it is hypothesized that this increase is not being matched by an increase in the application of Rasch-derived measures in applied research. We agree that Rasch has the potential to add much to rehabilitation and the process continues to develop. However, when discussing future issues we feel that it is amiss not to discuss issues with the use of Rasch measures in applied research. It has to be questioned what the use of the increase in Rasch analysis in rehabilitation research is, if then these measures are published in a non-useable format and are never used in research. There is an urgent need for investigation of the reasons for non-uptake. There is a plea for researchers to minimize the barriers already identified to uptake by ensuring Rasch scales are published in an accessible format and publish conversion tables, ideally through a website. It is also important to define when such tables could be used and the premises for their use. Outcome researchers need to change behaviour themselves, using Rasch-derived scales when possible and appropriate, to increase the visibility and advantages of using interval levels scales for investigating change scores over time so they are more familiar to other researchers and clinicians. There is a need for closer working between researchers and end users of measures, with measure development continuing beyond publication of sufficient psychometric properties, but continuing into uptake and application.

References

Submitted October 21, 2011; accepted November 30, 2011

Louise Connell1 and Christopher Sutton2

From the 1Clinical Practice Research Unit, and 2School of Health, University of Central Lancashire, PR1 2HE Preston, United Kingdom

Response to the Letter to the Editor by Connell and Sutton

While we welcome the continuing emphasis given by Connell & Sutton to the need for Rasch-based scales for rehabilitation outcomes, we would argue that the premise of their work, and that of Tesio and colleagues (1) beforehand, has limitations. Trying to find out if Rasch-based measures are used, they perform a systematic search in MEDLINE using the text word ‘Rasch’.

Why, we ask, should any Rasch developed scale which is subsequently used in a study, mention Rasch? With scales developed by classical approaches, would we expect to find them in use if, for example, we searched for ‘Confirmatory Factor Analysis’ (CFA)? Once the scale is developed, it can be widely used with just the reference to the original development paper, and no mention of the methodology used. This may not even appear in the title of the development paper, and so no amount of searching for ‘Rasch’ (or CFA) would uncover that such a scale, developed with a particular methodology, had been used. An example of this would be a paper on work productivity in ankylosing spondylitis (AS) (2). That paper used the AS Quality of life Questionnaire (ASQOL), a needs-based disease specific quality of life scale for AS developed with the benefit of Rasch analysis (3). The text word ‘Rasch’ does not appear at all in the paper by Maksymowych et al. (2). To do the search properly for those scales used in outcome studies, it would require that all scales developed (or modified) with the benefit of Rasch analysis should be first identified, and then each individually subjected to a MEDLINE search. For example, the ASQOL development paper mentioned above has currently 150 citations (Google Scholar), which gives an indication of how much it may have been used.

Where ‘Rasch’ should be mentioned in a paper using an existing scale is where the interval scale transformation is used for analysis. For many, this is the prime purpose of applying the Rasch model in health outcomes. Indeed it is this application that should be encouraged as the misuse of ordinal scales is still endemic in the literature, despite numerous exhortations to the contrary (e.g. 4). Thus the production of transformation tables which allow those in both clinical and outcome settings to make use of the transformation, where needed, should become a routine part of all development papers, given the sample size is sufficient for that purpose. In this way we can all benefit from the special properties conferred by the Rasch measurement model, whether the scale is used in a routine clinical setting to identify risk, or to monitor progress; whether it is used in an outcome study, or in a randomised controlled trial. When this transformation is used for the analysis of outcomes of any kind, then we would expect the word ‘Rasch’ to appear in any search.

References

Åsa Lundgren Nilsson1 and Alan Tennant2

From the 1Institute of Neuroscience and Physiology, Department of Clinical Neuroscience and Rehabilitation, University of Gothenburg, Gothenburg, Sweden and 2Department of Rehabilitation Medicine, Faculty of Medicine and Health, The University of Leeds, Leeds, UK