Isaäc Bos, MSc1, Jan B. M. Kuks, MD, PhD1 and Klaske Wynia, PhD1,2

From the 1Department of Neurology and 2Department of Health Sciences, Community and Occupational Health, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands

OBJECTIVES: To develop a measure that is based on the International Classification of Functioning, Disability and Health (ICF) and reflects the prevalence and severity of disabilities related to neuromuscular disorders, and to evaluate the psychometric properties of this measure.

METHODS: A preliminary questionnaire was developed, based on the categories of the ICF Core Set for Neuromuscular Diseases. Next a cross-sectional postal survey was carried out among 702 patients (70% response rate) diagnosed with a neuromuscular disease. Finally, psychometric properties were examined.

RESULTS: The preliminary Neuromuscular Disease Impact Profile (NMDIP) consisted of 45 items. Factor analysis showed that the NMDIP comprised domains representing 3 ICF-components: 5 factors in the Body Functions component, 2 factors in the Activities component, and 1 factor in the Participation component. Scales showed moderate to good internal consistency (α = 0.63–0.92) and mean inter-item correlation coefficients (0.38–0.77). Convergent and discriminant validity analysis indicated that the NMDIP measures the impact of neuromuscular disease on physical, mental, and social functioning. The NMDIP discriminates between groups who differ in extent of limitations.

CONCLUSION: The NMDIP is an ICF-based measure that reflects neuromuscular disease-related disabilities. It consists of 36 items divided over 8 scales with satisfactory psychometric properties and 4 single items.

Key words: neuromuscular disease; health measure; International Classification of Functioning Disability and Health; psychometric properties.

J Rehabil Med 2015; 47: 00–00

Correspondence address: Isaäc Bos, University Medical Center Groningen (UMCG), University of Groningen, Department of Neurology, PO Box 30001, NL-9700 RB Groningen, The Netherlands. E-mail: i.bos@umcg.nl

Accepted Oct 28, 2014; Epub ahead of print Feb 13, 2015

INTRODUCTION

Neuromuscular diseases (NMD) may be acquired or hereditary. Causes are dysfunction of the anterior horn cell or sensory ganglion cell (neuronopathy), peripheral nerve (neuropathy), neuromuscular junction (myasthenia), or muscle (myopathy) (1). Common symptoms of neuromuscular diseases include muscle weakness, impairment in muscle endurance, involuntary muscle activity (stiffness, myotonia, cramp, and fasciculation), sensory loss, autonomic dysfunction and impairment in control of voluntary movements. Sensations of pain and fatigue are common consequences of these muscle and nerve disturbances (2, 3). These symptoms have a profound impact on daily activities and participation in life situations (4, 5).

In clinical practice and research there is a need for reliable and validated assessment tools as well as outcome measures that cover the broad range of health problems in neuromuscular patients (6).

Over the last 2 decades many health measurement instruments have been developed for use both in clinical practice and in research. As a result, there are generic health-related quality of life (HRQoL) instruments with a broad scope, for example, the Medical Outcome study Short Form Questionnaire (SF-36) (7). An example of a disease-specific HRQoL instrument with a broad scope is the Amyotrophic Lateral Sclerosis Assessment Questionnaire (ALSAQ-40) (8). In addition, there are generic and domain-specific measures to assess limitations in daily living, for example the Groningen Activity Restriction Scale (GARS) (9), or to assess participation in life situations, for example, the Impact on Participation and Autonomy Questionnaire (IPAQ) (10).

The International Classification of Functioning, Disability, and Health (ICF) is a classification developed by the World Health Organization (WHO) and aims at providing a unified and standardized language for describing and classifying health domains and health-related states, and hence providing a common framework for the development of health outcome measures (11, 12).

The ICF comprises 4 key components. The first component, Body Functions and Structures, refers to functions of body systems, and to anatomic parts. The second component, Activities, refers to “task or action execution by the individual”. The third component, Participation, refers to “involvement in life situations”. The Environmental Factors that interact with these 3 components, are described in the fourth component of the ICF (11).

In the model of functioning that underlies the ICF classification system, the components body functions and structures, activities and participation are summarized under the concepts “functioning” and “disability”. These are associated both with health status and with personal and environmental factors. Functioning is an umbrella term encompassing all body functions, activities and participation. Similarly, disability is an umbrella term for impairments in body functions, limitations in activities and restrictions in participation (11). ICF Core Sets have been the first approach to providing ICF-based instruments for clinical practice and research (13). An example of such ICF-based measures is the Multiple Sclerosis Impact Profile (MSIP) (14). This measurement instrument has shown to be a feasible assessment tool in practice and psychometrically sound measures in research. To the best of our knowledge there is no broad ICF-based health measure covering all 600 NMDs. Therefore the aim of this study was to develop an ICF-based measure, the Neuromuscular Disease Impact Profile (NMDIP), with the intention of reflecting the prevalence and severity of a broad range of disabilities related to neuro­muscular diseases, using the ICF features such as ICF terminology and ICF qualifiers and to evaluate the psychometric properties of this new measurement instrument.

METHODS

Sample and procedures

A cross-sectional postal survey was conducted among patients diagnosed with a neuromuscular disease and registered at the Department of Neurology of the University Medical Center Groningen, the Nether­lands. Criteria for inclusion were: diagnosed with a neuromuscular disease, aged 18 years or older, and able to read and write in Dutch.

In total 1003 eligible patients were selected from the hospital patient record system with the aim of representing the 4 major neuromuscular disease (NMD) groups defined by Rowland: motor neurone disorders, muscle disorders, junction disorders and, peripheral nerve disorders (15). To prevent any inappropriate sending of the questionnaire, we cross-checked for deceased patients using the national population register.

Patients received information about the study and were invited to participate. Respondents completed the preliminary NMDIP, generic and domain-specific questionnaires, along with some demographic and disease-specific questions. Reminders were sent out after 2 weeks.

Ethical approval was obtained from the local ethics committee. Reference: METc 2009.310.

Preliminary Neuromuscular Disease Impact Profile

The preliminary NMDIP was developed as a disease-specific and ICF-based measure to assess disability among patients with an NMD. We used the 69 ICF categories of the NMD-Core Set (3). These categories are divided over the 4 ICF components. Selected categories were operationalized in order to estimate the patient’s objectified opinion (impairment in body functions, limitations in activities or restrictions in participation) of the incidence and severity of a disability, and to estimate the support from relevant environmental factors. Furthermore, ICF terminology for “disabilities” was applied, ICF item labels were used when formulating the subject of the question (e.g. “urination” functions instead of “bladder” functions), and ICF codes (e.g. b280 or p920) were documented for each question (14). Illustrative examples were annexed (using fourth-level ICF-items) to some questions to ensure an adequate response. To record the presence and severity of a problem in functioning, we applied response scales with scoring options specified for each ICF component, based on “qualifiers” proposed by the ICF (14).

The preliminary questionnaire was reviewed by patients, clinicians, nurse specialists, experts on the ICF and methodologists (n = 24) for clarity, comprehensiveness, redundancy and patient burden. A modified questionnaire was pre-tested in a random sample of 3 clinicians and 50 patients who were not involved in the first appraisal of the questionnaire. Unclear or ambiguous items and instructions were identified and some modifications of the questionnaire were made.

Finally, the preliminary NMDIP reflects an objectified view of the prevalence and severity of NMD-related disabilities and consists of 45 items representing the 4 ICF components.

Measurement instruments

For evaluating the psychometric properties of the NMDIP, 2 generic and 2 domain-specific measures were used.

The SF-36 is a broad and generic HRQoL measure that consists of 36 items divided over 8 domains (7). For each domain, item scores were transformed to a scale that ranges from 0 (worst health) to 100 (best health). In a previous study among Dutch multiple sclerosis patients the SF-36 domains showed satisfactory levels of internal consistency: Cronbach’s alpha ranged between 0.74 and 0.96 (14).

The World Health Organization Quality Of Life (abbreviation version) (WHOQOL-BREF) is a broad and generic measure of global QoL (16), and consists of 26 items divided over 4 domains. For each domain, item scores were transformed to a scale that ranges from 0 (worst health) to 20 (best health). In a previous study among Dutch multiple sclerosis patients the WHOQOL-BREF showed good levels of internal consistency: Cronbach’s alpha ranged between 0.80 and 0.81 (14).

The GARS is a domain-specific instrument to measure limitation, and consists of 18 items divided over 2 domains (9). A 4-category response format is used, ranging from 1 (no problem in performing without help) to 4 (impossible to perform). Scores are summed for each subscale. The GARS showed strong levels of internal consistency: Cronbach’s alpha ranged from 0.95 to 0.97 in a study in a Dutch sample of multiple sclerosis patients (14).

The IPAQ is a domain-specific questionnaire focusing on person-perceived participation and autonomy (10, 17). The instrument assesses 2 aspects of participation: perceived participation and the perceived problem. In this study we applied the perceived participation part that consists of 24 items divided over 5 domains. Items are scored on a 5-point rating scale with discrete responses, ranging from 1 (very good) to 5 (very poor). Scores are summed for each domain. In a previous study among Dutch multiple sclerosis patients, the IPAQ showed good levels of internal consistency: Cronbach’s alpha ranged from 0.86 to 0.94 (14).

Item reduction

Exploratory factor analysis with oblique rotation (Geomin) (20, 23) was used to examine whether the domains measured by the NMDIP represent the 4 ICF components. To improve the content validity the prevalence of each item was examined before entering items in the factor analysis. Items with a low prevalence (≤ 20%) were excluded from further analysis (18, 19). Factor analyses were performed using Mplus 6 software (20). Given the categorical nature of the variables, methods based on polychoric correlations and the robust-weighted least squares estimators (21) were used. Goodness-of-fit of the underlying factorial structure was measured by the root mean squared error of approximation (RMSEA, adequate if below 0.06) and the standardized root mean square residual (SRMR, adequate if below 0.08), the Comparative Fit Index and the Tucker-Lewis Index. For the latter 2 indices, it is recommended that values should be greater than 0.95 (22, 23). Items with factor loadings ≥ 0.40 were selected for scale construction.

Items that could not meet the criteria of scalability were taken into consideration for use as a single indicator.

Missing items

The maximum number of missing items allowed to be replaced by the mean scale score was determined by a sufficient Cronbach’s alpha in relation to the number of scale items (19, 24).

Psychometric evaluation

The distribution of scale scores was evaluated by calculating the median, mean and standard deviation and observed score range. Proportion of patients with worst and best possible scores (floor and ceiling effect) were calculated. Proportions ≤ 20% were considered acceptable (18).

Internal consistency was examined with Cronbach’s alpha (25) and the mean inter-item correlation coefficient (MICC) for each scale (26, 27). Alpha was considered sufficient if ≥ 0.70 (28, 29), and MICC if ≥ 0.30 (26).

To test whether NMDIP scales measure physical, psychological, social, and environmental domains of functioning, as they purport to measure, convergent and discriminant validity were assessed by examining the extent to which correlation values between NMDIP scales and concurrent measures were consistent with hypotheses (30, 31). Regarding convergent validity, we hypothesized that the NMDIP scales would have a strong correlation (≥ 0.70) (32) with scales that cover the same domain in concurrent measures. For example, NMDIP scales for physical functions should correlate highly with the SF-36 “Physical Functioning scale”. To support discriminant validity, we hypothesized that the NMDIP scales would correlate weakly (< 0.40) with scales measuring different domains in NMDIP or concurrent measures. For example, NMDIP scales for physical function would correlate weakly with mental or emotional scales of the SF-36.

Regarding known-groups validity (30, 31), we hypothesized that the NMDIP scales should be able to discriminate between subgroups of respondents known to differ on relevant clinical characteristics. The level of limitations due to a neuromuscular disease was used to create relevant subgroups of respondents. Therefore, the generic question “Extent of limitations” was used. Respondents were asked to answer the question “To what extent are you limited due to a neuromuscular disease?” on an 11-point scale with a score range from 0 (not limited at all) to 10 (severely limited). Next, respondents were divided into 2 groups: those with a “lower extent of limitations” (score 1–4), and those with a “higher extent of limitations” (score 5–10).

Statistical analysis

Patient characteristics were analysed using descriptive statistics. Spearman’s correlation coefficient (rho) was used to examine convergent and discriminant validity. Known-groups validity was assessed using the independent Mann-Whitney U test.

To estimate the magnitude of the difference in scores between subgroups of respondents, the non-parametric effect size (coefficient r) for unrelated samples was calculated for statistically significant group differences (alpha = 0.05) (33). Coefficient r is calculated by dividing the z statistic (obtained from the Mann-Whitney U test) by the root of the sample size (n). To interpret the non-parametric effect sizes using coefficient r. Cohen suggested the following thresholds for interpretation: an r of < 0.10 indicates a trivial effect, an r of ≥ 0.10 to < 0.24 a small effect, an r of ≥ 0.24 to < 0.37 a moderate effect, and an r ≥ 0.37 a large effect. An r ≥ 0.10 reflects a clinically relevant difference between groups (33, 34). IBM SPSS statistics version 20 was used.

RESULTS

Patient characteristics

In total 702 participants (70% response rate) completed the questionnaires. Demographics and disease-specific characteristics are described in Table I. Mean age was 59 years (SD 16, range 19–92 years), while slightly more than half of the patients were younger than 65 years. Mean number of “years since diagnosis” was 12 (SD 11, range 0–65 years). Approximately 30% of the patients were retired due to a neuromuscular disease. The motor neurone disorder subgroup was a relatively small sample compared with the other neuromuscular disease subgroups according to Rowland’s classification (15).

Non-respondents did not differ from respondents in terms of gender, but were significantly younger (mean 53, SD 19 years) than respondents (mean 59, SD 16 years).

Content validity

Nine of the original 45 items showed a low prevalence (≤ 20%) and were not entered in the factor analysis. These items were from the component “Activities” (“a350 Conversation”, “a360 Using communication devices and techniques”, and “a465 Moving around using equipment”), the component “Participation” (p510–p540 items concerning “Personal care”, “p360 Communication devices and techniques, “p630 Eating and drinking”, “p610 Acquiring a place to live” and “p850 Remunerative employment”), and from the component “Environmental Factors” (“e340 Personal care providers and personal assistants”).

Item reduction

The EFA models showed a (very) good fit for the 5-factor model for the Body Functions component, the 2-factor model for the Activities component, and the 1-factor model for the Participation component. Comparative Fit Index and Tucker-Lewis Index values were above 0.95, SRMR values were below 0.08 as recommended. The RMSEA values were below 0.06 for the Body Functions component and Participation component, while the value for the Activities component was slightly higher (0.069), but still acceptable. For the Environmental Factors component no satisfying fit was found. Factor analysis reduced the remaining 36 items of the initial NMDIP to 32 items:

• Five factors within the Body Functions component included 15 items (Table II). Interpretation of item content, using the ICF first- and second-level category labels led to the following scale labels: “Muscle Functions” (MuF), “Movement Functions” (MoF), “Swallowing and Speech Functions” (SSF), “Excretion and Reproductive Functions” (ERF), and “Mental Functions and Pain” (MFP).

• Two factors within the Activities component included 14 items (Table III). These factors were given the labels, “Activities of Moving around”(AMA) and “Self-care and Domestic Activities” (SDA). One factor within the Participation component included 3 items (Table IV). This factor was labelled “Participation in Life Situations” (PLS).

Finally, the Body Functions component items “Seeing functions” and the Environmental Factor component items “Immediate family”, “Social security services”, and “Health services” with a sufficient prevalence and clinical relevance were added to the questionnaire as single items.

Scale features are shown in Table V. Internal consistency for 7 NMDIP scales for the total sample was good and moderate for 1 scale. Cronbach’s alpha ranged from 0.63 to 0.92 and mean inter-item correlation coefficient ranged from 0.38 to 0.77. Four scales showed a high floor effect.

The NMDIP scales within the NMD groups showed acceptable to good internal consistency. (Table VI). For some scales the Cronbach’s alpha was weak, but this was compensated by a sufficient mean inter-item correlation coefficient, except for the “Swallowing and Speech Functions” scale in the peripheral nerve disorder group and the “Excretion and Reproductive Functions” scale in the muscle disorder group.

The final version of the NMDIP consists of 36 items divided over 8 scales and 4 single items (see Appendix I).

Psychometric testing

Table VII provides evidence of convergent and discriminant validity for the NMDIP scales, reflecting the impact of a neuromuscular disease on physical, psychological, and social aspects of functioning. The predictions were consistent with the direction, magnitude, and patterns of correlations.

Convergent validity was supported by a strong correlation between the NMDIP “Muscle Functions” scale and the SF-36 “Physical Functioning” scale, and a strong correlation between the NMDIP “Lower Extremity Activities” and “Upper Extremity Activities” scales, and the GARS “Activities of Daily Living” and “Instrumental Activities of Daily Living” scales. The NMDIP “Participation in Life Situations” scale was moderately correlated with the IPAQ participation scales.

Discriminant validity was supported by weak correlation values found, for example, between the NMDIP “Mental Functions and Pain” scale and the SF-36 “Physical Functioning” scale, and the GARS “Activities of Daily Living” and “Instrumental Activities of Daily Living” scales. Similarly, the NMDIP “Muscle Functions” scale correlated weakly with the SF-36 “Mental Health and “Role Emotional” scales, and the NMDIP “Participation in Life Situations” scale correlated weakly with the SF-36 “Mental Health” scale.

Evidence of known-groups validity was obtained for all NMDIP scales by statistically significant group differences and clinically relevant effect sizes (Table VIII). Patients classified as having a higher extent of limitation had statistically significant higher scores on all NMDIP scales compared with those classified as having a lower extent of limitation. Effect sizes were moderate for 2 scales, and large for 6 scales.

DISCUSSION

The objective of this study was to develop a psychometrically sound ICF-based measure for estimating the prevalence and severity of a broad range of disabilities related to neuromuscular diseases using ICF features such as ICF terminology and ICF qualifiers.

The results provide evidence to support the validity and reliability of the final version of the Neuromuscular Disease Impact Profile (NMDIP) as an instrument to measure the prevalence and severity of a broad spectrum of consequences of a neuromuscular disease including disabilities in Body Functions, Activities and Participation, and lack of support from Environmental Factors. The NMDIP can be used as a clinical and research instrument for the assessment of the impact of a neuromuscular disease.

The original 45 items in the preliminary NMDIP could be reduced to 36 items: 32 items covering 8 domains representing 3 ICF-components, and 4 clinically relevant items (1 Body Functions item and 3 Environmental Factors items), which were applied as single items in the questionnaire (see final version in Appendix I).

Although the NMDIP used the same items as the initial MSIP (14), results of the factor analysis showed some differences compared with the final MSIP scales. For example, the MSIP “Muscle and Movement functions” 4-item scale is represented in the NMDIP in 2 separate and recognizable scales “Muscle functions” with 2 items and “Movement functions”, also with 2 items. Furthermore, the 3-item “Mental functions” MSIP scale appeared in the NMDIP as a 5-item version: new scale items were Fatigue and Pain. This can be explained by the fact that pain and fatigue are the direct result of (using) weakened muscles, which is a common symptom in neuromuscular diseases. Unlike neuromuscular diseases fatigue in multiple sclerosis is most likely related to the process of inflammation, while pain originates from spasticity and/or neuropathy. Furthermore, scale construction also identified a construct that was not present in the MSIP: “Swallowing and Speech Functions”. This can be explained by the fact that some myopathies and the myasthenia’s tend to affect bulbar musculature. Finally, analysis showed no consistent factor for the Environmental Factors items.

Reliability of the NMDIP scales for the total sample was sufficient for 2 scales and good for 6 scales. The internal consistency of the scales per separate NMD group was generally sufficient, except for the “Swallowing and Speech Functions” scale in the peripheral nerve disorder group and the “Excretion and Reproductive Functions” scale in the muscle disorder group. Some caution is advised in the interpretation of the results of these scales. Convergent and discriminant validity analysis indicated that the NMDIP measures the impact on physical, mental, and social functioning for people with a neuromuscular disease.

The correlation between the NMDIP “Participation in Life Situations” scale and the SF-36 “Physical Functioning” scale was unexpectedly higher. It is likely that the activity-related participation items in the NMDIP scale are responsible for this moderate correlation.

Known-groups validity was supported for the 8 NMDIP scales. Scales discriminated sufficiently between groups of patients with a neuromuscular disease that differed in extent of limitations.

An important strength of this study is the large and broad group of participating patients with a neuromuscular disease, and the sound conceptual basis used in developing the NMDIP (3, 35).

A possible limitation of this study is the small sample size of the motor neurone disorder group, compared with the sample size of the 3 other NMD groups. However, in our opinion the disabilities in this group are sufficiently represented in the NMDIP because the basis of the NMDIP, the NMD ICF-Core set, covers all items of the disease-specific Amyotrophic Lateral Sclerosis Assessment Questionnaire-40 (8). Another limitation could be the high floor effect of some scales that might affect the reliability of these scales (36). However, these floor effects match the course of the slowly progressive nature of most NMDs. This means that some disabilities appear years after onset, such as speech and swallowing functions or upper extremity activities.

Further research should focus on psychometric evaluation concerning stability and sensitivity to change of the NMDIP scales, and validation across other populations of neuromuscular disease patients in other cultures. It would also be interesting to examine the differences in prevalence and severity of disabilities between the 4 major NMD groups, as defined by Rowland (15). Finally, it would be interesting to investigate the impact of the broad range of NMDIP-related disabilities on HRQoL of patients with neuromuscular diseases.

We considered the possibility of undertaking group invariance testing; however, the sub-groups are relatively small and will affect the test of Differential Item Functioning. We therefore suggest further examination of the factor structure in a new sufficient sample.

In clinical practice, especially in multidisciplinary rehabilitation teams, the NMDIP may contribute to better understanding the patients’ health problems when used as an assessment tool. Although positive results were found in the feasibility studies with the preliminary NMDIP and the MSIP, it is advisable to combine this application with research; for example, in order to investigate the effects on the healthcare plan when using the NMDIP.

ACKNOWLEDGEMENTS

The authors wish to thank the patients with a neuromuscular disease who participated in this study and were so kind as to share personal information about the consequences of their disease, and for taking the time to complete the questionnaires. We also wish to thank the students who sent out the questionnaires and entered the data from the questionnaires: Kyra van der Beek, Hanna Bosman, Annelies, Carolien and Marieke Verschure, and Ronald Brans. We finally thank Berrie Middel (PhD) for methodological support and Josué Almansa Ortiz (PhD) for statistical support.

The authors declare that they have no conflicts of interest regarding this manuscript.

REFERENCES