Torunn Askim, PhD1,2, Julie Bernhardt, PhD3,4, Leonid Churilov, PhD3,5 and Bent Indredavik, MD, PhD1,6

From the 1Department of Neuroscience, Faculty of Medicine, NTNU, Norwegian University of Science and Technology, 2Department of Physiotherapy, Faculty of Health and Social Science, NTNU, NorwegianUniversity of Science and Technology, Trondheim, Norway, 3Stroke Division, Florey Institutes of Neuroscience and Mental Health, 4Faculty of Health Sciences, La Trobe University, 5School of Mathematical and Geospatial Sciences, RMIT University, Melbourne, Australia and 6Stroke Unit, Department of Medicine, St Olavs Hospital, Trondheim, Norway

BACKGROUND: The National Institutes of Health Stroke Scale (NIHSS) is the first choice among stroke scales. The Scandinavian Stroke Scale (SSS) is an alternative scale that is easy to apply in the clinic.

Aim: To compare the ability of the SSS with that of the NIHSS in identifying patients who are dead or dependent at 3-month follow-up.

METHODS: A prospective study including patients with acute stroke. NIHSS and SSS measurements were obtained during index hospital stay. The receiver operating characteristics curve was used to determine the optimal dichotomization of the NIHSS and the SSS by using a modified Rankin Scale (mRS) >2 at 3-month follow-up as the criterion standard. Positive and negative predictive values (PPV and NPV) were calculated.

RESULTS: A total of 104 patients (mean age 79 years, 57.7% men) were included. Median (interquartile range (IQR)) NIHSS and SSS score were 6.0 (2.0–11.8) and 43.5 (30.0–51.0), respectively. The areas under the curve were 0.769 and 0.796 for NIHSS and SSS, respectively, χ2 (p = 0.303). The best cut-off point for NIHSS was 6/7 points (PPV = 76.2%, NPV = 69.0%) while for SSS it was 42/43 points (PPV = 71.4%, NPV = 73.2%).

CONCLUSION: The SSS was as good as the NIHSS in identifying patients who had died or were dependent at 3-month follow-up. The measurement properties of the SSS should be investigated further.

Key words: acute stroke; prediction; outcome assessment; ROC curve; neurological examination.

J Rehabil Med 2016; 00: 00–00

Correspondence address: Torunn Askim, Department of Neuroscience, NTNU, Faculty of Medicine, Postbox 8905, NO-7491 Trondheim, Norway. E-mail: torunn.askim@ntnu.no

Accepted Aug 22, 2016; Epub ahead of print Oct 13, 2016

INTRODUCTION

The National Institutes of Health Stroke Scale (NIHSS) is a well-established and extensively used measure in acute stroke treatment, which is recommended for use in clinical trials after stroke (1). The reliability of the scale is clear, but some items have consistently shown moderate to low inter-rater reliability (Kappa score less than 0.75). These items are level of consciousness, gaze, facial palsy, ataxia and dysarthria (2, 3). The ability of the NIHSS to identify 3-month outcome has shown to be superior to other stroke scales, such as the Canadian Neurological Scale and the Middle Cerebral Artery Neurological Score (1). Furthermore, a baseline NIHSS score ≤ 6 is associated with a high probability of good recovery, and the ability to predict 6-month outcome is shown to be equally good at 2 days vs 5 days or 9 days post-stroke (4, 5).

The Scandinavian Stroke Scale (SSS) is an alternative stroke scale, which is frequently used in Scandinavian countries and has recently also been validated in the Portuguese language (6). Inter-rater reliability of the items varies from excellent for conscious level, orientation and gait, (kappa 0.84, 0.86 and 1.0, respectively) to moderate for facial palsy (kappa 0.59) (6). In a multivariate logistic regression model, neurological recovery, as measured by the SSS change score during the first week after onset of stroke, was shown to be an independent predictor of good functional outcome (7, 8).

The advantage of the SSS is its simplicity, which makes it easy to perform repeated measures in the very acute phase after stroke (9). However, the ability of the SSS to identify outcome at 3 months after onset of stroke has not been validated. As the NIHSS is regarded as the gold standard measure, the purpose of the present study was to compare the SSS with the NIHSS to identify patients who are dead or dependent at 3-month follow-up. A secondary aim was to compare their ability to identify outcome in patients from different age groups and with different severity levels.

MATERIAL AND METHODS

Methods

This was a prospective cohort study with an initial assessment within 14 days after onset of stroke and a follow-up assessment conducted in the patient’s home 3 months later.

All patients admitted to the Stroke Unit at Trondheim University Hospital, Norway, with the diagnosis of stroke were eligible for inclusion, except for those with a devastating stroke receiving end-of-life palliative care. Eligible patients were included if they were able and willing to sign informed consent. Patients who were not able to give informed consent were also included if their next of kin gave oral consent to participation. The study was approved by the Regional Committee of Medical and Health Research Ethics and Norwegian Social Science Data Services.

Age, sex, time since stroke, stroke type, Oxfordshire Classification of Stroke, NIHSS, total score ranges from 0 (no symptoms) to 42 and Scandinavian Stroke Scale, total score ranges from 0 to 58 (no symptoms) were assessed at baseline. At 3 months follow-up a home visit was conducted for all surviving patients. Death or dependency 3 months after stroke was determined by modified Rankin Scale (mRS) (11). Two well-trained assessors performed the baseline assessments. The same assessor obtained the NIHSS and the SSS score. A third assessor, who was blinded to the initial assessment, performed the follow-up assessments.

Statistical analysis

Descriptive statistics were used for the baseline characteristics. Primary outcome was comparison of the area under the receiver operating characteristics (ROC) curve (AUC) for both NIHSS and SSS. The χ2 test was used to compare the AUCs for the NIHSS and the SSS. ROC curve analysis was used to determine the optimal dichotomization of the NIHSS and the SSS, respectively, using Youden’s criteria (12). The criterion standard was mRS > 2. Since the SSS ranges from 0 to 58, with 0 as the worst score, and the NIHSS ranges from 0 to 42, with 42 as the worst score, the SSS sum score was transformed to an SSS inverse score according to the following equation; SSS inverse = maximum SSS score minus original SSS score. The SSS inverse score was used in the ROC analysis to make it possible to compare the 2 scales. In all other analyses, the original score of the SSS has been reported. Sensitivity and specificity were reported for the optimal dichotomization, while positive and negative predictive values (PPV and NPV) were standardized to a 50% pre-test chance of responding. For the subgroup analysis, patients were divided into subgroups according to age (dichotomized at 80 vs 81 years) and according to stroke severity (NHISS< 8, NIHSS 8–16, NIHSS> 16). The statistical analysis was conducted using IBM SPSS Statistics version 19 and Stata v13.

RESULTS

Over 18 months, 124 patients were included in the study; 23% of all patients admitted to the stroke unit during this period. Two patients were excluded because inclusion exceeded 14 days post-stroke, one patient was found not to have stroke, and 6 patients were excluded because of incomplete data at baseline. Eleven patients were lost to follow-up, leaving a total of 104 patients in the analysis.

At baseline, the median age was 81 years and 57.7 % were male. Median (IQR) NIHSS and SSS scores were 6.0 (2.0–11.75) and 43.5 (30.0–51.0), respectively (Table I).

At 3-month follow-up, a total of 59 patients were classified as dependent (mRS > 2) and 45 as independent. Nine patients (15.3%) died in the 3-month follow-up interval. The ROC curve (Fig. 1) revealed an AUC of 0.769 and 0.796 for the NIHSS and the SSS, respectively, p = 0.303. The optimal dichotomization was between 6 and 7 points for the NIHSS (64.4% sensitivity and 80.0% specificity), while for the SSS it was between 42 and 43 points (69.5% sensitivity and 82.2% specificity).

Fig. 1. Receiver operating characteristics (ROC) curves for the National Institutes of Stroke Scale (NIHSS) and the Scandinavian Stroke Scale (SSS) with the modified Rankin Scale > 2 as criterion standard.

Ninety-three patients (89.4%) were equally classified as dependent or independent with the SSS and the NIHSS, while 11 (10.6%) patients were incongruent in classification.

Table II shows the predictive values standardized to a 50% pre-test chance of being dependent. The PPVs showed a 76% chance of being dependent at 3-months post-stroke if the NIHSS score was 7 points or more, and a 71% chance of being dependent if the SSS score was 42 points or less. The NPVs were 69% and 73% for NHISS and SSS, respectively.

The results of subgroup analysis are shown in Table III, showing no significant differences between the two scales for any of the subgroups. However, both scales showed better measurement properties for patients of older age (≤ 80 years) and those with moderate strokes (NIHSS 8–16).

DISCUSSION

This study showed that the SSS was equally as good as the NIHSS in identifying patients who were dead or dependent 3 months after stroke. Subgroup analysis showed that the measurement properties for both scales were better for patients with moderate strokes and for those of older age.

Modified Rankin Scale>2 is the most widely used definition of death and dependency after stroke (13). Using this definition as the criterion standard for the ROC analysis revealed an AUC of 0.769 for NIHSS and 0.796 for SSS, which should be regarded as adequate (14). Even though the AUC for SSS was slightly larger compared with NIHSS, there were no statistically significant differences between the two scales.

From a clinical perspective the PPV and NPV are of even greater interest. However, these two values are known to be prevalence dependent. To make comparison between scales more feasible the predictive values should be standardized to a 50% pre-test chance of responding. A NPV of 73% for the SSS and 69% for the NIHSS means that patients have a 73% chance of truly being independent if the initial SSS score is 43 points or more, or a 69% chance if the NIHSS score is 6 points or less, indicating that the SSS is superior to the NIHSS in ruling out the problem. However, the PPV was 71% for the SSS vs 76% for the NIHSS, indicating that the NIHSS might be better than the SSS in predicting the chance of being dependent at 3 months. Whether one should use the PPV or the NPV is a matter of discussion. From the patients’ perspective it might be of greater value to know the chance of being independent, while for the healthcare system it might be of greater value to know the chance of being in need of future healthcare services.

Although the measurement properties of the NIHSS and the SSS are equal in their ability to identify outcome, the superiority of the SSS lies with its simplicity and ease of use in the clinic. An example is the difference in measuring motor function between the two scales. While in the NIHSS patients are asked to keep their limb against gravity for 10 s, which means that you need a watch, in the SSS patients must keep their limb against manual resistance to get a full score. This advantage of the SSS is of particular importance when repeated measures of selected items are used to detect early neurological deterioration (9).

In the NIHSS muscle power is measured in both the affected and the unaffected limb, while SSS measures only the affected limb. This might indicate that the two scales act differently in patients with first ever and recurrent stroke. Future studies should therefore assess measurement properties in these subgroups of patients.

The present study included a rather unselected stroke sample; however, the prevalence of death (15.5% at 3 months) and dependency appears to be a little lower than in the general stroke population (15). Hence, the next step should be to validate the predictive capacity of the SSS and the NIHSS in a new patient sample.

In conclusion, the SSS and the NIHSS showed equally good measurement properties. The SSS is a simple tool, which is easy to apply in the acute clinical setting, and its measurement properties should be further investigated.

REFERENCES