Accepted Sep 28, 2017; Epub ahead of print Dec 19, 2017

J Rehabil Med 2018; 50: 00–00

Correspondence address; Angela Vratsistas-Curto, Musculoskeletal Health Sydney, School of Public Health, The University of Sydney, Sydney, Australia. E-mail: angela.vratsistas@sydney.edu.au

Falls and fall-related injuries pose a significant burden to older people, their families and healthcare systems. The importance of falls in healthcare will increase substantially with the ageing global population. Falls occurring during hospital admissions are common and costly and can result in injury and increased length of stay (1). In Australia approximately 14% of older people fall during inpatient rehabilitation (2, 3).

Routine use of inpatient fall risk screening tools is suggested in some guidelines (4, 5). The commonly-used Ontario Modified STRATIFY (OMS) (6) is a revised version of the STRATIFY falls screening tool with 5 items (history of falls, mental status, vision, toileting, and transfers/mobility) that are predictive of falls in acute or mixed settings (7). Predict_FIRST (Prediction of Falls In Rehabilitation Settings Tool) (2) was developed by some of the present authors and contains 5 items (male sex, central nervous system (CNS) medication use, falls in the past year, frequent toileting and inability to tandem stance). A simple question about past falls has been identified as a strong predictor of future falls (8). Experienced clinicians may accurately predict falls risk without using formal tools (9, 10).

The optimal approach to fall prediction in inpatient rehabilitation is not clear. OMS is widely used in rehabilitation internationally and recommended in Australian best practice guidelines (1), but has not been externally validated in rehabilitation (11). Predict_FIRST was developed for use in rehabilitation, but has not been externally validated. The benefit of using formal fall risk prediction tools rather than clinical judgement or falls history is unknown. This study aimed to address these gaps by comparing several fall prediction methods for use with adults undertaking inpatient rehabilitation. A valid and quick falls screening tool is more likely to be acceptable to clinicians than a longer tool. This study also explored the validity of a shorter fall prediction tool.

The aims of the study were: to test the external validity of 4 approaches to fall prediction in a rehabilitation setting (Predict_FIRST, Ontario Modified STRATIFY (OMS), physiotherapists’ judgement of fall risk (PT_Risk), and falls in the past year (Past_Falls); and to develop and test the validity of a simpler fall prediction tool in rehabilitation (Predict_CM2).

The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines were used in reporting this study (12, 13). Therefore the design of the present study, OMS and Predict_FIRST development studies were compared (see Supplementary Material¹).

Prospective inception cohort study. The study was approved by the SWSLHD ethics committee(s) (HREC number QA2010/005). Approval was granted to audit patient records without gaining written consent from patients.

A consecutive sample of 300 inpatients admitted to the general rehabilitation unit at a public hospital in Sydney, Australia participated in the study. Recruitment occurred between April 2010 and May 2011. All admitted patients were considered for inclusion except those who were not receiving rehabilitation, e.g. acute medical or palliative care patients.

The lead author (AV) extracted demographic data (primary diagnosis, age and sex) and falls experienced in the previous 12 months from participants’ medical records. Medications were transcribed from the medication chart.

The OMS was scored by rehabilitation nursing staff on admission as part of usual care and data for this study were extracted from participants’ medical records. The Sydney_Scoring version of the OMS was used. This version was devised by Lord and colleagues (1) in response to clinician feedback that the transfer and mobility components of the original OMS were confusing to score. The Sydney_Scoring comprises a simple modification to the mobility score (MS) and transfer score (TS) so that scores on these items are graded in the same direction as all other OMS items. This modification (devised solely for ease of use of the scale) provides the same TS + MS weighted sub-score or overall OMS score as the original scale.

Three versions of the OMS were tested in this study: (i) OMS (Total_Score) using the continuous total score (/30); (ii) OMS (Dichotomous) using the dichotomous scoring/categories from the original OMS (“At risk” [< 9] or “No risk” [≥ 9]); and (iii) OMS (Risk_Categories) using low (0–5), medium (6–16) and high (17–30) risk categories (1). OMS (Risk_Categories) has not been previously validated, so our study also tested the validity of this tool. OMS items were defined in accordance with the original paper for all versions (6).

History of falls was defined as whether the patient presented to hospital with a fall, had fallen since admission or fallen within the last 2 months (6). Impaired mental status was defined as being confused, disorientated or agitated (6). Poor vision was defined as the patient requiring glasses continuously; having blurred vision; or glaucoma, cataracts or macular degeneration (6). Toileting assistance was defined as alterations in urination, i.e. frequency, urgency, incontinence and nocturia (6). Impaired transfers/mobility was defined as needing at least the help of one person to walk and major help of one person to transfer (the use of aids is allowed) (6).

Predict_ FIRST scores were calculated on admission using information from participants’ medical records. Male sex was extracted from the file. CNS medication use was defined as taking sedatives/hypnotics, anti-anxiety agents, antipsychotics, antidepressants, anticonvulsants, movement disorder medications or other CNS agents (2). Falls in the past year was defined as a reported or documented history of falls in the previous 12 months (2). Frequent toileting was defined as alterations in urination, i.e. frequency, urgency, incontinence and nocturia (2). Impaired tandem stance was defined as the inability to maintain the tandem stance position for 10 seconds on initial physiotherapy assessment (2).

On admission to rehabilitation, the treating physiotherapist rated and recorded the patient’s likelihood of falling during rehabilitation with a simple yes/no response.

Primary outcome. Number of falls during rehabilitation stay. A fall was defined as unintentionally coming to rest on the ground or other lower surface without overwhelming external force or major internal event (14). Falls were monitored and recorded by the lead author (AV) during the admission from incidents reported in medical records and the ward’s fall incidents book, both completed as part of usual care. Supplementary¹ checks of the hospital incident reporting system and regular communication with treating physiotherapists were conducted to optimize falls ascertainment (15). Falls reported by staff and/or recorded in the clinical record, but not entered in the hospital incident reporting system were included in the analysis.

External validation study. Discrimination, i.e. the ability of a model to distinguish fallers from non-fallers (16), was assessed using the area under the receiver-operating characteristic (ROC) curve (AUC) and AUCs for different models were compared with the roccomp command in Stata 13. The roccomp command was used to test the equality of the ROC areas obtained from applying each of the tools to our sample. The command uses an algorithm developed by DeLong and colleagues producing a χ² statistic and associated p-value (17). A p-value < 0.05 was interpreted as 2 tools being significantly different in their predictive ability, with the tool with the higher AUC being significantly better at predicting falls. Predicted and observed probability of falling for individuals with different scores on OMS and Predict_CM2 were calculated.

Predict_CM2 development study. To develop the Predict_CM2 and determine its ability to discriminate between fallers and non-fallers, we used the same sample as in our external validation study of 300 people.

Associations between individual predictor variables (from Predict_FIRST and OMS) and falls were assessed using univariate logistic regression analyses from which odds ratios (OR) and 95% confidence intervals (95% CI) were obtained. Variables measuring similar constructs were grouped into domains: past falls; impaired cognition; impaired vision; frequent toileting; impaired mobility; CNS medication use; and sex. Five of these variables from each domain for which the individual p-values were ≤ 0.2 and the odds ratios were greater than 1.5 when dichotomized at the median were identified as candidate predictor variables for multivariate logistic regression models. To increase the confidence in variable selection for the new tool multi-variable analyses were repeated on boot-strapped samples with user written code in Stata 13, to establish the proportion of models in which each predictor was retained. The boot-strapping used 1,000 samples, each with n = 300 and involved separate backwards selection logistic regression models in which variables with a p-value < 0.2 was removed from the models. This was repeated for each sample as the percentage of models in which each variable as retained was reported. Predict_CM2 was developed selecting measures retained in at least 75% of samples.

Calibration of existing tools and new tool (Predict_CM2). Calibration (the extent to which predicted probabilities agree with observed probabilities) (16) of Predict_CM2 and OMS was tested with the Hosmer-Lemeshow statistic in Stata 13. A p-value <  0.05 was interpreted as indicating the model did not fit the data. Multi-level (stratum-specific) likelihood ratios were also calculated.

Flow of participants through the study. During the study 331 patients were admitted to the participating ward. Of these, 300 were eligible for and included in the study (Table I). Thirty-one patients were excluded as they were admitted for acute medical care and did not receive rehabilitation. Complete data were available for all 300 participants.

Falls. Forty-one people (14%) experienced 53 falls during their rehabilitation stay. Ten people (3%) had ≥ 2 falls and eight people (3%) had ≥ 3 falls.

Individual items and falls. Univariate associations between Predict_FIRST and OMS items and faller status are shown in Table II. Variables with the strongest association (OR ≥ 2.0, p < 0.05) with falls included impaired cognition, frequent toileting and impaired mobility/transfers.

Predictive ability of Predict_FIRST, OMS, PT_Risk and Past_Falls. The AUC for Predict_FIRST was 0.66 (95% CI: 0.57–0.74), lower than the AUC from the tool’s development and internal validation study (AUC: 0.73, 95% CI 0.68–0.79). The AUC for OMS (Total_Score) was 0.71 (95% CI: 0.61–0.81), higher than, but not statistically different from, Predict_FIRST (p = 0.28 for comparison). The AUC was not reported in the internal validation study for OMS (6) and therefore cannot be compared.

The AUC for OMS (Total_Score) was compared with other scoring versions of OMS. Using dichotomous scoring (“At risk” [< 9] or “No risk” [≥ 9]), OMS (Dichotomous) had an AUC of 0.64 (95% CI: 0.57–0.72), significantly lower than OMS (Total_Score) (p = 0.003 for comparison). Using the risk category scoring (high, medium, low), OMS (Risk_Categories) had an AUC of 0.69 (95% CI: 0.60–0.78), lower than, but not statistically different from, OMS (Total_Score) (p = 0.27 for comparison).

Physiotherapists’ judgement of fall risk (PT_Risk) had an AUC of 0.65 (95% CI: 0.57– 0.73), lower than, but not statistically different from, OMS (Total_Score) (p = 0.22). Falls in the last 12 months was a poor predictor of future faller status, with an AUC of 0.52 (95% CI: 0.46–0.60), significantly lower (p < 0.001) than OMS (Total_Score) (see Fig. 1).

Development of the predict_CM2. The analyses to determine predictors of falling compared 41 fallers with 259 non-fallers, as shown in Table II. The 5 variables that were found to be significant predictors of falls in the univariate analysis (male sex, mobility/transfers, cognition/mental status, CNS medication use, and frequent toileting) were entered into a multivariate regression model. Only these of 3 variables were also found to be significant predictors of falls in the multivariate logistic regression model (male sex, mobility/transfers, and cognition/mental status) and were retained in more than 75% of multi-variate models tested on 1,000 boot-strapped samples (Table III). The final tool was created using these 3 variables and named Predict_CM2 based on the first letter of each of the included variables (i.e. cognition, male, mobility).

Predictive ability of the Predict_CM2. The simple tool with risk factors equally (unit) weighted had an AUC of 0.73 (95% CI: 0.66–0.81, coefficient from logistic regression model 1.08, constant –3.31). The AUC for Predict_CM2 was significantly higher than Predict_FIRST (AUC: 0.66, 95% CI: 0.57–0.74, p = 0.04 for comparison) or Past_Falls (AUC: 0.52, 95% CI 0.46–0.60, p < 0.01) and higher than, but not statistically different from, OMS (Total_Score) (AUC: 0.71, 95% CI: 0.61–0.81, p = 0.52 for comparison) and PT_Risk (AUC: 0.65, 95% CI: 0.57–0.73, p = 0.08 for comparison). The AUC for Predic_CM2 was significantly higher than OMS (Dichotomous) (AUC: 0.64, 95% CI: 0.57–0.72, p = 0.008 for comparison) and higher than, but not statistically different from, OMS (Risk_Categories) (AUC: 0.69, 95% CI: 0.60–0.78, p = 0.17 for comparison).

Table IV shows the absolute probability of falling, with different scores on the different tools. A person with no Predict_CM2 risk factors (0/3) had a 3%, or very low probability of falling in hospital, while those with 3 risk factors had a 48%, or high probability. The resulting tool is shown in Appendix I. A person in the OMS (Dichotomous) “No risk” category (0–8/30) had a 7% probability of falling in hospital, while those in the “At risk” category (9–30/30) had a 21% probability. A person in the OMS (Risk_Categories) “Low risk” category (0–5/30) had a 3% probability of falling in hospital, while those in the “High risk” category (17–30/30) had a 30% probability.

The Hosmer-Lemeshow test did not detect a lack of fit between predicted and observed fallers for Predict_CM2 or OMS (Predict_CM2: p = 0.62, OMS Dichotomous: p < 0.01 and OMS Risk_Categories: p = 0.57). Likelihood ratios (Table IV) for Predict_CM2 (0.15–6.32) have a greater range than for OMS (Total_Score) (0.49–1.63) suggesting a greater usefulness of this tool.

This study found relatively poor performance of Predict_FIRST (AUC = 0.66), PT_Risk (AUC = 0.65) and Past_Falls (AUC = 0.52) at predicting falls in inpatient rehabilitation. While fall history has been found to be a predictor of falls in similar populations and settings, our study, as well as a more recent study, has found it to be a poor predictor of falls in rehabilitation (18). The performance of the OMS (Total_Score) (AUC = 0.71) provides good external validation of this tool and demonstrates it is significantly better in predicting falls than “Past_Falls” (p < 0.001). Given the time taken to administer the 5-item (with 11 sub-questions) OMS and relatively poor performance of the other screens, we developed a simpler 3-item tool for predicting falls in rehabilitation, Predict_CM2, on the basis of the strongest individual predictors in this sample. The predictive ability of Predict_CM2 (AUC = 0.73) was comparable to OMS (Total_Score), approached significance when compared with PT_Risk (p = 0.09) and was significantly better than Predict_FIRST (p = 0.04), or Past_Falls (p < 0.001) in predicting future fallers.

This study used data routinely collected from participants, whereas the Predict_FIRST development study involved additional assessment requiring consent. These design differences appear to have led to important differences in the results, with the present sample more representative of routine care. Only 3 predictor variables used in Predict_FIRST and OMS were strongly associated with falls in the present sample: mobility/transfers, mental status/cognition, and male sex. Cognition was not a strong predictor of falls in the Predict_FIRST study, probably due to exclusion of people with cognitive impairment for whom no person responsible was available to give consent. Tandem stance inability and CNS medication use were not strong predictors of falls in this sample, probably due to a higher proportion of participants with these risk factors compared with the Predict_FIRST study. We therefore propose a simplified fall prediction tool (Predict_CM2) including the 3 variables, mobility/transfers, mental status/cognition, and male sex, which are also predictive of falls in similar settings (8, 18–20).

The predictive ability of Predict_CM2 or OMS (Total Score) was better, but not statistically different (AUC = 0.73 and 0.71, respectively) from clinical judgement alone (AUC = 0.65). Previous studies have found clinical judgement to be a comparable or better predictor of falls than formal assessment tools (9, 10, 21). We suggest these differences may be statistically significant in a larger sample and by using a screening tool, clinicians with differing backgrounds and levels of experience are likely to more confidently and accurately determine the patient’s level of risk. The ability of Predict_CM2 and now OMS (Dichotomous and Risk Category versions) to provide an absolute risk of falling as a percentage may assist in communicating level of risk to staff as well as to patients and significant others.

The use of formal fall risk screening tools is not supported by all (4). UK guidelines recommend a multifactorial assessment, and do not support using risk prediction tools (5). American and British guidelines recommend fall risk screening by asking about past falls, fall frequency and difficulties in gait and balance (22). We suggest using a simple externally-validated tool addresses some concerns raised about using fall prediction tools. First, the Predict_CM2 provides an absolute risk of falling, which gives the clinician more information about falls risk than tools such as the OMS that provide risk categories alone. Secondly, using the tool will identify more patients at risk of falling than clinical judgement or past falls. Thirdly, such a tool can also be used to guide provision of interventions, but this approach should be evaluated in trials. For example, patients at high risk with cognitive impairment may need closer monitoring, relocating closer to the nurses’ station or require a special nurse. Patients with impairments in mobility and transfers may require more assistance when carrying out activities of daily living, and rehabilitation can be targeted to improve mobility and transfers.

Guidelines recommend that independent investigators should externally validate prediction models (23). We have independently validated the OMS, and while the investigators who developed Predict_FIRST were involved in external validation, 2 independent investigators were also involved. We recognize in developing a new tool, external validation of Predict_CM2 is required (24). Any comparisons with other predictors are likely to favour the Predict_CM2 and should be interpreted with caution. We recognize that our setting had a relatively high proportion of patients with fractures/orthopaedic conditions and a low proportion of patients with neurological conditions compared with other rehabilitation settings. Further validation of the tool is recommended in rehabilitation settings with a higher neurological case mix.

As we were unable to collect data regarding the number of past falls or fall-related injuries, we could not explore comparisons between single and multiple fallers or injurious and non-injurious falls. Other limitations include the relatively small sample size and use of a single site. Some suggest that external validation studies should involve a minimum of 100 events and 100 non-events (25). A sample of this size was not feasible for the present study, given the limited resources available.

The use of multi-modal prospective approaches to capturing falls in hospitals is recommended due to the generally poorer accuracy of single or retrospective approaches (26, 27). The use of multi-modal prospective methods to report falls is a strength of the present study. The systematic approach used to select items for the tool overcomes the risk of spurious findings in small samples; inclusion of participants with cognitive impairment improves generalizability; and the availability of complete data for all 300 participants minimizes the risk of bias associated with incomplete follow-up.

In conclusion, Predict_CM2 was significantly better than Predict_FIRST and a single question on past falls in predicting falls in rehabilitation. Predict_CM2 was better, but not statistically different from OMS (Total_Score) or PT_Risk at predicting falls in rehabilitation. Predict_CM2 is a simplified screening tool, providing a quick way to quantify the probability of a patient falling during rehabilitation. After external validation, Predict_CM2 could be used routinely in an inpatient rehabilitation setting.

This study was supported by an infrastructure grant from the Ingham Institute for Applied Medical Research. AT, SL and CS are supported by Fellowships from the Australian National Health and Medical Research Council.