Catherine Sherrington, PhD1,2,3, Stephen R. Lord, DSc2, Jacqueline C. T. Close, MD2,4,
Elizabeth Barraclough, BAppSc1,2, Morag Taylor, BAppSc2,4, Sandra O’Rourke, BMedSc (Hons)1,2, Susan Kurrle, PhD5,6, Anne Tiedemann, PhD1,2, Robert G. Cumming, PhD3 and
Robert D. Herbert, PhD1

From the 1The George Institute for International Health, University of Sydney, 2Prince of Wales Medical Research Institute,
University of New South Wales, 3School of Public Health, University of Sydney, 4Prince of Wales Hospital Clinical School,
University of New South Wales, 5Hornsby Ku-ring-gai Hospital and 6Faculty of Medicine, University of Sydney, Sydney, Australia

OBJECTIVE: To develop and internally validate a simple falls prediction tool for rehabilitation settings.

DESIGN: Prospective cohort study.

PARTICIPANTS: A total of 533 inpatients.

METHODS: Possible predictors of falls were collected from medical records, interview and physical assessment. Falls during inpatient stays were monitored.

RESULTS: Fourteen percent of participants fell. A multivariate model to predict falls included: male gender (odds ratio (OR) 2.70, 95% confidence interval (CI) 1.57–4.64), central nervous system medications (OR 2.50, 95% CI 1.47–4.25), a fall in the previous 12 months (OR 2.21, 95% CI 1.07–4.56), frequent toileting (OR 2.14, 95% CI 1.27–3.62) and tandem stance inability (OR 2.00, 95% CI 1.11–3.59). The area under the curve for this model was 0.74 (95% CI 0.68–0.80). The Predict_FIRST tool is a unit weighted adaptation of this model (i.e. 1 point allocated for each predictor) and its area under the curve was 0.73 (95% CI 0.68–0.79). Predicted and actual falls risks corresponded closely.

CONCLUSION: This tool provides a simple way to quantify the probability with which an individual patient will fall during a rehabilitation stay.

Key words: accidental falls; aged; clinical prediction; rehabilitation.

J Rehabil Med 2010; 42: 482–488

Correspondence address: C. Sherrington, The George Institute for International Health, PO Box M201, Missenden Road Sydney NSW 2050, Australia. E-mail: csherrington@george.org.au

Submitted June 28, 2009; accepted February 1, 2010

INTRODUCTION

Falls present an important challenge to those delivering clinical services for older people and for those charged with producing meaningful healthcare policy. The increasing proportion of older people in the population means that falls will have an increasing impact on health services in years to come. A fall occurring in hospital is an adverse event that can, in some cases, result in prolongation of the hospital admission, severe injury or death. A recent report (1) from the UK calculated that in an average 800-bed acute hospital there would be approximately 24 falls every week or 1260 falls every year. Associated healthcare costs were estimated to be a total of £92,000 per year for the average acute hospital. Falls tend to be more frequent in aged care and rehabilitation settings where patients often have transfer and mobility problems; risk factors consistently found to be important predictors of falls (2). In a recent study, it was reported that 11% of older people fell during an inpatient rehabilitation stay (3).

A systematic approach to care of older people when in the hospital is thought to be necessary for prevention of falls during hospital stays. This is likely to be achieved through a combination of policy and clinical guidelines. Few validated fall risk assessment tools have been developed for inpatient rehabilitation settings. The St Thomas Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) (4) was developed and validated in both acute aged care and rehabilitation settings (4). In consequence, this tool may not be ideal for use in rehabilitation settings, as predictors of falls in rehabilitation settings may be different from predictors in acute aged care settings. To overcome this limitation, a second tool, the Peter James Centre Falls Risk Assessment Tool (PJC-FRAT), has been developed to guide the provision of interventions in rehabilitation settings (5). Although this tool can classify rehabilitation inpatients as fallers and non-fallers with reasonable accuracy, it relies primarily on clinical judgement of benefit from receipt of interventions, making its generalizability uncertain.

Existing in-hospital fall risk assessment tools classify people as being at “high” or “low” risk. It is often assumed that assessment of risk of falling for hospital inpatients using these tools is a useful activity. This assumption has been challenged recently (6) due to: the focus on the classification of individuals as “high” or “low” risk rather than looking at absolute probabilities of falling; the time required to complete these assessments; the lack of demonstrated “added value” compared with clinical judgement; fluctuating risk factor status; the lack of action to address risk factors identified, and uncertainty about whether such a tool is needed to implement an intervention strategy.

We sought to develop a tool for use in rehabilitation inpatient settings that addresses several of these concerns, particularly the dichotomization of people into “high” and “low” risk categories. An important clinical question is: How likely is it that a patient will fall? (7). Tools that can classify individuals according to their level of absolute risk of falling could be more valuable in guiding care than tools that simply classify people as high or low risk. For example, differing amounts of assistance or supervision during the inpatient stay could be provided according to the identified probability of falling. People at very high risk (e.g. more than 50% probability of falling during the inpatient stay) may need constant supervision, those at intermediate levels of risk (e.g. 25% probability of falling during the rehabilitation stay) may need intermittent supervision, and those at low risk (e.g. 10% probability of falling during the rehabilitation stay) may be able to be safely left unattended for longer periods of time. By understanding the level of risk for an individual, staff could tailor supervision strategies depending on local availability of resources. We suggest that such a tool would be sufficiently useful to justify the time taken to use it, if it contained easy to assess items shown to be associated with falls in a rehabilitation inpatient population and had a demonstrated link between predicted and actual probability of falling.

In this paper we describe the development and testing of an easily applied, simple falls prediction tool for rehabilitation settings (Prediction of Falls In Rehabilitation Settings Tool, Predict_FIRST), which provides an individualized estimation of the risk of falls based on the presence of risk factors. We hypothesized that the predictive ability of this new tool would be comparable to, or better than, existing risk assessment tools, and we tested this hypothesis.

METHODS

Design and recruitment

To develop the prediction tool, we conducted a prospective inception cohort study in which baseline data were collected from consecutive consenting new admissions to rehabilitation wards at 2 metropolitan public hospitals in Sydney, Australia. There were two participating wards at Hospital 1 and one participating ward at Hospital 2. Falls during the admission were recorded. Recruitment was carried out between August 2005 and April 2007.

All people aged 50 years and older admitted to the participating wards during this time period were considered for inclusion in the study unless they did not speak conversational English and an interpreter was not available, or they were deemed medically unable to safely complete the assessments (by study staff in consultation with treating medical staff).

Informed consent was sought directly from all eligible patients with a Mini-Mental State Examination (MMSE) score of 24/30 or more. For those with lower scores, consent was sought from the patient and the person responsible (usually a family member). The study was approved by Human Research Ethics Committees at the University of Sydney and the 2 participating hospitals.

Predictor variables

Data were collected by physiotherapists from a number of sources, including medical records, interviews with staff and participants and from physical assessments within the first 48 h of admission to the ward. Medical (medical conditions) and sociodemographic (age, gender, living arrangements) data were recorded. Medications were transcribed directly from the ward prescription chart. The Functional Independence Measure (FIMTM) score and the presence of documented postural hypotension were extracted from routine nursing documentation, and nursing staff were asked directly to assess the risk of falls using the STRATIFY (4) score and to estimate each individual’s number of day-time and night-time toilet visits and episodes of incontinence since admission. Participants were interviewed about medication conditions, reported dizziness, pain and daily tasks abilities in the 3 months prior to hospitalization.

The physical assessment took around 30 min to complete and included performance on individual items from the Short Physical Performance Battery (8), items selected from the Physiological Profile Assessment (9) and QuickScreen tool (10), and other tests of balance and mobility previously used in this population. The order of test administration was altered to suit individual patients and their location within the ward environment. Rest breaks were given as required. Tests included: visual impairment (low contrast visual acuity chart (9)), peripheral sensation (using a series of microfilaments (10)) and seated isometric knee extensor muscle strength (spring balance (9), peak force in kg, best of 3 attempts). Standing balance was assessed by recording the time that each of 5 positions could be held without assistance or arm support (feet apart, feet together, semi-tandem stance, tandem stance and single leg stance; maximum of 10 sec for each position recorded separately (8)). Sit to stand ability was assessed by recording the time to complete 5 stands from a 45 cm chair and coding the level of assistance from another person and arm support needed (on a 4-point scale (8), 1 trial only). Stepping ability was assessed using the Hill step test (11) (the number of steps onto 7 and 15 cm blocks in 15 s) and by using the alternate step item from the Berg balance scale, which involves alternate placing of the feet onto a 15 cm block. Gait was assessed using the Timed Up and Go Test, which tests the ability to stand up, walk 3 m, turn around, return and sit down again. The number of steps required to complete the turn was also counted. The time to walk 4 m was measured and the number of steps taken during the walk was also counted. At the completion of the assessment delirium was assessed using the criteria within the Confusion Assessment Method (12).

Falls monitoring

A fall was defined as unintentionally coming to rest on the ground or other lower surface without overwhelming external force or a major internal event (13). Falls data were primarily extracted from hospital incident reports. Supplementary checks of medical records and regular verbal communication with nursing staff were also undertaken by the study physiotherapists. Falls reported by staff and/or recorded in the clinical record but not entered in the hospital incident reporting system were included in the analysis.

Statistical analysis

Analyses were conducted using the SPSS and Stata statistical software packages. Missing data for continuous predictor variables were imputed using multiple regression with the “Missing Value Analysis (MVA)” routine in SPSS.

We developed, tested and compared 2 multivariate models and 2 clinical prediction tools. The associations between potential predictor variables and falls were first assessed using univariate logistic regression. Variables for which the individual p-values were less than 0.05 and the odds ratios were greater than 2.0 or less than 0.5 when dichotomized at the median were identified as candidate predictor variables for multivariate logistic regression models. This method has been used previously in falls research (14) and is consistent with the recommendation that no more than 10 candidate predictor variables per outcome event be included in multivariate models (15). Candidate predictor variables were then grouped into 6 domains: sociodemographic, medication, falls history, continence/toileting, cognition/communication and balance/mobility. Allocation of variables to domains was done jointly by 3 of the authors (CS, SRL and JCTC). A bootstrapped backward selection procedure (16) (the Stata user-written “swboot” command) was used to identify one variable from each domain for inclusion in the full multivariate model. The variable from each domain that appeared in the greatest number of bootstrapped samples was chosen for inclusion in the full model. We then developed a brief multivariate model, which incorporated variables that were more easily tested in clinical practice. We used bootstrap-adjusted coefficients to increase likely generalizability (17). We then created 2 clinical prediction tools from the brief multivariate prediction model. For the first clinical prediction tool, bootstrap-adjusted coefficients (18) were rounded to integer weights (7). For the second clinical prediction tool we assigned unit weights (i.e. the total prediction score was the number of risk factors present).

We then assessed discrimination and calibration. Discrimination (the ability of a model to distinguish high-risk participants from low-risk participants) was quantified using the area under the receiver-operating characteristic (ROC) curve (AUC) (15). AUCs for different models were compared using the “roccomp” command in Stata. To ascertain the likely performance of our models in another sample (15), estimates of the optimism of AUCs were obtained by averaging the differences between AUCs of the original data-set and each of 1000 bootstrapped samples (19). Calibration (the extent to which predicted probabilities agree with observed probabilities) (15) of the simpler tool was then tested with the Hosmer-Lemeshow statistic. A p-value of < 0.05 was interpreted as indicating that the model did not fit the data. We also calculated multilevel (stratum-specific) likelihood ratios.

RESULTS

Participant numbers and characteristics

During the study period 1227 people were admitted to the participating wards. A total of 533 people were eligible for the study and they (or a proxy) gave consent to participate in the study. Reasons for exclusion of the remaining 694 people were: study participation during a previous admission (n = 55), left the ward before assessment could be undertaken (n = 67), cognitive impairment and no person responsible available to give proxy consent (n = 106), refused to participate (n = 195), too medically unwell or otherwise unable to carry out the assessment (n = 128), and not able to be assessed due to staff leave (n = 143). Medical record data were available for all 533 participants. The interview and physical assessment was conducted with 517 participants because 16 participants could not be assessed during the first 48 h after admission due to staff or participant unavailability. Data on falls during the hospital admission were available for all 533 participants.

The average age of participants was 82 years (standard deviation (SD) 8.3) and 377 (71%) were women. Seventy-four participants (14%) were living in residential aged care facilities prior to being admitted to hospital. The mean length of stay on the rehabilitation wards was 25 days (SD = 15). A total of 124 people (23%) had a primary diagnosis of any fracture, 37 (7%) had a primary diagnosis of a neurological condition and 57 (11%) people had a primary diagnosis of fall or syncope.

Predictors of falls during rehabilitation ward stays

There were 110 falls in 75 people (14% of participants). Nineteen people (4%) fell two or more times and 9 people (2%) fell three or more times.

Many of the potential predictor variables were significantly associated with falls. Table I lists variables associated with a marked increase in risk of falls (OR ≥ 2.0, p < 0.05). These variables included male gender, prescription of medications targeting the central nervous system (CNS), past falls, and variables assessing difficulties with continence/toileting, cognition/communication and balance/mobility.

The variables included in the highest proportion of multivariate models on the bootstrapped samples were: male gender (included in 99% of models), use of CNS medications (98%), a fall in the past 12 months (83%), the frequent toileting item from the STRATIFY scale (87%), the FIMTM communication score (67%), and a lower total standing balance time (94%). The area under the ROC curve for this combination of variables on the original data-set was 0.76 (95% CI 0.71–0.82). The bootstrap-adjusted AUC was also 0.76. Removal of the cognition/communication domain did not significantly alter the predictive ability (discrimination) of the model (p = 0.29) and each of the remaining variables remained in more than 80% of models on the bootstrapped samples (male gender 99%, use of CNS medications 99%, fall in the past 12 months 82%, frequent toileting 89%, standing balance 96%). The AUC for this model was 0.76 (95% CI 0.70–0.81) and the bootstrap-adjusted AUC was 0.75.

As the full standing balance item would be relatively time-consuming to assess, a model with this item dichotomized to able/unable to perform a tandem stance (one foot directly in front of the other) was assessed. This model had an AUC of 0.74 (95% CI 0.68–0.80) and a bootstrap-adjusted AUC of 0.73. The difference between the discriminations of the full model (including the cognitive domain) and the brief model (without the cognitive domain and with standing balance dichotomized) was not statistically significant (p = 0.12). We believe the benefit of an increased ease of administration outweighs the small, statistically non-significant loss of discrimination, so we based further development of the clinical prediction tool on the brief model. The relative importance of variables in these two models is shown in Table II.

Development of the clinical prediction tool: Predict_FIRST

The AUC for the clinical prediction tool developed using integer weights based on the bootstrap-adjusted regression coefficients shown in Table II was 0.74 (95% CI 0.68–0.80). The simpler tool in which the risk factors are equally (unit-) weighted had an AUC of 0.73 (95% CI of 0.68–0.79). As the difference between the discrimination of the integer-weighted and unit-weighted models was small and not statistically significant (p = 0.28), we proceeded with the simpler unit-weighted model. The bootstrap-adjusted AUCs for the integer-weighted and unit-weighted versions of the tool were also 0.74 and 0.73.

The resulting tool (Predict_FIRST) is shown in Appendix I. It can be used to estimate the probability with which an individual, with a particular number of risk factors, will fall during a period of hospitalization. As Table III and Appendix I show, a person with no risk factors has a 2% probability of falling in hospital, while a person with 5 risk factors has a 52% probability. For example, the person without risk factors would be a woman, not prescribed any CNS medications, with no falls in the past year, who did not need frequent visits to the toilet and could perform a tandem stand. In contrast, an example of the person with 5 risk factors would be a man prescribed CNS medications, who needed frequent visits to the toilet, had fallen in the last year and could not perform a tandem stand.

Table III shows the proportion of people predicted to fall based on their risk factor profile and the proportion of people with each risk factor score who actually fell during their inpatient rehabilitation stay. The Hosmer-Lemeshow test did not detect a lack of fit between predicted and observed fallers (p = 0.61).

Comparison with other tools

As shown in Table IV, the AUC for the Predict_FIRST is significantly better than the AUC for the STRATIFY score, the total FIMTM score, Short Physical Performance Battery (SPPB) score and the Timed Up and Go Test in our study population.

DISCUSSION

In this study, we developed a fall prediction tool for use in rehabilitation settings using data collected from a brief assessment from a large prospective inception cohort study. Findings from prospective cohort studies have previously been used to develop clinical tools to predict an individual’s probability of a broad range of health events, including hip fracture (20), severe post-operative pain (21), health outcomes in people with chronic obstructive pulmonary disease (22) and the prognosis of shoulder pain (23). Similar approaches have been used in falls studies in assisted care (24) and general community settings (14, 25), but not in inpatient settings.

The resultant tool included 5 easily assessable items: male gender, use of medication targeting the CNS, past falls, need for frequent toileting and a simple balance measure. Predict_FIRST was better able to discriminate high risk from low risk participants than other tools that have been used to identify people as being at risk of falling, including STRATIFY (4) and the Timed Up and Go Test. It also had better discrimination than tests of functional abilities (FIMTM) or physical performance (SPPB (8)). Furthermore, while designed primarily to identify people at increased risk of falls, the tool also provides important information about fall risk factors, which may guide in falls prevention strategies including review of CNS medications use, appropriate placement of patients on the ward who need to toilet frequently, adoption of a toileting plan and a focus on balance retraining as part of rehabilitation intervention.

Some of the predictors identified have also been found to be predictive of falls in other inpatient settings (i.e. use of medication targeting the CNS, past falls, the need for frequent toileting (26–28)). However, our findings highlight the need for falls prediction tools to be designed and tested in the settings for which they are to be used, as some of the independent predictors identified here differed from those found in community and acute hospital aged care settings. In community dwellers, tests of strength, balance and mobility have been consistently found to be important predictor of falls (2). We found that in the rehabilitation setting, other risk factors were of equal or greater importance, a finding consistent with that of Haines et al. (29) who reported a poor predictive ability of balance tests alone in rehabilitation inpatients. Furthermore, while several measures of cognition and communication were associated with falls in univariate analyses, these measures proved to be either less important or too closely associated with the final set of variables included in the multivariate models, and subsequently did not add predictive value. It is also likely that people accepted for inpatient rehabilitation have better cognition (median MMSE score in current sample = 28) than the broader population of older people in acute aged care wards. Community studies (30) have found women to be more likely to fall than men, whereas we found male gender to be a predictor of falls. A recent national study (31) of falls in English and Welsh hospitals also found falls to be more common in male inpatients. It is possible that this is due to behavioural differences between the sexes, although this requires further investigation.

The strengths of this study include the relatively large sample size, the analytical approach and the systematic way in which we developed the tool. Most importantly, the analysis provided prediction scores in terms of absolute risk for individuals on a categorical scale. This approach has significant advantages over the calculation of sensitivity and specificity, which requires dichotomization of the prediction variable (i.e. people have to be classified as predicted fallers or predicted non-fallers). The use of internal validation (bootstrapping) procedures also provides support for the tool’s generalizability. However, we recognize that the ultimate test of generalizability involves external validation (32) and that the Predict_FIRST will need to be evaluated in other rehabilitation ward settings. It is possible that the absolute probability of falling for individuals with particular risk profiles would be different in settings with very different fall rates.

The fact that more than half of the people admitted to the participating wards during the study period did not participate in our study requires comment. Non-participation occurred for a range of reasons, most of which would not substantially distort the predictive ability of the model. However, the exclusion of people who were too medically unwell or otherwise unable to complete the assessment would suggest that the tool has limited applicability to this patient group. However, most of these patients were unlikely to be undergoing active rehabilitation at the time when they were too unwell to participate in the study and are likely to have been awaiting transfer or treatment elsewhere. We acknowledge that our exclusion of people with cognitive impairment for whom no person responsible was available to give consent may have limited our ability definitively to establish the strength of cognitive impairment as a predictor of falls in rehabilitation settings.

The study may also have benefited from the inclusion of a clinical judgement of fall risk, as previous studies in residential aged care (33) and hospital settings (34) have found that clinical judgement can be a comparable (35) or better predictor of falls than formal assessment tools (33, 34). We suggest that experienced clinical staff may well be able to predict which patients are more likely to fall, without undertaking a formal assessment of risk factors, but would be less able to quantify the risk without using a tool such as ours. That is, experienced clinicians may use discriminative but not well-calibrated models. In addition, an explicit prediction tool such as ours would be of particular use to less experienced staff.

In conclusion, the Predict_FIRST tool provides a simple way to quantify the probability with which an individual patient will fall during a rehabilitation stay. We feel that this tool could be used routinely in aged rehabilitation centres and hope that this will contribute to the development of more effective inpatient falls prevention strategies.

ACKNOWLEDGEMENTS

This study was funded by the NSW Health Department. C. Sherrington, S. R. Lord and R. D. Herbert are funded by Australian National Health and Medical Research Council Fellowships. The funders had no role in the study design or execution or in preparation of the manuscript.

REFERENCES