Objective: The Wheelchair Mobility Performance (WMP) test is a reliable and valid measure to assess mobility performance in wheelchair basketball. The aim of this study was to examine the sensitivity to change of the WMP test by manipulating wheelchair configurations.

Methods: Sixteen wheelchair basketball players performed the WMP test 3 times in their own wheelchair: (i) without adjustments (“control condition”); (ii) with 10 kg additional mass (“weighted condition”); and (iii) with 50% reduced tyre pressure (“tyre condition”). The outcome measure was time (s). If paired t-tests were significant (p < 0.05) and differences between conditions were larger than the standard error of measurement, the effect sizes (ES) were used to evaluate the sensitivity to change. ES values ≥0.2 were regarded as sensitive to change.

Results: The overall performance times for the manipulations were significantly higher than the control condition, with mean differences of 4.40 s (weight – control, ES = 0.44) and 2.81 s (tyre – control, ES = 0.27). The overall performance time on the WMP test was judged as sensitive to change. For 8 of the 15 separate tasks on the WMP test, the tasks were judged as sensitive to change for at least one of the manipulations.

Conclusion: The WMP test can detect change in mobility performance when wheelchair configurations are manipulated.

Accepted Mar 7, 2018; Epub ahead of print May 8, 2018

J Rehabil Med 2018; 50: 00–00

Correspondence address: Annemarie M. H. de Witte, Faculty of Health, Nutrition & Sport, The Hague University of Applied Sciences, Meester P. Droogleever Fortuynweg 22, NL-2533 SR Den Haag, The Netherlands. E-mail: a.m.h.dewitte@hhs.nl

In this study we measured the performance times on the Wheelchair Mobility Performance (WMP) test during different test conditions to see if the performance times changed when wheelchair settings were changed. The overall performance time on the WMP test increased when the tire pressure was reduced and also when extra mass was attached to the wheelchair. It can be concluded that the WMP test is sensitive to changes in wheelchair settings. It is recommended to use this field-based test in further research to investigate the effect of wheelchair settings on mobility performance time.

In wheelchair basketball, the interaction between the player, the wheelchair and the environment determines overall performance. More specifically, and in agreement with several other studies (1, 2), all actions a wheelchair basketball player can perform using the wheelchair, such as turning, blocking, stopping and accelerating, are considered to be part of mobility performance. In order to repeatedly monitor athletes’ mobility performance, standardized field-based tests are informative and helpful (3, 4). Recently, de Witte et al. (5) developed a standardized field-based Wheelchair Mobility Performance (WMP) test to assess mobility performance in wheelchair basketball. Extensive analyses of matches with elite wheelchair basketball athletes were performed in order to determine the most common wheelchair handling activities and their characteristics (1, 5, 6). These characteristics were combined in a test-circuit consisting of 15 specific wheelchair basketball mobility performance tasks (Appendix SI¹). The WMP test covers the full range of relevant mobility performance tasks in wheelchair basketball, meaning that all aspects of an athlete’s mobility performance can be assessed in a single standardized test.

The reliability and construct validity of the WMP test has been determined previously (5). The reliability of the WMP test for overall performance outcome was found to be excellent (intraclass correlation coefficient (ICC) = 0.95) (5). Furthermore, the construct validity of the WMP test was confirmed by showing that it can detect differences in mobility performance between athletes who were expected to differ in terms of their level of physical capacity (7, 8). In line with expectations, men performed better than women and international male athletes performed better than national male athletes on the WMP test. A borderline significant difference in mobility performance was found between low classification (1.0–2.5) and high classification (3.0–4.5) athletes. It was concluded that the WMP test was reliable and valid and could be used to assess the capacity of mobility performance of elite wheelchair basketball players.

In addition to being reliable and valid, the WMP test should also be sensitive to change in order to apply the test in sports practice and research (9). Sensitivity to change can be defined as the ability of a test to detect change in its outcome when it has occurred (10–12). In elite sports, differences in performance are very small and, therefore, it is important to be able to detect changes in the determinants of performance (13). If the WMP test is sensitive to change, the change or difference in performance time assessed using the test can be truly attributed to a systematic change in mobility performance in-person and not to noise or random error. The psychometric characteristic sensitivity to change of the WMP test can be studied by such manipulation of the mobility performance, for which it can be expected that the WMP test is able to detect the change in mobility performance. Potential manipulations that can be studied to explore the sensitivity to change of the WMP test are: configuration of the wheelchair (e.g. wheel diameter, mass), characteristics of the athlete (e.g. body weight), or manipulations in the interface between wheelchair and athlete (e.g. seat height). If the WMP test is able to detect a change in performance time when the wheelchair, athlete or interface configurations are manipulated, it is justified to use the test in practice and in research. The test can be used, for instance, to optimize the design of the wheelchair in wheelchair basketball. Therefore, the objective of the present study was to examine the sensitivity to change of the standardized field-based WMP test in wheelchair basketball by systematically manipulating the wheelchair configuration.

Sixteen wheelchair basketball players (15 men, 1 woman) with a mean age of 23.5 years (standard deviation (SD) 8.4), a mean body weight of 71.1 kg (SD 21.4) and 7.8 years (SD 6.6) of experience in wheelchair basketball volunteered to participate in this study. All participants trained at least twice a week and played in the B- or C-division of the Dutch wheelchair basketball competition. An overview of their classification is shown in Fig. 1. Prior to participation, all participants were informed about the study objectives and procedures, and signed an informed consent form. The study was approved by the ethics committee of the Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam (VCWE 2016-091).

The WMP test consists of 15 sport-specific tasks based on extensive observation of wheelchair basketball matches (5) (see Appendix SI for a description of the test). The test-retest reliability of the WMP test was excellent (ICC = 0.95) for the overall performance time and the WMP test is a valid tool to assess mobility performance in wheelchair basketball players (5).

The participants performed the WMP test 3 times in their own wheelchair: (i) in the “control condition” (CC) participants performed the test with normal tyre pressure (standardized at 7 bar) and with no extra mass attached to the wheelchair; (ii) in the “weighted condition” (WC) participants performed the test with normal tyre pressure, but with an additional mass of 10 kg attached to the wheelchair (distributed over the wheelchair frame using 5 masses of 2 kg (Fig. 2); (iii) in the “tyre condition” (TC) participants performed the test in their own wheelchair with the tyre pressure reduced by 50% (3.5 bar) and with no additional mass attached to the wheelchair. Tyre pressure was determined using a high-pressure pump (Lezyne Alloy Drive SE Floor Pump).

Prior to the WMP tests, verbal instructions were given to the participants about the test procedure, and participants had to practice the WMP test tasks in the presence of a researcher who gave verbal instructions for each task. After being given the instructions, the participants completed a form concerning general information: age, body weight, type of impairment, years of experience in wheelchair basketball, and classification. After a self-selected warm-up, the participants performed the 3 experimental conditions of the WMP tests in a randomized and counterbalanced order to avoid learning effects. All standardized tasks of the WMP test were carried out in succession in a fixed order, separated by standardized rest periods, as described in the test protocol (5). The WMP tests were performed indoors on a synthetic soft-top basketball court on 1 day. Each WMP test took approximately 6.5 min and was followed by a rest period of 10–15 min.

All WMP tests were video-recorded from the side of the court with a Casio Exilium EX-ZR1000 camera (Casio, Tokyo, Japan) or a Samsung Galaxy S5 (Samsung, Seoul, South Korea), both at 30 frames/s. The outcome of the WMP test was time (s), which was manually assessed from the video analyses using Kinovea (Kinovea 0.8.15, available for download at: http://www.kinovea.org). These analyses resulted in 16 performance time values, one for each of the 15 tasks of the WMP test (time tasks numbers 1–15) and the overall performance time, which was the sum of the performance times of the 15 separate tasks. The measurement time was accurate to 0.03 s (30 Hz).

The normality of the data was checked with the Shapiro–Wilk test, the Z-values for kurtosis and skewness, Q-Q plots and box plots. For all performance time data, the assumption of normality was not violated. Descriptive statistics for performance measurements (time WMP test tasks numbers 1–15 and overall performance time) were presented as mean (SD). In line with previous research (11, 12, 14, 15), sensitivity to change of the measurements was examined using paired t-tests, the standard error of measurement for agreement (SEM_agreement) and Cohen’s d effect sizes (ES).

Paired t-tests were used to examine the differences in performance time between WC and CC, and between TC and CC. All data were analysed with IBM SPSS Statistics 23 (IBM Corporation, Armonk, NY, USA) using a significance level of p < 0.05.

The SEM for agreement was calculated using equation [1], below. This analysis has been performed previously and published in a study of the reliability and validity of the WMP test (5). Variance component analyses were used to estimate variance attributable to observers (Var_observer) and residual error (Var_residual), with the square root of their summation resulting in SEM_agreement.

Cohen’s d effect size (ES) was calculated to assess the meaningfulness of the different test conditions (see equations 2 and 3). For calculation of ES, the SD of the 2 testing conditions to be compared were converted into 1 pooled SD (SD_pooled).

where SD₁= SD of the control condition, and SD₂= SD of the weight or tyre pressure condition

where Mean₁= Mean of the control condition, and Mean₂= Mean of the weight or tyre pressure condition.

For assessment of sensitivity to change of the WMP test, a significant difference in performance time must be detected between the manipulation conditions (WC and TC) and CC. Furthermore, the observed differences between both conditions must be larger than the SEM_agreement. If the results met both requirements, the ES was used to evaluate the magnitude of the differences between the manipulated and control conditions. Cohen’s d cut-off points for ES values were: trivial (d < 0.2), small (0.2 ≤ d < 0.5), moderate (0.5 ≤ d < 0.8) and large (d ≥ 0.8) (16). In our case, the WMP test was judged not sensitive to change for ES values lower than 0.2, while values equal to or higher than 0.2 were judged as sensitive to change.

The mean overall performance time on the WMP test for the CC was 101.59 (SD 9.63) s, for the WC 105.99 (SD 10.52) s and, for the TC, 104.39 (SD 11.03) s (Table I). The overall performance time for the WC and TC was significantly higher than for the CC (p < 0.05). The observed overall differences between the manipulated and control conditions (∆WC–CC = 4.40 ± 2.05 s, ∆TC–CC = 2.81 ± 2.25 s) were larger than the reported standard error of measurement (SEM) (> 0.98). The ES for the WC–CC was 0.44 and for the TC–CC 0.38. Therefore, the overall performance time was judged as sensitive to change (ES ≥ 0.20). The individual differences in the overall performance times between the different conditions per wheelchair basketball player are shown in Fig. 3.

For the performance times of the separate tasks, only the 3-3-6-m sprint, for both WC and TC condition, was sensitive to change (ES: WC–CC = 0.31, TC–CC = 0.37). In the WC, performance times for 7 out of the 15 WMP tests tasks were significantly different from those in the CC, while at the same time the differences were larger than the SEM. The tasks 180° turn on the spot left (ES = 0.43), 12-m sprint (ES = 0.51), 3-3-6-m sprint (ES = 0.31), 3-3-6-m rotation to the right (ES = 0.50), 12-m-dribble (ES = 0.35), 12-m-rotation dribble to the left (ES = 0.34), and the combination task (ES = 0.61) were judged sensitive to change. For the TC, as indicated above, only the performance time on the 3–3–6-m sprint (ES = 0.37) and the overall performance time (ES = 0.27) were significantly different from the CC.

This study determined the sensitivity to change of the standardized field-based WMP test, in order to assess whether the WMP test can detect changes in mobility performance in wheelchair basketball players. The mean total performance times for the 10 kg extra mass condition and the reduced tyre pressure condition, were significantly more than for the control condition. The overall performance time of the WMP test was judged sensitive to change. It can, therefore, be concluded that the WMP test can detect changes in mobility performance when wheelchair configurations are manipulated. The separate tasks of the WMP test showed different levels of sensitivity to change dependent on the manipulation condition. For 8 of the 15 separate tasks on the WMP test, the tasks were judged sensitive to change for at least one of the manipulations.

In the present study sensitivity to change was investigated in order to determine whether the WMP test can detect changes in mobility performance. The term “sensitivity to change” is generally used as a common measure to detect change when it has occurred (11, 12, 17). The cause of the change may vary; for instance, as to the topic of the present study, because of changes in wheelchair configuration, but also because of changes over time. However, the term responsiveness is also used in the literature, specifically when it concerns changes over time in the construct to be measured (9, 18). As the aim of this study was to investigate whether the WMP test is able to detect changes in mobility performance due to changes in wheelchair configuration, we decided to use the term “sensitivity to change”. This does not mean that the WMP test is not sensitive to changes in mobility performance over time. The test was shown to be able to detect manipulated changes in mobility performance, and we expect the test to be able to detect change in mobility performance over time due to training or injury. Change should, however, be beyond the limits of agreement described in the validity and reliability study of the WMP test (5). Furthermore, De Vet et al. (9) state that responsiveness is relevant to measurement instruments used in evaluative applications, and that if an instrument is used only for discrimination between patients at 1 time-point, then responsiveness is not an issue. According to Deyo & Centor (19), responsiveness relates to a true change in clinical (health) status over time. This means that the outcome measure must remain stable when no (clinical) change has occurred (specificity) and it must detect meaningful (clinical) change when it has occurred (sensitivity). However, in the present study, differences in performance times on the WMP test between conditions are assumed to be caused by the manipulations in wheelchair configuration. In order to measure whether change occurred and the magnitude of that change, the current study used sensitivity to change.

Sensitivity to change was examined by manipulating wheelchair configuration, which can have a significant impact on mobility performance (20). Other manipulations could have been chosen to study sensitivity to change. For example, manipulation of the athlete or the wheelchair-athlete interaction could change mobility performance, e.g. by limiting trunk function, the movement of the trunk will be limited and performance may decrease. In this study, a 10 kg extra mass and a 50% reduced tyre pressure were used to examine sensitivity to change. These manipulations were chosen because they were relatively easy to apply to the athlete’s own wheelchair (CC) and they clearly increase the external work required, and thus reduce mobility performance. The magnitude of the manipulations was chosen in agreement with previous studies (21–24). Beekman et al. (21) found that, in a wheelchair that was 7.8 kg lighter, the speed and distance travelled were greater than for a heavier wheelchair. Cowan et al. (23) found that velocity decreased as the weight of the wheelchair increased by with 9.05 kg. Therefore, we used 10 kg additional mass in the weight condition. Booka et al. (22) and de Groot et al. (24) stated that a lower tyre pressure needs more work, even on a hard level surface. To increase the work, the tyre pressure was reduced to 50% in this study. In both manipulated conditions, the power output was increased, while this may not impact the skill of mobility performance because the wheelchair-athlete settings have remained unchanged.

In the weighted condition, all tasks that were not judged sensitive were related to rotational tasks. In this study, the masses (5 × 2 kg) were attached on the outside of the frame (Fig. 2). It could be that the weight distribution had less effect on the performance time for the rotational tasks compared with translation tasks, or that the amount of weight had less effect on rotational tasks. Moreover, the extra mass was 10 kg for all participants, which may mean that the relative weight gain was different between participants. This may have led to an overestimation of the results. If the amount of additional mass is determined relative to the total mass of the athlete and wheelchair, the disadvantage of extra mass is the same for all the athletes. Based on this study, it can be concluded that mass influences performance times, but this does not provide insight into to what extent mass influences performance time. To research that relationship, future studies should examine the effect of relative, not absolute, additional mass.

In the TC, only the performance on the 3–3–6-m sprint and the performance time on the entire WMP test were judged sensitive to change. A recent study by Leboeuf et al. (25) showed that a lower tyre pressure (5 compared with 9 bar) only decreases sprint performance in a straight line and not when other movements are included, such as stops and half-turns. This is in line with the results of the present study. It could be that the differences between the conditions on the separate tasks were too small to appear as sensitive to change, but the sum of the separate tasks was large enough to appear as sensitive to change. Another explanation could be that the tyres deformed during changes in directions and stops. By inflating the tyres as much as possible, the friction between the ground and the tyres reduces, which may result in skidding. Skidding leads to loss of grip and thus waste of time. This can be an explanatory hypothesis for the comparable time between the tyre pressure conditions.

In the present study, outcome measure time (in s) was used, which can be assessed using a timer or, as in the present study, video. Therefore, the test is easy to use in practice to determine changes in performance. However, information about kinematic outcomes, such as (rotational) acceleration, could provide additional information and can be measured with inertial sensors on the wheelchair (6, 26). The use of additional kinematic outcome measures could provide more in-depth information about sensitivity to change. However, specific knowledge and equipment, such as inertial sensors, are required and this is therefore more difficult in practice. For research purposes, it is recommended that additional kinematic outcomes are used to analyse sensitivity to change.

The WMP test was developed to assess the capacity of mobility performance of wheelchair athletes in wheelchair basketball. For research purposes, Mason et al. (20) recommended that a standardized field-based test can be used to examine the impact of different wheelchair configurations on mobility performance. However, the test should be reliable, valid and sensitive to change. A previous study determined the reliability and construct validity of the WMP test (5) and the present study determined sensitivity to change. Together, these 2 studies included 2 analyses concerning sensitivity to change (tyre pressure and weight), a reliability analysis and 3 analyses of construct validity (sex, playing standard, and classification) in order for the WMP test. The authors decided that the reliability must have an ICC ≥ 0.70 (indicated as satisfactory) and that a minimum of 4 of the 5 remaining analyses must meet the requirements in order for the WMP test to be judged as valid and sensitive to change. Based on this requirement, it can be concluded that the WMP test is reliable, valid and sensitive to change for the 3-3-6-m sprint task, the combination task (sprint, turn, slalom, turn), and the overall performance time. If the cut-off was set at the level that all analysis must meet the requirements, than only the 3-3-6-m sprint task and the overall performance time appear to be useful outcome measures. The sensitivity to change of the combination task in the tyre pressure manipulation was borderline significant (p = 0.07). The selected measurement outcomes provide an overview of the mobility performance capacity of a wheelchair basketball athlete. The WMP test is not able to detect change in separate tasks in a reliable, valid and sensitive way. Further research is needed, focussing on the 3 described performance outcomes (3-3-6-m sprint, combination task and overall performance outcome), in order to draw a conclusion on mobility performance capacity.

The WMP test can be used in a reliable and valid way to assess the capacity of mobility performance of elite wheelchair athletes in wheelchair basketball (5). The test can be used in a reliable and valid way to periodically monitor the capacity of the mobility performance of an athlete, to detect the strengths and limitations of an athlete, to detect talented athletes, and to examine whether an athlete is sufficiently recovered from an injury. Furthermore, the selected outcomes are sensitive to change and can be used to assess differences in performance time when wheelchair-athlete configurations are changed. The difference should, however, be larger than the limits of agreement as reported in the reliability and validity study (5) . The test is easy to perform for athletes, little material is required and measuring time (in s) does not require specific knowledge. In addition to the applications mentioned above, the test can be used to optimize the wheelchair-athlete configuration or wheelchair design. The selected parts of the WMP test showed that performance time was sensitive to change when configuration settings were changed, and this can be used in further research. However, as mentioned earlier, performance time is one outcome measure. Kinematic outcomes such as (rotational) acceleration could provide more in-depth information about the effects of configurations on mobility performance.

A limitation of this study was that the test was not blinded. The sequence of test conditions was randomized, but the participants could see or hear the manipulations being applied to their wheelchairs. This may have biased the results, but it is unknown to what extent this affected the test results. Future research must take into account this potential effect. Furthermore, in the weighted condition, for all participants, 10 kg extra mass was attached to the wheelchair. The magnitude of the effect was different for all participants, which may have affected the measurements. It is possible that the results were overestimated because the relative weight gain was not the same for all participants. In future research a relative value should be determined so that the effect is the same for all participants. Another limitation of the WMP test is that not all separate tasks can be used to analyse mobility performance. For example, the single rotational tasks could not be used in assessing mobility performance.

The WMP test can detect changes in mobility performance; for instance, when the wheelchair configuration is manipulated. Taking into account the results of the current study together with those of the reliability and construct validity study, it is recommended that performance on the 3-3-6-m sprint (task 7), the combination (task 15), and the entire WMP test are monitored in future research and practice.

The authors would thank the students Josi Klomp and Chloé Bras (Vrije Universiteit Amsterdam) for their assistance with data collection. This work was supported by the Taskforce for Applied Research (part of the Netherlands Organization for Scientific Research).

The authors have no conflicts of interest to declare.