OBJECTIVE: The Southampton Hand Assessment Procedure (SHAP) evaluates the functionality of normal, injured or prosthetic hands. The aim was to evaluate the learning effects of SHAP tasks and the appropriateness of the time limits applied per task in novice prosthetic users.

METHODS: Right-handed unimpaired volunteers (n = 24, mean age 21.8 years) completed 8 SHAP sessions over 5 consecutive days using a prosthetic simulator. The execution times of SHAP tasks were transformed into 6 prehensile patterns, the functionality profile, and the index of function, a general functionality score. Learning effects in task times were analysed using multilevel analysis.

RESULTS: Learning effects occurred in all SHAP tasks. Tasks, sex, sessions, tasks-sessions interaction, and the first session of the day contributed (p < 0.01) to the execution times. Tasks were performed more slowly by females and more slowly on the first session of the day. In several tasks time limits were exceeded by > 25% of participants in at least the first 3 sessions, which affected the calculation of the functionality profile and index of function scores.

CONCLUSION: The learning effects of SHAP in novice prosthetic users require consideration when conducting a reliability study. SHAP scores in novice prosthetic-hand users are confounded by learning effects and exceeded time limits.

Key words: learning effects; upper-limb prostheses; assessment tool; Southampton Hand Assessment Procedure (SHAP); repeated measurements; multilevel analysis.

J Rehabil Med 2014; 46: 00–00

Correspondence address: Ecaterina Vasluian, Rehabilitation Medicine, University of Groningen, University Medical Centre Groningen, NL-9700 RB Groningen, The Netherlands. E-mail: e.golea.vasluian@umcg.nl

Accepted Feb 25, 2014; Epub ahead of print May 21, 2014

INTRODUCTION

Several instruments for assessing the functionality of normal, injured, or prosthetic hands are currently available (1). The international Upper Limb Prosthetics Outcome Measures group (ULPOM) was established to evaluate the validity and reliability of tests measuring the functionality of upper limb prostheses (1–5). One of the instruments that the ULPOM group advocates is the Southampton Hand Assessment Procedure (SHAP) (1, 2), a standardized instrument to evaluate the functionality of healthy, pathologic, or prosthetic hands (6, 7). The reliability of SHAP has been established only in a healthy population (6). Therefore, the ULPOM group has recommended further research regarding the psychometric properties of SHAP in prosthetic users (1, 8). One important aspect of the psychometric properties that has not been studied in SHAP regards learning effects. Learning effects are attributable to memory carry-over by the person from previous performances, and lead to an increment or decrement in skill level (9). We examined the learning effects of the SHAP after repeated administration.

Why is SHAP generally considered a good test? SHAP data are collected in an objective way by requiring the assessed person to time the performance him/herself. Twelve abstract tasks and 14 activities of daily living (ADL) require the person being assessed to use 6 hand grips (spherical flexion, tripod, power, lateral, tip, and extension) (6). The behaviours in each of the tasks are transformed into a compound score for each of the 6 grips, known as prehensile patterns, which form the functionality profile (FP); these are then combined with an overall score for hand function, the index of function (IOF).

Provided the instrument is reliable, prosthetic training may be adjusted based on the functionality scores. A person who scores low on a grip, for instance, might need extra practice for the respective grip, or persons learning fast and scoring high may stop earlier with the prosthetic training. However, it has yet to be established that improvement in functionality scores follows from actual improvement in functionality and not from the testing effects. Importantly, a time limit of 8 times the norm is applied per task (6, 10), but no study has been conducted to verify whether these time limits are appropriate in novice prosthetic users.

The aim of this study was to evaluate the consistency of SHAP during repetitive testing by documenting the learning effects in novice prosthetic users. We formulated 3 hypotheses:

(A) SHAP is found to be consistent over 3 repeated sessions performed by unimpaired non-prosthetic users (6). Theories of behavioural psychology, on the other hand, suggest that learning effects occur after repeated sessions (11). We hypothesized that when SHAP is applied repeatedly in novice prosthetic users, the separate tasks, the 6 prehensile patterns of FP, and the IOF will demonstrate learning effects.

(B) During learning, changes in behaviour are present at the beginning of each new learning session; a phenomenon known as “warm-up decrement” (11). Thus, if learning occurs in SHAP tasks, we hypothesized that each new measurement day should show an initial decrement in performance.

(C) Based on our clinical experience with assessing functionality in prosthetic users with SHAP, the time limits of more difficult tasks are often exceeded by prosthetic users. We hypothesized that the time limits of the more difficult SHAP tasks would frequently be exceeded by novice prosthetic users.

METHODS

The measurements used in this study were part of the protocol used in another study (12) that investigated the inter-manual transfer effects from the trained to the untrained upper limb in novice prosthetic users. The data from the current study were not used in that study (12).

Participants

Twenty-four volunteers (11 men and 13 women, mean age 21.8 years; standard deviation (SD) 4.8 years), unimpaired and right-handed, were recruited from the local university. All participants signed an informed consent prior to the study. Participants received a gift voucher of EUR10 after completion of the study. Ethical approval was obtained from the Medical Ethical Committee of the University Medical Center Groningen, The Netherlands (number NL35268.042.11).

Materials

Prosthetic simulator. Participants performed SHAP tasks using a prosthetic simulator (OIM Orthopedie, Haren, The Netherlands). The simulator operates as a myoelectric transradial prosthesis (MyoHand VariPlus Speed, Otto Bock, Duderstadt, Germany) and was adapted for use by a normal hand (Appendix I) (13). Myoelectric electrodes were placed on the flexor and extensor muscle bellies of the forearm and secured with a sleeve equipped with Velcro (Velcro USA Inc., Manchester, New Hampshire, USA) straps.

SHAP. The authors of SHAP established a normative benchmark using data from unimpaired young adults (6, 10). Each task score contributes to the calculation of 1 or 2 prehensile patterns (Appendix II) (6, 10). The prehensile patterns and the IOF are calculated relative to normative data using z-values and the Euclidean squared distance (Appendix III), and are rated on a scale of 100; a score of 95 or above is considered to be normal functionality (10). A boundary condition, a time limit of 8 times the norm time is applied per task. Exceeding this limit is deemed minimal function. From the existing literature about calculation of FP and IOF, it was not completely clear to us how the exceeded time limits affected prehensile patterns of FP and the IOF (6, 10, 14). After consulting with an experienced statistician, it seems that, when calculating each prehensile pattern, the exceeded time limit determines rescaling to zero of the z-value (6; p. 99–103). This means that the task has no further contribution to make in the calculation of prehensile patterns of FP and of the IOF (Appendix III, z-scores rescaling).

Procedure

Setting up the simulator. The optimal locations of maximum electrical activity of the flexor and extensor forearm muscles were determined for each participant using Otto Bock’s PAULA system (Otto Bock, Duderstadt, Germany) connected to the MyoBoy (757M11 Myoboy and 13E200 Myo-Bock electrodes, Otto Bock, Duderstadt, Germany) training system. These locations were marked with a permanent marker, thus limiting variability in electrode placement over the ensuing days. Muscular signals captured by PAULA were used to set the sensitivity of the electrodes so that the participant could maintain the threshold of 1.5 V for 2 s. These sensitivity values, determined on the first day, were used throughout all sessions.

Familiarization with the simulator. To become familiar with the simulator and to avoid direct learning of the simulator by the test hand, participants practiced with the opposite hand, using a prosthetic simulator. They randomly performed (3 × 3 tasks) picking up a mug, opening and closing a pen-case zip with the normal hand while holding the pen-case with the simulator, and opening a jar lid with the normal hand while holding the jar with the simulator. To learn to control different levels of grip force, participants practiced (3 × 3 tasks) with 3 deformable objects (a spring of different tension in-between 2 metallic plates). These tasks were not part of SHAP.

After familiarization and before starting the first SHAP session, participants practiced opening and closing the simulator 5 times with the test hand. They were also informed before the first session that positioning of the prosthetic wrist was allowed in order to optimally orient the prosthetic hand for each task.

Measurements. Over a period of 1 week, participants executed SHAP tasks on 5 consecutive days: on the first and fifth days (1 session), and on the second, third, and fourth days (2 sessions). The subjects were randomly assigned to perform SHAP with the dominant (6 males: 6 females) or the non-dominant hand (5 males: 7 females). Exact instructions and demonstrations of the tasks were provided (15). Practicing the tasks before taking the measurement was not allowed. Trying out wrist rotation of the prosthetic hand for optimal performance was permitted for a few seconds. For the actual measurement, the prosthetic hand was repositioned to the neutral position, and after the chronometer was started the wrist could be rotated to the desired wrist position. Participants were instructed to position both arms with their hands resting on the table (the chair was adjusted to allow 90° elbow flexion), to have the prosthetic hand closed before starting the task, and to perform the task as accurately and quickly as possible. They started the chronometer using the prosthetic hand and stopped it in the same way after completion of the task. A pause of 2–3 min was allowed between the 2 consecutive sessions to avoid fatigue. No time limit was applied for the execution of the tasks.

Statistical analysis

Hypothesis A (Learning effects in SHAP tasks, FP, and IOF). Multilevel analysis was used to analyse the learning effects of the tasks and to verify Hypothesis B (Warm-up decrement). In addition, we evaluated the influence of dominancy and sex on the learning effects.

A 3-level model was constructed: participants (level 1), SHAP tasks (level 2), and sessions (level 3). The natural logarithm of the task times resembled a linear evolution. Therefore, a linear modelling of the log-transformed time was used for the analysis. Residuals were checked for a normal distribution.

The logarithm of task time in seconds was the outcome variable. Predictor variables were: tasks (1–26), sessions (1–8), age, sex, dominancy (dominant, non-dominant hand), and “warm-up decrement” effect (first session on a new day). From the explored interactions – sex × session, dominancy × session, warm-up decrement × session, sex × task, dominancy × task, and task × session – only those with a significant increase in model fit (–2log likelihood) were included. The model was estimated for each task using random intercept and fixed regression coefficients.

The level of significance was set at p < 0.05. Calculations were made in SPSS version 20. SHAP software was used to generate the FP prehensile patterns and IOF scores (15).

Hypothesis C (Time limits of SHAP tasks). The time limits were calculated by multiplying 8 times the means of the normative data for each task (6, 10, 14). Furthermore, the tasks that exceeded these time limits were determined; for a specific task, the time limit was subtracted from the participant’s time on the same task. For each prehensile pattern, the number of participants who exceeded the time limits in 1 or several tasks contributing to the corresponding prehensile pattern was counted.

RESULTS

Hypothesis A (Learning effects in SHAP tasks, FP and IOF)

SHAP tasks. Multilevel analysis demonstrated a substantial learning effect in all SHAP tasks during the 8 measurement sessions. Tasks, sex, sessions, task-session interaction, and “warm-up decrement” significantly predicted the natural logarithm of the time needed to perform the tasks (p < 0.01 for all variables, Table I). Significant task-session interaction was found in heavy extension, moving an empty tin can and opening/closing a zip. More difficult ADL tasks such as picking up coins, undoing buttons, food cutting, opening/closing a zip, and rotating a screw 90° varied greatly among the participants (Fig. 1), and required longer execution times compared with a simpler task, in our case rotating a door handle (Table I).

Fig. 1. Variation in participants’ performance per Southampton Hand Assessment Procedure (SHAP) task. LSphere: light sphere; LTripod: light tripod; LPower: light power; LLateral: light lateral; LTip: light tip; LExtens: light extension; HSphere: heavy sphere; HTripod: heavy tripod; HPower: heavy power; HLateral: heavy lateral; HTip: heavy tip; HExtens: heavy extension; Coins: picking up coins; Buttons: undoing buttons; Cutting: food cutting; Page: page turning; Jar Lid: removing jar lid; Pour Jug: pouring water from jug; Pour Cart: pouring water from carton; Move Jar: moving a full jar; Move Tin: moving an empty tin can; Tray: moving a tray; Key: rotating a key 90°; Zip: opening/closing a zip; Screw: rotating a screw 90°; Handle: rotating a door handle.

IOF and FP. The mean scores of IOF and of the prehensile patterns of FP improved over the 8 sessions (Fig. 2), but the improvement differed between prehensile patterns. In other words, some prehensile patterns improved gradually (e.g. Spherical), whereas others showed increments and decrements over learning (e.g. Tip, Power). Participants scored highest in the Spherical prehensile pattern during the measurement period and lowest in the Tip prehensile pattern (in the first 2 sessions meanTip= 21 and 25, respectively). The mean scores of IOF improved from a score of 40 in the first session to 70 in the last session.

Fig. 2. Learning effects of the prehensile patterns of functionality profile (FP) and the index of function (IOF). The Southampton Hand Assessment Procedure (SHAP) scores for prehensile patterns of FP and IOF on the y-axis represent the mean of participants’ scores for prehensile patterns and IOF per session.

Hypothesis B (Warm-up decrement)

Every first session of the day (sessions 1, 2, 4, 6 and 8), participants were slower than expected from the linear trend (Table I and Fig. 3). This indicates that the learning curves evolved in steps, which follows from the saw-tooth shape of the lines in Fig. 3.

Fig. 3. Learning curves predicted by the general model for each task.

Differences in performance by sex. Female participants required, on average, more time than male participants to perform tasks (Table I).

Hypothesis C (Time limits of SHAP tasks)

Exceeded time limits in SHAP tasks. The time limits were exceeded in several tasks during the 8 measurement sessions (Table II). ADL tasks were the most affected: picking up coins, food cutting, opening/closing a zip, and rotating a screw 90°. In the picking up coins task, even in the last session the time limit was exceeded by half of the participants.

Exceeded time limits in tasks of FP. The percentage of participants that exceeded the time limits in 1 or several tasks of the prehensile patterns decreased during the sessions (Fig. 4a, b). For Lateral, the percentage decreased from 75% in the first session to 25% in the last session. For Tip, the decrement was from all participants (100%) to 29% (seventh session). Up to session 7, more than 21% of participants exceeded the time limits in at least 1 task that contributed to Tripod and Power.

All 26 tasks. None of the participants were able to complete all the tasks within the time limits in the first session (Fig. 4a). They improved during the sessions, but in the end in session 8 only 42% of participants succeeded in performing all the SHAP tasks within the time limits.

Fig. 4a-b. Percentage of participants exceeding the boundary time limits in one or several tasks per separate prehensile patterns of functionality profile, and in one or several tasks relative to all 26 SHAP tasks.

DISCUSSION

This is the first study that addresses the learning effects of SHAP. For all tasks, the learning effects were identified, and these were also reflected in the 6 prehensile patterns of the FP and of the IOF scores of novice prosthetic users, thus confirming Hypothesis A. These findings should make clinicians aware of the fact that using SHAP frequently for monitoring the functionality of a novice prosthetic user may induce learning effects. Therefore, the prehensile patterns and the IOF may not fully represent the actual functionality. To account for the learning effect and to ensure a steady performance, clinicians may be required to perform several baseline SHAP sessions for novice prosthetic users. Future studies should determine after which session the performance plateau is reached for the SHAP tasks.

One other study examined repeated SHAP performances (16), which revealed no learning effects. However, the design for that study used a single participant who had had extensive previous experience with SHAP and with the prosthetic hand. Learning effects were reported to have occurred in a variety of other measurement instruments for different body functions (17–20). These effects depended on the learning period, the number of trials, and the motivation of the participants when performing the tasks (17). We observed the motivation of our participants; they sought out optimal strategies in order to execute their tasks faster, which enhanced the learning effects. Undoing buttons, for example, requires fine manipulation skills (21), and we especially expected this task to pose more difficulties in simulator users. In this task, the participants rapidly learned how to perform the task more efficiently and with minimal use of the prosthesis. The participants secured the fabric in a specific position with their good hand and gripped the buttoned piece of the fabric from a certain angle, which made the unbuttoning process easier. Other examples of strategies that emerged during learning were orienting the knife slightly downwards and pressing firmly during cutting, holding the zip’s slider at different angles between the index finger and thumb in order to unzip by pushing with the thumb, and zipping up by pushing with the index finger.

Hypothesis B was also accepted based on the finding that there was a decrease in learning in the first session on each measurement day. Theories concerning motor learning describe this decrement as systematic short-time changes in performance that decrease proportionally with learning (11). Therefore, clinicians should take into account the fact that SHAP scores in the first session in a set of multiple sessions, or possibly in isolated sessions as well, may be affected by this “warm-up decrement”, and thus not represent the true functionality.

SHAP, by means of the speed in executing tasks, is deemed to be related to the dexterity of a person (22). Whereas the literature stresses that manual dexterity in women is better than in men (23, 24), the women in our study were slightly slower at performing the tasks. One possible explanation may be the fact that the simulator was relatively heavy, which inhibited less strong persons. The type of task may also induce sex-specific differences (25, 26). Men were found to be better at simple speed tasks, such as finger typing and grip strength (24, 25, 27), whereas women excelled at fine motor tasks, such as handwriting or using a pegboard (24, 28, 29). We found no interaction effect, however, for sex when it came to the type of task.

The time limits of several SHAP tasks were exceeded by more than 25% of our participants during at least the first 3 sessions, confirming Hypothesis C. The prehensile patterns of FP and the IOF were consequently affected, because the tasks that exceeded the time limits were rescaled to zero, according to our interpretation, and thus did not contribute any longer to the calculation of FPs and IOF. Time limits being exceeded may be due to the difficulty of the respective tasks. Difficult SHAP tasks have been previously identified (22, 30). The prehensile patterns and the IOF are thus represented by the remaining number of tasks, which for some prehensile patterns may be quite limited. For instance, when completing 2 tasks contributing to the Tip prehensile pattern, picking up coins and opening/closing a zip, participants frequently exceeded time limits and thus these no longer contributed to the calculations. As such, the Tip prehensile pattern might suffer from under-representation, since 2 of its tasks were zero and there were learning effects present in the tasks; these issues also affect the other prehensile patterns.

Lower scores for the Tip prehensile pattern compared with other prehensile patterns of the FP in prosthetic users have been reported previously (7, 22, 30) and may have been caused by limitations related to the design of the prosthetic hands (7). However, lower SHAP scores were also found for Tip in the more sophisticated I-Limb Pulse hand (31). This was unexpected, since the technical design of the I-Limb Pulse accounts for tip grip (31). According to our findings, another explanation for the lower scores in the Tip prehensile pattern might be the under-representation of Tip caused by exceeded time limits in tasks such as picking up coins and opening/closing a zip. Note that, for the prehensile patterns of FP and on the IOF, we were limited when interpreting the consequences of the exceeded time limits in more detail. A clear and detailed description of the formulas of the prehensile patterns and of the IOF is needed for such analysis, but the SHAP owners would not provide us with these data due to intellectual property issues. According to our statistical expert, who reviewed the existing information on SHAP, our current interpretation of time limits is plausible, but more details would be needed in order to be certain. Irrespective of the interpretation, the fact that the time limits were repeatedly exceeded by many of the participants remains an issue for SHAP in novice prosthetic users. Our research group provided some initial suggestions for improving SHAP. Recently, we submitted a proposal for a new, simplified calculation of SHAP scores, along with an alternative way to relate task time to the time limits1.

1Burgerhof JGM, Vasluian E, Dijkstra PU, Bongers RM, Van der Sluis CK. On a new scoring system for the Southampton Hand Assessment Procedure [submitted].

More research should also be allocated to accommodating the time limits for novice prosthetic users. The tasks in which time limits were repeatedly exceeded (picking up coins, food cutting, page turning, opening/closing a zip, and rotating a screw 90°) are the ones which are more difficult for prosthetic users, although not impossible. Our participants were able to perform the tasks within a longer time-frame relative to the normative values of unimpaired persons. In a post-hoc analysis, we extended the time limits to 10, 12, 14 and 16 times the normative data (Appendix IV). For page turning and rotating a screw 90°, the percentage of exceeded boundaries dropped considerably (8.3% and 12.5%) for a time limit of 16 times the norm, but not for picking up coins, food cutting, and opening/closing a zip (> 37%). Another solution would be to not use time limits. Time limits were originally imposed to prevent participants from taking excessive time (or being unable) to complete the tasks (14), which would increase the assessment time. However, removing the time limits would make a person’s execution time for each SHAP task directly comparable to the norm. This might be preferable, because including all SHAP tasks in calculating the functionality scores would enable the prehensile patterns of FP and of the IOF to be better represented.

This study has some limitations. The improvement seen in task times, quantified as faster performance, may have been the conjoint result of 2 factors: learning the SHAP tasks and novice users’ learning how to use the prosthesis. From the literature it is known that novice users learn rapidly (within 1 session) how to open and close a myoelectric prosthesis (13), but prosthetic force control is learned gradually (32). However, all SHAP tasks require mostly opening-closing skills on the part of the prosthetic user, except for the 2 tasks that also require force-control skills (pouring water from a carton and moving an empty tin can). We believe that our participants had already acquired basic manipulation skills during their familiarization with the simulator phase, which thus should have enabled completion of any SHAP task. Therefore, the performance of novices using a prosthetic simulator might not have influenced learning effects to any great extent. Given the strategies used to perform the observed tasks more quickly and the results of the learning curves presented (Fig. 3), we believe that studying novice prosthetic users did enable detection of learning effects in SHAP tasks. Nevertheless, we cannot rule out that the observed learning effects were partly due to the fact that the participants were novices with the myoelectric prosthesis. Another limitation of the study might be the fact that the participants did not use the prosthesis between the sessions, as would have been the case with prosthesis users. This practicing between measurement sessions might lead to less evident learning effects in SHAP tasks, although such effects have yet to be investigated. In not allowing unsupervised practice, our aim was the collection of reasonably unbiased results, free from the influences of practicing. The current study may be regarded as a first step in unravelling learning effects in SHAP, and future research should also study learning effects in experienced prosthetic users.

Although studying novice prosthetic users might be a limitation for this study, this also has its positive aspects, such as evaluating the appropriateness of time limits in novice prosthetic users. Furthermore, our participants had relatively comparable capabilities and experience with using a prosthesis, while they had no experience with SHAP. Finding experienced prosthetic users, all with rather equal characteristics of prosthetic use and with no experience with SHAP, would not be easy. Many rehabilitation teams use SHAP across the world. Using healthy participants allowed us to employ SHAP 8 times, which might not have been feasible with experienced amputees. Importantly, the simulator allowed us conveniently to study a larger number of participants.

The current study provided extended knowledge regarding SHAP learning effects in novice prosthetic users. However, to generalize these findings, learning effects should also be studied in different prosthesis users (experienced, occasional, or full-time users) and in other groups, or by including an older population, or people with sensory and motor pathologies, who might show different learning effects (33, 34).

Using a prosthetic simulator, we documented the learning effects in all SHAP tasks for novice prosthetic users. Clinicians should be aware of the fact that, when assessing functionality with SHAP in consecutive sessions, the improvement in functionality in prosthetic users may be partly due to the testing effects. In more difficult tasks, the time limits were exceeded in several sessions by more than 25% of participants. The FPs and the IOF, consequently, suffer from loss of information, and are determined by the rest of the tasks with scores within the time limits.

ACKNOWLEDGEMENTS

The authors wish to thank Suzanne Broeks and Sietske Romkema for contributions to the data collection; Johan Horst and Theo Schaaphok, OIM Orthopedie, for technical assistance with the simulator; and the participants for their participation.

The study was supported by the Netherlands Organisation for Health Research and Development – ZonMW (http://www.zonmw.nl/en/, Grant 60-62300-98-119). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

REFERENCES