Accepted Mar 23, 2021; Published Apr 29, 2021

Jrm-CC 2021; 4: jrmcc00056

Correspondence address: Erwin C. Baars, De Vogellanden, Centre for Rehabilitation, Hyacinthstraat 66a, 8001 BB Zwolle The Netherlands. E-mail: e.c.t.baars@vogellanden.nl

Doi: 10.2340/20030711-1000057

Using a checklist, this study assessed prosthesis user’s satisfaction with their prosthesis in persons with a below-knee amputation, and evaluated the potential benefits of using a checklist. A total of 82 persons with an amputation and 19 prosthesis specialists participated in completing the checklist and evaluation forms during the prosthesis-checking consultation. By using the checklist, more issues and problems were mentioned than prior to consultation, mostly concerning the prosthesis fit and the occurrence of pressure points on the residual limb. Evaluation showed that, by using the checklist, more information was gathered during the consultation concerning issues with which the prosthesis user was dissatisfied. Prosthesis users reported that using the checklist made them feel more understood by the prosthesis specialist.

Satisfaction with the prosthesis is an important goal in the rehabilitation of persons with a lower limb amputation. Satisfaction with a prosthesis is influenced by the characteristics of the prosthesis, including appearance, weight, smell, sound, use, function and fit, as well as the characteristics of the residual limb, such as pain and phantom pain (1). Sixty percent of patients with an amputation are not satisfied with their prosthesis, 57% are dissatisfied with the comfort, and 50% report pain during prosthesis use (2–4). In armed forces service members, dissatisfaction with the prosthesis resulted in rejection of 31% of prostheses prescribed (2). During consultation at an orthopaedic workplace, a certified prosthetist orthotist (CPO) checks the prosthesis. The CPO assesses prosthesis fit, use and function and takes an inventory of the prosthesis user’s problems and (dis)satisfaction with the prosthesis (5). In practice, this assessment is often not standardized (6). Standardization of the prosthesis checking process may improve the efficiency and quality of the consultation, which can be attained by systematically collecting prosthesis user’s information and experiences, including issues of dissatisfaction with the prosthesis and/or problems with the residual limb. In the field of surgery, use of a checklist resulted in a substantial reduction in complications and mortality, due to improved teamwork and communication between professionals (7–9).

The orthopaedic prosthesis industry has recently launched an initiative to facilitate communication between prosthesis users and CPOs, by providing printed cards with standard questions for the prosthesis user to ask during consultations (10). These cards contain general, non-specific questions; for instance, “Does it take long to learn how to walk with a prosthesis?” and “Can I choose how my prosthesis will look?”. It is unclear if and how these cards facilitate the CPO consultation.

The current study developed a checklist to systematically assess prosthesis user’s satisfaction with their prosthesis during CPO consultation (Fig. 1) and evaluated its added value. The aims of this study were 3-fold: (i) to systematically assess prosthesis user’s satisfaction with (aspects of) their trans-tibial prosthesis and problems with the residual limb by using a checklist during CPO consultation; (ii) to evaluate the potential benefits of checklist use as perceived by prosthesis users and CPOs; and (iii) to summarize issues and problems with the prosthesis and/or residual limb presented by prosthesis users during CPO consultation.

Checklist

A checklist, containing factors influencing satisfaction with the prosthesis, was developed, based on a systematic review (1) and input from prosthesis users and CPOs employed by 2 fitting centres in the city of Zwolle located in North East part of the Netherlands (ProReva Orthopaedic specialist and suppliers and OIM Orthopaedics). Factors were grouped in the following domains: prosthesis appearance, prosthesis properties, prosthesis fit, prosthesis use, and residual limb problems and pain (1). Evaluation forms containing statements for CPOs and prosthesis users were constructed to evaluate the use of the checklist. The checklist was pilot-tested among 2 CPOs and suggested changes were implemented. CPOs and prosthesis users evaluated the use of the checklist separately, by means of answering statements on a visual analogue scale (VAS; range 0–100 mm) (Appendix 1). For example, the statement: “I felt that I was more professional while using the checklist” was anchored with “0” meaning “less professional”, “100” meaning “very professional” and at midpoint “50” meaning “neutral/no difference”. Room for free-text comments was provided below each statement for the CPO to explain his or her VAS score. The prosthesis user evaluation form contained statements concerning the feeling of being understood and helped by the CPO and the speed of the consultation, again answered on a VAS (range 0–100) (Appendix 1). Finally, the prosthesis user was asked to give a general satisfaction score (range 0–10) for the consultation, with “0” meaning “very poor” and “10” meaning “perfect”.

Inclusion

CPOs with experience in lower limb prosthesis fitting from the 2 fitting centres mentioned above, were invited to participate. A meeting was held by the researcher (EB) informing the CPOs about the purpose of the study and how to use the checklist and evaluation form. They were asked to assess (dis)satisfaction with the prosthesis and problems with the residual limb by using the checklist for prosthesis users visiting the prosthetic workplace. Inclusion criteria for prosthesis users were: trans-tibial amputation level, age > 18 years, and sufficient understanding of the Dutch language to fill in the forms.

Procedure

Prosthesis users with a trans-tibial amputation were asked by a desk attendant to participate when consulting the prosthetic workplace. When persons agreed to participate, they received an envelope containing the checklist and 3 forms: (i) an informed consent form, (ii) a form assessing date of visit, reason for consultation, the presence of issues or problems with the prosthesis or residual limb and personal characteristics including age, employment, participation in sports activities and place of residence. Furthermore, a prosthesis user checklist evaluation section and a CPO checklist evaluation form were included.

During consultation, the CPO completed the checklist to assess the prosthesis user’s satisfaction with the prosthesis. After consultation, the CPO and the prosthesis user completed their evaluation form independently. The forms were collected by the desk attendant and placed in an envelope, which was sealed and stored for collection by the researcher. To stimulate participation by the fitting centres, regular e-mails were sent to location managers and printed cards with the text: “Think about using the checklist” were distributed among CPOs. Data were entered in a database, while the following rules were followed. When only the year of amputation was written on the form, the date in the database was set as 1 July of that year; when the year and month was written, the date in the database was set as the 15^th of that month in the given year. Regarding prosthesis use, when “whole day” was written on the form, 16 h of prosthesis use was noted in the database. The study was presented to the Medical Ethics Board (METC) of Isala Clinics Zwolle the Netherlands, and found that the Dutch Medical Research Act was not applicable (form number 171115).

Statistical analysis

SPSS (SPSS Inc. IBM Corporation, NY, USA) version 23 for Windows was used to analyse the data. Data were described in terms of percentage (%), mean, mean difference, standard deviation (SD) and median interquartile range (IQR) in case of skewed data distribution. A multilevel analysis was tried but the model did not converge, for some CPOs helped multiple prosthesis users while others helped only one. Hence CPO influences could not be analysed. A 1-sample t-test was used to analyse whether outcomes of the checklist evaluation differed significantly from 50 (the neutral value).

Checklist and forms

Two hundred envelopes with checklists (Fig. 1) and sets of evaluation forms were distributed among CPOs and 84 (42%) returned. One prosthesis user had 2 consultations, 1 was younger than 18 years old, and 1 of the distributed forms was not used; thus, 82 (41%) usable sets of forms were available and analysed. The participants’ characteristics are summarized in Table I.

Checklist use

A total of 52 prosthesis users mentioned a total of 93 issues or problems as reasons for consultation prior to CPO consultation. These issues mostly concerned prosthesis fit (39%). Thirty (32%) of these problems concerned satisfaction factors enclosed in the checklist (Table II). Sixty-three issues/problems and reasons for consultation were not included in the checklist. Other reasons for CPO consultation not mentioned in the checklist ranged from routine control visits, substitution of prosthesis components and prosthesis maintenance and prevention of malfunctioning. During CPO consultation and checklist use, 126 issues/problems were reported (Table III). Most dissatisfaction with the prosthesis concerned fit (33%), mainly regarding the movement of the residual limb in the socket (16%). In total 49% of prosthesis users were satisfied with all factors of the prosthesis, 12% were dissatisfied with 1 factor, 17% with 2 factors, 11% with 3 factors, 2% with 4 factors, 6% with 5, and 2% with 8 factors (total not 100% because of rounding off). Fifty two percent of prosthesis users (n = 43) reported residual limb problems, mainly concerning pressure points on the skin (n = 21). Twenty seven percent (n = 22) reported phantom pain and 4 persons had residual limb and phantom pain (Table III).

Thirty prosthesis users mentioned no issues or problems prior to CPO consultation. However, during assessment with the checklist, 15 of these persons (50%) were dissatisfied with one or more prosthesis factors. When asked, during assessment with the checklist, what the most important problem was, 29 prosthesis users mentioned a total of 36 factors related to dissatisfaction.

Evaluation of checklist use

Evaluation scores given by CPOs were significantly higher than 50 (p < 0.001) regarding the use of the checklist in helping them clarify with what issues the prosthesis user was dissatisfied concerning the prosthesis and residual limb, and that the checklist helped gather more information. In addition, the atmosphere of the consultation and contact with the prosthesis user was regarded as better with checklist use, and the CPO felt more professional. Some CPOs were less positive, and mentioned that using the checklist mainly cost more time (n = 11), had no extra value (n = 11), and hindered the consultation procedure (n = 2). Evaluation scores given by prosthesis users were significantly higher than 50 (p < 0.001) regarding use of the checklist during CPO consultation, and made them feel more understood by the CPO. Other prosthesis users mentioned that the consultation mainly took longer. The interpretation of VAS scores was sometimes difficult. Some CPOs did not comply with the instructions for VAS scoring of the statements. Some marked an “×” above the VAS line instead of a vertical line crossing the VAS line as instructed. Others gave a number above the VAS line, and still others used various symbols showing that instructions for VAS scoring were not interpreted correctly by all CPOs and prosthesis users (Table IV). Prosthesis users rated the consultation with a 7 or higher (mean 8.8 (standard deviation (SD) 0.9)).

Checklist use

The use of a checklist during prosthesis assessment identified more factors related to dissatisfaction in almost all domains than mentioned prior to CPO consultation. When the CPO asks for possible factors of dissatisfaction, the prosthesis user may be less reluctant to mention dissatisfaction than if they have to mention this beforehand, possibly explaining why more problems were identified during the use of the checklist. In addition, issues were systematically enquired for, with the checklist facilitating consultation. Prosthesis users can thus choose from a list of items rather than having to suggest issues themselves. The overall frequency of dissatisfaction with the appearance of the prosthesis was low. It is possible that the prosthesis user is influenced by the presence of the CPO, and thus reluctant to complain about the appearance of their prosthesis. Prior research into the patient-doctor relationship found a “satisfaction paradox”, meaning that patients often tolerate unsatisfactory aspects of healthcare, when care in general is satisfactory (11, 12). One participant mentioned having a skin allergy and itching prior to consultation, while no skin problems could be assessed using the checklist. Prosthesis users with skin allergy reactions may only experience itching without any visible skin redness or swelling. Thus, “itching” should be included in the checklist for future use. Relatively few “most important factors/ problems” were given while many problems and factors were assessed with the checklist (Table IV). This discrepancy may be because the prosthesis user regards all problems as being equally important when more than 1 problem existed, or the problem assessed is not that bothersome or important. Furthermore, results show that many reasons are given for CPO consultation besides issues concerning prosthesis user’s dissatisfaction. The checklist focuses on prosthesis satisfaction and residual limb problems. In assessing residual limb integrity, suggestions have been made regarding residual limb monitoring outside the clinician’s office using sensor-based technology incorporated in the socket to preserve an adequate socket fit and prevent problems (13). That research was conducted in 2 focus groups; 1 comprising 7 persons with an amputation on trans-tibial and trans-femoral levels with different aetiologies, and the other comprising CPOs with a minimum of 2 years’ experience in prosthesis care. In that study no systematic assessment of the prosthesis was conducted (13).

Evaluation of checklist use

Checklist use during the CPO consultation process was evaluated by CPOs and prosthesis users as being moderately positive. Differences in evaluation may be caused by the variety of problems that prosthesis users mentioned. In general, checklists have been used to achieve standardization of processes (14). The use of checklists can improve quality of healthcare by promoting systematic preparation of procedures and aiding memory during complex procedures and treatment outcomes in persons with lower limb amputations (7–9, 12, 15). The benefit of using the current checklist for assessing simple problems concerning the prosthesis or residual limb is probably limited, while for assessing complex multifactorial problems the checklist may be more beneficial. The latter, however, should be explored in future research. Also, the flexibility of the CPO in changing consultation routines may differ. Some seem to adhere more to their routine, making checklist implementation more difficult and time-consuming. Furthermore, the time available can differ per consultation, and using a new consultation procedure, such as a checklist, may be extra bothersome when little time is available. These findings show that checklist use sometimes had benefits for the prosthesis user in relation to clarifying problems and factors related to dissatisfaction. Prosthesis users rated the consultation with a mean of 8.8. This is in accordance with a study assessing patient satisfaction with emergency department characteristics, showing an overall mean satisfaction score of 7.7 (scale 0–10) (12). This result, however, should be interpreted with caution, for patient satisfaction scores are generally high and do not always reflect the quality-of-care services (16). Regrettably some CPOs did not complete the evaluation section at all, possibly due to lack of time. Also, some CPOs did not score the evaluation form correctly. One CPO wrote numbers ranging from 0 to 10 above the VAS line instead of correctly marking the line. More prior training in VAS scoring may have diminished interpretation difficulties (13). Some CPOs seemed to show patterns of scoring with consequent low or high scores.  Patterns in VAS scoring have been found in research  regarding quality of life. For example, a “midpoint bias” is known to occur, meaning that respondents tend to choose the middle category for scoring, while the precise value at midpoint is unclear, sometimes making comparisons of VAS scores between respondents difficult (17). In a multilevel analysis of our study the model did not converge, therefore we were unable to analyse CPO influences.

Study strengths

Evaluation of prosthesis satisfaction factors was analysed using a brief and comprehensive checklist, constructed with known factors influencing prosthesis satisfaction and input from prosthesis users and CPOs. In general, checklist implementation and use is improved when checklists are short and the user is involved in its construction (14). Checklist use was implemented in the regular consultation procedure and evaluated with a VAS directly following the consultation. This procedure provides for the CPO’s and prosthesis user’s impressions of the consultation, limiting self-selection of participants and loss of information. An open-text section was available so the prosthesis user could mention issues of dissatisfaction not included in the checklist. The study was performed in multiple locations in the north-east of the Netherlands, covering a large region of prosthetic limb care, and participating CPOs had 2–42 years of experience in prosthesis care.

Study limitations

Organizing checklist use in workplaces was challenging, for the organizational structure of these workplaces did not facilitate easy implementation of a new procedure. An informative session was held with location managers and CPOs, but checklist implementation nonetheless remained difficult. Difficulty in implementation is in accordance with results found during the implementation of checklists in high-reliability organizations, including the aviation industry and healthcare (9, 14).

Some CPOs seem to be more fixed in their consultation routine, making the use of a new procedure (checklist) difficult. This tendency was also seen in the checklist evaluation responses; some CPOs mentioned that checklist use increased the time needed and hindered the consultation without providing extra benefits. This finding may reflect the limited ownership and freedom to use common sense that the CPOs felt in using the checklist. Furthermore, not all forms were adequately filled in, possibly caused due to difficulties encountered during the consultation procedure, limited time available to complete the checklist, or limited motivation to complete the evaluation forms. Due to the range of prosthesis users treated by one CPO, CPO influence on filling in the forms could not be studied. In retrospect, it may have been useful to limit the number of prosthesis users treated by one CPO, in order to gain more insight into CPO influences on filling in the forms.

More prior training in checklist use and more involvement during its construction may have increased motivation and limited difficulties in checklist use during consultation (14). Participants with a trans-tibial amputation consulting orthopaedic workshops in the north-east of the Netherlands were included, limiting generalization of findings to a larger population of prosthesis users with other amputation levels and to lower limb prosthesis care in general.

Suggestions for future research and development

Many reasons for CPO consultation were found, ranging from routine control visits to replacement of defective prosthesis components and maintenance. Given the many reasons for CPO consultation identified, useful insight was gained into the relevant issues the prosthesis user mentions during CPO consultation. The checklist focuses mainly on factors associated specifically with prothesis satisfaction, and helps the CPO to target these factors systematically to improve the prothesis for the user and increase satisfaction. Further expansion of the checklist in the area of “reasons for consultation” may contribute to the efficiency of CPO consultation, but may distract the CPO from factors specifically related to prothesis satisfaction. When limiting the checklist to factors related to prosthesis satisfaction, the checklist should be expanded to a limited degree. “Itching” is a relevant factor to include in the residual limb domain of the checklist. Improvement in checklist implementation and compliance may be gained by closer involvement of CPOs in future checklist development and revisions (14). In addition, prosthesis users could complete the checklist prior to consultation, which could be achieved by making the checklist available online and requesting users to complete it at home before consultation. Prosthesis users would then have the opportunity to think about factors causing dissatisfaction, and the CPO could view the checklist and prepare for the consultation beforehand, increasing efficiency and saving time. In some cases, online consultation may be possible for minor problems, making a consultation visit unnecessary. In addition, the checklist could be incorporated into the electronic patient records for future reference.

Conclusion

Checklist use in systematic assessment of prosthesis satisfaction in trans-tibial prosthesis users is beneficial in identifying factors related to dissatisfaction. In the evaluation of checklist use, CPOs mentioned that checklist use facilitated them in identifying problems causing dissatisfaction with prostheses, and prosthesis users felt that they were understood better by the CPOs. The implementation of checklist use is, however, challenging.

The authors wish to thank the prosthesis users and the CPOs, from OIM Orthopaedics and ProReva Orthopaedic specialists and suppliers, for participating in this study.

The authors have no conflicts of interest to declare.