Review

Accepted Oct 15, 2018; Epub ahead of print Oct 15, 2018

Acta Derm Venereol

Corr: Josip Car, Centre for Population Health Sciences, 11 Mandalay Road, Level 18 Clinical Sciences Building, Singapore 308232. E-mail: josip.car@ntu.edu.sg

Digital education is a promising new approach with advantages such as scalability, flexibility, portability and adaptability of education. This study synthesized  effectiveness evidence for health professions’ digital education in dermatology, and assessed whether it can improve knowledge, skills, attitudes and satisfaction as compared to traditional learning. We found 12 studies involving in total 955 health professionals. The main learning outcomes were comparable in terms of knowledge improvement, skills enhancement and satisfaction, suggesting the potential of digital health education to be used as a complementary or alternative method to traditional learning in dermatology. It has the potential to address the increased demand for dermatology education but requires further rigorous research to maximise its potential.

There is a growing burden of skin conditions among the general population (1–3). Several studies have shown that up to 7% of primary care consultations are for skin related complaints (4–6). At the same time, there is an increasing worldwide shortage of healthcare professionals, including dermatologists and dermatology nurses (7). The world is short of 17.4 million healthcare professionals (8) and this shortage is projected to remain, with a deficit of 14 million in 2030 (9). This shortage may be further accentuated by inadequate dermatology education; it is estimated that the formal dermatology education in the undergraduate medical and nursing curriculum represents only 0.24–0.3% of teaching time. Nonetheless, according to the American Academy of Dermatology’s (AAD) 2007 practice profile survey, dermatology practices have tended increasingly to employ dermatology nurse practitioners in order to augment dermatology care services (10, 11). As for medical students, one survey claimed that they received no more than 18 h of dermatological education in medical schools (10). This paucity of dermatology training is of concern as patients with skin problems are encountered in many clinical specialties including general medicine, paediatrics, venereology and general practice.

To fulfill the increasing need for dermatology education, it is necessary to provide high-quality teaching among pre- and post-registration health professionals (12). A recent study highlighted the potential for digital health education (DHE) in dermatology (13). In that study, digital education was found to significantly increase the effectiveness of dermatology learning, enhance the quality of education, and improve the teacher’s resources. DHE (also known as eLearning) is a broad construct that includes digital technology delivered or improved approach to teaching and learning in health-care which encompasses many different modalities offline and online, including virtual patient (VP), virtual reality environment (VRE), mobile learning (mLearning or mobile digital education), psychomotor skill trainers (PST), digital game-based learning (DGBL), Virtual Learning Environments (VLEs), Learning Management Systems (LMSs) and Massive Open Online Courses (MOOCs) (14).

DHE, compared to traditional intervention, has advantages such as flexibility, portability and especially, cost-effectiveness. It helps create an efficient and convenient learning mode for health professionals and students which further improves learning outcomes (15). DHE also enables healthcare professionals to learn and update their knowledge remotely without restriction of time and location (16). Moreover, DHE could potentially optimize resources utilization and reduce healthcare costs (17). Dermatology is a particularly suitable discipline for digital education due to its strong dependency on visual clues. Although many RCTs have evaluated the effectiveness of DHE few have focussed on dermatology. Therefore, in this systematic review, we critically evaluate the evidence for use of digital education of dermatology.

The review protocol was registered at PROSPERO (Prospero registration no. 42016051156; http://www.crd.york.ac.uk/prospero/).

We conducted our systematic review following Cochrane methods (18). Our review is part of a larger series of systematic reviews synthesizing evidence on different types of DHE (19–21). The databases were first searched in 2014 and the searches updated on a yearly basis, the most recent in August 2017. The following electronic databases were searched: Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (via Ovid), Embase (via Ovid), Web of Science, Educational Resource Information Centre (ERIC) (via Ovid), PsycINFO (Ovid) and CINAHL (via EBSCO). Reference lists of included studies and relevant SRs were also searched as potential sources.

Two authors (XX and GL) independently screened titles and abstracts to identify studies potentially meeting the eligibility criteria. Randomised controlled trials (RCTs), cluster RCTs (cRCTs) or quasi-randomised trials of all types of DHE or blended Education (DHE plus traditional education) were considered eligible in this SR. Cross-over trials including stepped wedge design were excluded due to high likelihood of carry over effect. For a study to be included, it had to compare the effectiveness of full DHE or blended education versus traditional or no education. If an RCT had more than one DHE intervention group, we compared the relevant DHE arm with the least active control arm (22). Based on the Health Field Education and Training of the International Standard Classification of Education(ISCED-F) (UNESCO), the pre- and post-healthcare professionals’ learning subjects were dental studies, medicine, nursing and midwifery, medical diagnostic and treatment technology, therapy and rehabilitation, and pharmacy (23).

Primary outcomes included (i) changes in dermatology knowledge (e.g. quantified differences in post-intervention scores), (ii) changes in clinical skills (e.g. pre- and post-test scores, time to perform a procedure, number of errors made whilst performing a procedure), (iii) changes in attitudes (e.g. pre- and post-intervention scores related to attitudes) and changes in learners’ satisfaction with the DHE intervention (e.g., Likert-like scales related to satisfaction). Secondary outcomes included (i) the economic outcomes of the intervention (e.g., cost-effectiveness, implementation cost, return on investment), (ii) outcomes related to patient care, including patients’ satisfaction, improvement in clinical signs and symptoms (e.g., remission rate, disease-related clinical and behavioural indices) and (iii) adverse/unintended effects of the interventions.

Two authors (XX and GL) independently extracted the data using a custom-made data extraction form. Cochrane’s Risk of Bias Tool (ROB) was used to assess the following domains: random sequence generation (selection bias); allocation concealment (selection bias); blinding of participants, personnel (performance bias); blinding of outcome assessment (detection bias); completeness of outcome data (attrition bias), selective outcome reporting (relevant outcomes reported); other sources of bias (baseline imbalances) (18). Judgements concerning the ROB for each study fell under 3 categories: high, low, or unclear risk of bias. This was done by 3 reviewers independently of each other. Any disagreements were resolved through a discussion and a third reviewer acted as arbiter (CS).

The quality of evidence was evaluated using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) criteria (24). The quality of the evidence was downgraded by 1 or 2 levels due to limitations in the study (ROB), inconsistency, indirectness, imprecision, and publication bias. Any disagreements were resolved through discussion with an arbiter (CS).

The initial searches yielded 6,676 potential studies of which 234 studies remained after the primary screening of the title and abstract. Screening the full text of these studies identified 12 trials that met our review criteria (22, 25–35) (see Fig. 1). All the included studies were RCTs and conducted between 2002 and 2016 in Brazil (28, 32), China (22), France (34), Kingdom of Saudi Arabia (26), Norway (27, 31), Sweden (35), Spain (33), US (29, 30) and UK (25). Participants ranged from year one undergraduates (22, 25, 26, 28, 30, 32, 33, 35) to certified physicians (29, 31, 34) and nurses (27). The sample sizes in the studies ranged from 46 (30) to 141 (34) and the duration of the intervention varied, from 15 min (28) to 6 months (31). The digital education included offline computer-based tutorials (22, 26, 29, 32, 35), online computer based tutorial (30, 31, 34) and computer-based learning-software (25, 27, 28, 33). The comparators were traditional education (22, 25, 26, 28, 30, 32, 33, 35), or no education (27, 29, 31, 34). Nine studies (75%) assessed the knowledge improvement by MCQ (multiple choice questions) (27, 30–32, 34), written examinations (22, 33, 35) and questionnaires (28). Five studies (42%) evaluated skill enhancement using grades of student performance subjectively assessed by tutors (22), objective examinations of clinical cases (25, 26, 29), checklist (10 lists with one score for each) and global assessment (9 assessments with a 1–5 score for each) (28). Satisfaction was evaluated in 5 studies (41.7%) using a variety of Likert-type scales and questionnaires (22, 26, 31, 32, 35) none of which were validated. None of the studies evaluated attitudes.

Judgments about each ROB item for each study are presented in Figs 2 and 3. Three studies (25%) (25, 26, 29) were judged to have unclear ROB for random sequence generation. Seven studies (58%) (22, 25, 26, 29, 30, 33, 35) were judged to have an unclear ROB for allocation concealment. Six studies (50%) (25–27, 29, 30, 35) were judged to have an unclear ROB and one study was judged to be a high ROB for blinding of personnel and participants. Seven studies (58%) (25–27, 29, 31, 33, 34) were judged to have an unclear ROB and one study was judged have a high ROB for blinding of outcome assessors. One study (8.3%) (27) was judged to have a high risk of ROB for incomplete outcome data. Aldridge et al.  (25) and Amri et al. (26) studies (100%) were judged to to have an unclear ROB for reporting bias while others were judged as a low ROB. Only one study (1/12) (27) was judged to have an unclear ROB in other bias domain due to baseline imbalance.

None of the studies mentioned a power calculation. For knowledge, the evidence was of very low quality due to the serious concern about the ROB, inconsistency, and indirectness, based on the results from 9 RCTs with 638 participants. For skills, the evidence was of low quality because of serious concern about the ROB and inconsistency among 5 RCTs with 338 participants. The evidence for the satisfaction of 5 RCTs with 380 participants was of very low quality due to the serious concerns of bias, inconsistency, and indirectness (Table SI).

Due to the high heterogeneity of the included studies, we did not employ meta-analytic techniques. Instead we used a narrative approach to synthesize the data. Detailed information of the included studies is shown in Table SII. Seven studies (22, 27, 30–32, 34, 35) found no difference in digital education, compared with traditional or no intervention, in improving post-intervention knowledge scores. Only two of the 9 studies (28, 33) found digital education resulted in a significant improvement in knowledge compared with a traditional learning intervention with low to unclear ROB (p < 0.01 (28); p < 0.001 (33)). A subgroup analysis by registration level revealed distinctions of learning outcomes. One of 4 studies showed significant difference among post-registration (nurse and physicians) (29). Among those studies with pre-registration healthcare professionals (medical students and nursing undergraduate students), 2 of 8 studies (25, 28) showed significant differences in knowledge improvement. Three studies (3/5) favoured digital education for skills enhancement compared with traditional or no learning intervention. Skill was demonstrated significantly better in flap skin surgery validation (checklist: p < 0.02; global assessment: p < 0.017) (28), skin cancer triage (diagnostic accuracy rate: p < 0.001; evaluation plan: p < 0.001) (29) and dermatological knowledge (diagnostic accuracy rate:p < 0.00001) (25), following 15 min or 10 days of digital education (software) (25, 28, 29). Five studies (22, 26, 31, 32, 35) measured satisfaction. Of these, two reported (22, 31) that digital education may have little effect or no difference on learners’ satisfaction (measured with Likert-like scales) when compared with controls (no education and traditional learning). In the remaining 3 studies (25, 32, 35) there was no comparison group. As for the attitude measurement, none of the included studies reported the attitude related outcomes as well as secondary outcomes, including economic outcomes, patient-related outcomes and adverse and unintended effects of the digital education.

The overall findings of this review are inconclusive due to the significant methodological, clinical and statistical heterogeneity of the included studies. The uncertainty of the results stems from inconsistency, imprecision, and indirectness in terms of diverse participants, educational interventions, different measurement instruments, or various comparison groups. Participants included were heterogeneous, ranging from medical students, healthcare professionals including specialists, primary care physicians, nursing students to registered nurses.

Interventions were also heterogeneous, ranging from offline learning (digital photos), software-based learning, online learning, to blended learning (combination of conventional learning and computer-based learning) (22, 27, 32–35). Comparison groups were homogeneous too. Eight out of 12 studies (22, 25, 26, 28, 30, 32, 33, 35) used traditional learning, while others had no intervention (education) group. Traditional learning interventions also differed including paper-based learning (22, 25, 26, 28) and lectures (30, 32, 33, 35).

Outcome measurement instruments were varied. Among included studies, knowledge improvement was measured with different instruments including MCQs, written examinations, and questionnaires. Similarly, the variability was also found in skills measurement tools. Objective clinical skills examinations (diagnosis accuracy measurements; disease recognition; management plan) were employed in 4 studies (22, 25, 26, 29) while subjective checklist assessment was used in De Sena’s study (28). For satisfaction, 4 studies used Likert-like scales, but the criteria assessed were different. The remaining study (35) used 4 point rating scale questionnaires. The variety of assessment tools employed by different studies hinder the comparison among the studies and weaken the interpretability of the findings (14, 21).

This study has several strengths including comprehensive searches, rigorous adherence to high methodological standards and critical appraisal of the evidence. Our comprehensive search strategy had no language restrictions and included a wide range of databases and peer-reviewed journals. However, several limitations should be kept in mind while interpreting its results. Included studies reporting sample sizes without power calculation left uncertainty of detecting an effect. Besides, non-randomized experiments are the most commonly used design for evaluation of digital education, but we exclude those in order to maintain the quality of the included studies.

It should be highlighted that blinding study participants and personnel to digital education is difficult as there is always a possibility of unidentified contamination between the study arms as digital education materials are easily accessible. Participants who are assigned to the control group may easily visit the webpage or download the app without the researcher’s awareness, resulting in potential performance bias. Moreover, learning content and learning theories were barely mentioned in the included papers. Unverified and incomprehensive learning content may mislead learners and even endanger patients’ health. Authoritative digital education practices and strategies emerging from different learning theories may contribute to better knowledge improvement, skill enhancement and higher satisfaction with less cognitive overload (36, 37). Theoretical education allows learners to optimize others’ learning experience, provides the possibility of deeper understanding of the learning materials and helps to connect the learning content with applications in practice. Besides, no economic outcomes were reported (28).

Our study demonstrated little or no improvement in dermatological knowledge, skills, attitudes and satisfaction among healthcare professionals when compared DHE to traditional learning or no interventions. The results are not specific for dermatology area but are also observed in other medical education setting including nursing, neurosurgery training, palliative care, dental education (14, 21, 38–41). Comparable results show the potential of DHE as a complementary or even alternative intervention to traditional learning. Given the flexibility, portability, adaptability and scalability of DHE, it could be considered as the promising solution to meet the increasing demand and address the shortage of highly trained dermatologists (42, 43).

Further research is needed to address the methodological limitations mentioned above. Also, there is a need to adopt learning theories in the design of future DHE interventions as well as employ power and sample size calculations. Moreover, to avoid contamination between groups, researchers need to track and follow-up control group participants to make sure they are not exposed to DHE. Furthermore, future studies should provide information on cost, cost-effectiveness and potential unintended effects of digital education.

The evidence for the effectiveness of DHE in dermatology is mixed and the overall findings are inconclusive mainly because of the predominantly very low quality of the evidence. More methodologically robust research is needed to further inform clinicians and policymakers.

We thank Ms. Chen Si for assistance with resolving the disagree-ments when screening the papers for inclusion in this review.

This study was funded by NTU Research Scholarship.

The authors have no conflicts of interest to declare.