Ingela Ahnlide1, Mats Bjellerup1, Fredrik Nilsson2 and Kari Nielsen1

1Department of Dermatology, Helsingborg Hospital, Department of Clinical Sciences, Faculty of Medicine, Lund University, and 2Research and Development Centre Skane, Unit for Medical Statistics and Epidemiology, Skane University Hospital, Lund, Sweden

The ABCD rule of dermoscopy was developed to facilitate the dermoscopic differentiation between benign and malignant melanocytic lesions. However, there is a lack of studies on its validity in clinical practice. The aim of this study was to evaluate the accuracy of the algorithm used bedside, compared with the accuracy of the preliminary preoperative diagnosis, and to rate physicians’ level of confidence in the diagnosis. Melanocytic tumours were preoperatively scored bedside, according to the ABCD algorithm; 309 cases (46 melanomas and 263 naevi) were included. A sensitivity of 83% and specificity of 45% were found for the ABCD algorithm. A comparable sensitivity (74%), but a significantly higher specificity (91%), was found for the preliminary diagnosis. Interestingly, there was a considerable percentage (19.6%) of early melanomas for which a malignant diagnosis was not preoperatively expected, indicating that it is important to maintain generous indications for excision or to practise short-term follow-up of ambiguous lesions in order to detect early melanomas. Key words: dermoscopy; melanoma; melanoma in situ; dysplastic naevus; naevus; algorithms.

Accepted Sep 2, 2015; Epub ahead of print Sep 9, 2015

Acta Derm Venereol 2015; XX: XX–XX.

Ingela Ahnlide, Department of Dermatology, Helsingborg Hospital, Clinical Sciences, Lund University, S Vallgatan 5, SE-251 87 Helsingborg, Sweden. E-mail: ingela.ahnlide@skane.se

Dermoscopy is a non-invasive diagnostic tool in melanoma diagnosis, which, in the hands of experts, increases diagnostic accuracy (1). However, since dermoscopy requires training and experience (2, 3), a number of simplified algorithms for dermoscopic diagnosis of pigmented skin lesions have been developed (4–8). The first attempt to facilitate dermoscopic diagnosis for non-experts was the ABCD rule of dermoscopy (Table I), introduced by Stolz et al. in 1994 (8). In this algorithm a semi-quantitative scoring system, based on Asymmetry, Border, Colour and different Dermoscopic structures, leads to a total dermoscopy score (TDS) for each lesion. The higher TDS, the higher the risk of the lesion being a melanoma. The ABCD rule of dermoscopy has the benefit of being comparatively easy to memorize and to teach to physicians who are not experienced in dermoscopy. It has therefore been taught and used at our department for many years. Several previous studies have addressed the question of accuracy of the different simplified algorithms for dermatoscopy. The majority of these studies were, however, conducted in a more or less experimental setting using photographs.

For a dermatologist the preliminary (preoperative) diagnosis is based on the sum of all information gained, not only from dermoscopic examination, but also from the clinical history and physical examination of the patient.

The aim of this study was to examine the diagnostic value of the ABCD rule of dermoscopy in a clinical setting, and to compare it with the accuracy of the preliminary preoperative diagnosis. The physicians’ self-assessed confidence in his or her preliminary diagnosis was also assessed.

This study was approved by the Regional Ethical Review Board in Lund, Sweden (2011/195).

MATERIALS AND METHODS

The study was performed at the Department of Dermatology at Helsingborg Hospital, Helsingborg, Sweden. All skin tumours surgically excised at the department were preoperatively and consecutively registered in a registry integrated in a computerized patient file system (Melior®, Siemens AB, Upplands Väsby, Sweden) as a standardized patient file with fixed answer options. Registered data included the sex and age of the patient as well as tumour site and size. The registry has been described in detail previously (9). During the study period, 7 March 2013 to 28 April 2014, a total of 1,135 primary excisions were made for different benign and malignant skin tumour diagnoses, including both pigmented and non-pigmented skin tumours (see Fig. 1). Previously biopsied lesions and wide excisions were not included. Thirty-four cases were excluded because of an invalid report or missing data.

The hospital is a teaching hospital and, during the study period, 7 consultants and 6 residents in dermatology were working at the department. There were also visiting residents in family medicine working at the department for shorter periods; their registrations were excluded (66 cases).

Dermoscopy and the use of the ABCD algorithm of dermo­scopy, has been practiced by physicians at our department for more than 10 years. We have repeated joint feedback sessions evaluating the preoperative dermoscopy photographs of excised lesions, enrolment in dermoscopy courses for both residents and senior consultants and daily continuous education in dermoscopy for residents.

For clinically or dermoscopically pigmented skin tumours the physicians who made the decisions for surgery were encouraged to use dermoscopy and to score suspected melanocytic lesions according to the ABCD rule of dermoscopy (Table I). The recommended 2-step procedure was used, i.e. prior to application of the ABCD rule, non-melanocytic lesions and benign melanocytic lesions with special patterns (e.g. papillomatous, congenital naevi and mucosal lesions) were excluded. Thereafter the ABCD algorithm of dermoscopy was applied. During the bedside examination the physician noted the different values for asymmetry (0–2), border (0–8), colour (1–6), and dermoscopic structures (1–5). Preoperatively these data were entered in the computerized patient file. The TDS was computed automatically when the data were extracted.

It was mandatory for the physician who made the decision for surgery to register a preliminary preoperative diagnosis of the skin tumour. This was done after physical examination of the patient, including dermoscopic assessment.

The physician also registered his or her self-assessed confidence in the preliminary preoperative diagnosis on a 5-grade scale (5 = very confident and 1 = very unconfident). For discriminatory reasons the confidence level was multiplied by +1 if the physician had registered melanoma (in situ or invasive) as the preliminary diagnosis and by –1 for naevus (common or dysplastic). Consequently a confidence level of 5 was interpreted as showing that the physician was very confident that the lesion was a melanoma, while –5 meant that the physician was very confident that the lesion was a naevus.

All excised tumours were sent for histopathological analysis and assessment. A nurse registered the histopathological diagnoses postoperatively in the patient file. The histopathological diagnosis was regarded and registered as the correct diagnosis.

A total of 377 of the excised tumours had a histopathological diagnosis of common naevus, dysplastic naevus, melanoma in situ (MMis) or invasive melanoma (MM). Of these, 320 cases were preoperatively scored according to the ABCD algorithm, while 57 cases were not scored. Table II shows preliminary and histopathological diagnoses in the 57 cases not scored.

Cases with a preliminary diagnosis of lentigo maligna or Spitz naevus were not included, as the ABCD rule of dermoscopy is not suitable for assessing these diagnoses (5 cases) (10). Five cases, where the physician had made a non-melanocytic preliminary diagnosis but had still made a scoring, were regarded as invalid registrations and were consequently excluded, as was one case with ambiguous histopathology (Fig. 1).

Fig. 1. Included and excluded cases. CN: common naevus; DN: dysplastic naevus; MM: invasive melanoma; MMis: melanoma in situ.

Skin type was registered during the last 4 months of the study period, January–May 2014 (n = 54), during this period 92.6% (n = 50) of the included cases were skin type I–III.

Statistics

Data from the described surgical register were extracted and processed using the program QlikView® (QlikTech International AB, Lund, Sweden) and Microsoft Excel® (Microsoft Corp., Seattle, WA, USA).

The diagnostic accuracy, sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), positive likelihood ratio (LR+) and negative likelihood ratio (LR–) were calculated for the ABCD algorithm, with a cut-off value for the TDS value of >4.75 (TDS >4.75) and of >5.45 (TDS >5.45) separately, as recommended by Stolz et al. (8) (Table I). Thus the material was dichotomized; lesions with TDS values smaller than the cut off were dermoscopically considered naevus, while lesions with TDS values larger than the cut off were considered melanoma. Calculations were also made for the preliminary diagnosis; for these calculations, the material was dichotomized into naevi (including benign naevi and dysplastic naevi) and melanomas (including MMis and MM). As the residents were continuously encouraged to consult the specialists in difficult cases, the data were not analysed with respect to the level of experience of the physicians.

Diagnostic accuracy is defined as the proportion of correct classified lesions, the LR+ is the true positive rate divided by the false positive rate, and the LR– is the false negative rate divided by the true negative rate.

When a parametric test was suitable, means were compared using Welch’s t-test if there were 2 groups and an analysis of variance (ANOVA) when there were several groups. Post-hoc testing in the latter was done using the step-down procedure to produce p-values that maintain the family-wise error rate. These results are reported as difference in means.

For non-parametric comparisons between groups, we used the exact Wilcoxon-Mann-Whitney test if there were 2 groups, and the Monte Carlo exact test if there were more than 2 groups (different melanomas and different naevi). Where the latter was significant, post-hoc testing was done using the Nemenyi-Damico-Wolfe-Dunn test (joint ranks). The calculations were done in R-3.1.2 using the coin package 1.0–24 (11). In order to relate differences between groups to observable quantities we report non-parametric results as differences in medians with 95% bootstrapped confidence intervals (CIs) (10,000 replicates).

CIs for proportions are the Clopper-Pearson intervals; these guarantee 95% coverage. CIs for relative proportions (such as LR+ and LR–) were calculated using a log-link in a logistic regression. Comparisons between different sensitivities and specificities were done using the exact McNemar’s test with Bonferroni-Holm adjustments for multiple comparisons.

RESULTS

After exclusions, altogether 309 cases (46 melanomas and 263 naevi) met the inclusion criteria. Consultants assessed 66% (n = 205) of the cases and 34% (n = 104) were assessed by residents in dermatology. Fifty percent of the melanomas were invasive and the median Breslow thickness of MMs was 0.7 mm (mean 1.34 mm, interquartile range 0.50–1.35). Mean and median ages at diagnosis, sex, tumour site and size, and TDS for the included diagnoses are shown in Table III. The mean diameter of all melanomas (in situ and invasive) (11.3 mm) was significantly larger than the mean diameter of naevi (common and dysplastic) (6.95 mm) (p < 0.001) (difference 4.93, 95% CI: 5.72–3.07). The median TDS values differed significantly between melanomas (in situ and invasive) and naevi (common and dysplastic) (p < 0.018), but not between in situ and MMs (p = 0.393) nor between common naevi and dysplastic naevi (p = 0.617) (see Table IV). The distribution of the TDS values, rounded to the nearest integer, in the 4 included diagnostic groups is shown in Fig. 2, illustrating that the individual TDS values overlap considerably between diagnoses.

Fig. 2. Distribution of the total dermoscopy score rounded to the nearest integer.

The sensitivity for the ABCD algorithm with cut-off values for TDS of >4.75 and >5.45 was 83% (95% CI: 69–92%) and 74% (95% CI: 59–86%), respectively (Table V). The sensitivity for the preliminary preoperative diagnosis was 74% (95% CI: 59–86%). The highest specificity, 91%, was seen for the preliminary diagnosis, compared with the ABCD algorithm (67% for TDS >5.45 and 45% for TDS >4.74). The diagnostic accuracy was significantly higher for the preliminary diagnosis (89%, 95% CI: 85–92%) compared with both TDS >4.75 (51%, 95% CI: 45–56%) and TDS >5.45 (68%, 95% CI: 63–73%) (both p < 0.0001). The positive likelihood ratio (LR+) for TDS >4.75 was 1.51 (95% CI: 1.24–1.77), compared with 8.45 (95% CI: 5.60–13.3) for the preliminary preoperative diagnosis.

The physicians’ self-assessed confidence in the preliminary diagnosis is shown in Table VI. In 9 out of 46 melanomas (19.6%, 95% CI: 9.4 – 34%), the physician was fairly confident (–4) that the lesion was a naevus. The mean TDS value for this group was 5.1 (median 5.6, range 3.3–6.2). Four of the 9 cases had TDS values > 4.75. All of the cases were early melanomas: 8 were MMis and one was an early MM (Breslow thickness 0.5 mm, with a 4-mm diameter and a TDS value of 3.8). The self-assessed confidence and the different preliminary and final diagnoses are shown in Table SI1.

Finally, we compared the physicians’ confidence in the preliminary diagnosis and the predictability of a correct diagnosis made between MMis and MM. There was a significant difference (p = 0.018), i.e. the physicians were more confident about and more correct regarding the preliminary diagnosis of MM, than they were regarding diagnosis of MMis.

DISCUSSION

In this study on correctly diagnosing melanoma based on bedside use of the ABCD rule of dermoscopy on pigment tumours scheduled for surgical excision, we found that the ABCD algorithm as a diagnostic tool showed fairly high sensitivity, but low specificity. The diagnostic accuracy for the ABCD rule of dermoscopy was significantly lower than the diagnostic accuracy for the preliminary preoperative diagnosis of melanoma. This was due to a substantially higher specificity for the preliminary diagnosis. The preliminary diagnosis, defined as the physicians’ single most probable preoperative diagnosis, was based on all accessible clinical and dermoscopic information, i.e. patient and lesion history and physical examination, including dermoscopy.

When dermoscopy was introduced in the late 1980s, the different dermoscopic criteria for assessing pigmented skin lesions and discriminating melanomas from non-melanomas were described (12). Diagnosis made by simultaneous assessment of these morphological criteria is known as “pattern analysis”. This qualitative approach has been shown to require experience to enhance the diagnostic ability (2). Algorithms for dermoscopy (e.g. the ABCD rule of dermoscopy, Menzies’ scoring method, and the 7-point checklist of dermoscopy) have been developed to make dermoscopy easier for non-experts (4, 5, 8) and have been shown to be useful for this purpose (13, 14). The different algorithms are based on the same principles; benign naevi are symmetrical and exhibit few different dermoscopic structures, while melanomas are “chaotic”, i.e. asymmetrical, exhibiting many different structures. Several studies have addressed the question of which of the algorithms has the best diagnostic accuracy (5, 14–17). A meta-analysis in 2002 showed no significant difference between different algorithms (3). In a consensus meeting among experts in 2003 addressing the same question, the pattern analysis allowed the best diagnostic performance, with the simplified algorithms showing similar sensitivity, but lower specificity (18).

However, several factors affect the results when studying diagnostic accuracy of algorithms. There are intrinsic limitations in having experts evaluate and compare the accuracy of different algorithms, since the assessments will inevitably be influenced by their expertise and experience. When an image is presented to an expert, recognition of patterns will start immediately and a diagnostic hypothesis will take shape (19). Application of rules or algorithms will be affected by the first impression. Not only the diagnosis of the lesion, but also the assessment of the presence or absence of certain algorithmic criteria will be influenced (19).

Meta-analysis showed that the performance of the algorithms improved when the diagnosis was made in consensus between observers. It diminished as the prevalence of melanoma in the test materials increased (presumably indicating more difficult test samples) (3).

Another factor that may affect the results is whether the study setting is retrospective, based on photographs magnifying the lesions to different degrees, or the diagnosis is made bedside on the patients. In the bedside situation the patient’s level of anxiety may affect the decision-making, as may the fact that the assessment is made under pressure of time (20). In our study the different physicians made the assessments separately, at the bedside with no extra time allocated for the evaluation. Hence, the results of this study reflect the value of the ABCD algorithm in the clinical routine.

When Stolz et al. (8) introduced the ABCD rule of dermoscopy in 1994, they reported a sensitivity of 97.9% and a specificity of 90.3%; however, these results were achieved by experts making assessments based on photographs. A study by Nachbar et al. (21), published in the same year, attempted to assess the prospective value of the ABCD rule of dermoscopy. The scoring was done bedside and achieved a sensitivity of 92.8% and a specificity of 91.2%.

There is a striking difference in specificity between Nachbar et al.’s study (21) and the present study. This may at least partly be explained by differences in selection of study materials. In the study by Nachbar et al., the assessed pigment lesions were selected for surgery based on 2 different criteria. Lesions with a score <4.75 were excised at the patient’s request, while lesions with a score >4.75 were excised at the doctor’s advice, indicating that the study material consisted of 2 distinct groups. At our department, excisions at patient’s request, e.g. for mechanical or cosmetic reasons, are exceptional. Therefore, the absolute majority of lesions included in our study were excised at the doctors’ advice. This implies that there was a difference in studied populations between the 2 studies. Our study included only a few clinically clearly benign lesions. As previously proposed by others, it is reasonable to assume that the diagnostic accuracy, especially the specificity, of the algorithms, decreases when they are used to differentiate highly atypical lesions from thin melanomas (17). Most dermoscopic algorithms used today have been created to help clinicians in diagnosing pigmented MMs and not MMis. Today more than 40% of all diagnosed melanomas in Sweden are MMis, compared with 1996 when approximately 20% of melanomas were in situ (22). This implies that clinicians find melanomas at earlier stages today. A retrospective study on melanomas diagnosed during dermoscopic follow-up, showed that some very early melanomas are featureless and cannot be discerned from melanocytic naevi by any dermoscopic algorithm at first presentation (23).

What accounts for the higher specificity of the preliminary preoperative diagnosis compared with the diagnosis based on the ABCD rule of dermoscopy? When making the clinical assessment, leading to the preoperative preliminary diagnosis in this study, the physician could take into consideration both the dermoscopic examination, and clinical information such as the patient’s age, skin type, total number of naevi, signs of sun damage, and familial and personal history of melanomas, as well as information pertaining to the lesion, such as ugly duckling sign (24) and changes over time (25, 26). With the current study design, we do not know whether the physicians when applying the dermoscope, in addition to applying the ABCD algorithm, also used pattern recognition or indeed another dermoscopic algorithm and whether this contributed to the higher specificity for the preliminary diagnosis (27, 28). Nevertheless, it is reasonable to assume that the added clinical information accounts for a major part of the higher specificity seen for the preliminary diagnosis (25).

The ABCD algorithm was developed as a tool for making the decision about whether to excise a lesion. Consequently, it might be more accurate to compare the results of the algorithm with the clinical decision to excise, rather than with the most probable preliminary diagnosis provided. Since this study includes only excised lesions, in that case the results of the algorithm should be compared with all included lesions. From that perspective, strict application of the ABCD rule of dermoscopy to this material would have meant that 8 out of 46 melanomas would have been missed, but 119 excisions of benign naevi prevented.

A unique part of the present study was the self-assessed confidence in the chosen preliminary diagnosis. The physicians had to self-assess their confidence in the preliminary diagnosis on a 5-grade scale. The most interesting finding in this part of the study was that in 9 out of 46 melanomas (19.6%), the physician was preoperatively relatively sure (graded 4 on a 5-grade scale) that the lesion was a naevus, i.e. a malignant diagnosis was not expected. It is worth noting that all the misdiagnosed cases were very early melanomas. In 8 out of the 9 cases the final diagnosis was a MMis, and in 1 case the diagnosis was a small early MM with TDS 3.8 (29). This shows, as expected and discussed above, that diagnosing early melanomas causes most diagnostic difficulties to the physician (23). However, since this study includes only excised tumours, these misdiagnosed melanomas were excised and consequently not missed.

The present study has several limitations. Firstly, as mentioned, the study design included only excised lesions. Lesions that were judged as unambiguously benign were not excised and therefore never included. The results of the study are therefore potentially influenced by verification bias, i.e. the decision to proceed to the reference test (histopathology) partly depends on the methods that are analysed (ABCD rule of dermoscopy vs. preliminary diagnosis). Secondly, the histopathology slides, used as reference method, were read by different pathologists at the local pathology laboratory, according to our normal clinical routine. One benefit of this is that the results in the study reflect the ordinary clinical situation; on the other hand, it is well known that there are elements of subjectivity and interobserver variability in the reading of histopathology slides (30). Thirdly, the excised lesions were not photographed consistently. It would have been of value if the included physicians could have made a blinded evaluation of the dermoscopic criteria on the corresponding photographs to compare results between doctors, with the individual assessment made bedside and to be able to compare our results with previous studies.

In conclusion, this study showed a fairly high sensitivity, but a low specificity for the ABCD rule of dermoscopy when it is used bedside in a clinical setting. The study also indicates that clinical data add to specificity in preoperative clinical assessment in melanoma diagnosis. Further prospective studies on the value of different dermoscopic algorithms in a clinical setting are needed to answer the question whether the specificity for diagnosis based on dermoscopy would have been better using a different algorithm. It would also be of interest to compare the dermoscopic evaluations made bedside with blinded assessments on corresponding photographs to further address this question.

Finally, the study showed a considerable percentage of very early melanomas that were preoperatively not expected to be melanomas by the dermatologist. This indicates that in order to detect early melanomas it is important to retain generous indications for excision or to practise short-term follow-up of ambiguous lesions.

ACKNOWLEDGEMENTS

This study was supported by the Skane County Council’s Research and Development Foundation, Governmental funding of clinical research within the National Health Services (ALF), the Thelma Zoéga Foundation, Hudfonden, and the Stig and Ragna Gorthon Foundation. The authors would like to thank colleagues and staff at the Department of Dermatology, Helsingborg.

1http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-2239

REFERENCES