Predicting malignancy in longitudinally monitored skin lesions with ai

Summary: Abstract Body: Background: Evolution is a trait of the deadliest type of skin cancer, melanoma, highlighting the need for expert dermatologic monitoring. AI is a non-invasive tool that can aid clinicians in cancer detection. While many algorithms detect skin cancer in single images, the benefit of monitoring calls for investigation into enhancing diagnostic accuracy with prior data. Objective: To develop an algorithm that estimates malignancy probability using current and prior images and compare performance to a single-image classifier. Methods: Dermoscopic images of 20,000 lesions collected over multiple visits were acquired from the ISIC Archive. Lesions were split into five folds with a 3:1:1 ratio of training/validation/test. A Swin Transformer pre-trained on ImageNet extracted features from each image.These features were used to train a cross-attention layer between lesion pairs from separate visits, followed by a classification head to estimate malignancy probability of each pair. Test predictions of each lesion were generated and pooled into a common dataset. AUCs were computed for the two-image model and a single-image classifier (Swin features + classification head). Results: The two-image model had a test AUC of 0.8501. The single-image model had a test AUC of 0.8494. While not statistically significant (p=0.9696 using DeLong’s test), the two-image model shows higher sensitivity (0.85-0.95) at lower false positive rates. Limitations: The data has few positive samples, the Swin Transformer was not fine-tuned, and sampling bias is inherent in retrospective data. Conclusions: Utilizing longitudinal data through a cross-attention mechanism improves performance in AI-based diagnostic classification at certain sensitivity/specificity thresholds. Maura C. Gillis¹, Nicholas Kurtansky¹, Jochen Weber¹, Katy Bell², Pascale Guitera^{2, 3}, Jennifer Dy⁴, Kivanc Kose¹, Veronica Rotemberg¹ 1. Dermatology Service, Memorial Sloan Kettering Cancer Center, New York, NY, United States. 2. Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia. 3. Melanoma Institute Australia, Wollstonecraft, NSW, Australia. 4. Electrical and Computer Engineering, Northeastern University, Boston, MA, United States. Bioinformatics, Computational Biology, and Imaging