Artificial intelligence in dermatology: A guide for clinicians to evaluate and compare models

Summary: Abstract Body: This study aims to address the growing need for clinicians in dermatology to effectively evaluate artificial intelligence (AI) and machine learning (ML) tools by providing a structured framework. Dermatology, with its basis in visual data and histopathological analysis, is well-positioned to benefit from AI and ML advancements. However, claims promising enhanced diagnostic and predictive capabilities must be approached critically. We developed a checklist encompassing five domains: model development, model selection, validation, performance evaluation, and real-world impact. The checklist was refined using 25 key publications and tailored for relevance in dermatological contexts such as lesion classification, outcome prediction, and diagnostic assistance. Three case-based examples demonstrate the checklist’s application in dermatology, emphasizing clinical utility. Key findings include gaps in external validation and limited use of explainability tools for dermatological AI models. Existing tools often lack fairness metrics and generalizability testing across diverse skin types and populations. For example, the checklist applied to a risk-prediction model showed the need for stratified validation to address skin phototype variability and evaluation when integrated with clinical workflows. Furthermore, randomized controlled trials are needed to quantify the impact of AI/ML tools on patient care. Our results highlight the importance of active participation in evaluating AI/ML models to ensure clinical relevance, transparency, and equity. This checklist provides a practical resource for dermatologists and clinicians, enabling informed decision-making and supporting the evidence-based integration of AI/ML into dermatological care. Nadia Siddiqui¹, Yazan Bouchi² 1. University of Washington School of Medicine, Seattle, WA, United States. 2. Morehouse School of Medicine, Atlanta, GA, United States. Bioinformatics, Computational Biology, and Imaging