AI in Radiation Oncology Consultations: Boosting Efficiency and Quality in HPI Documentation

Summary: As the demand for radiation oncology services increases, the availability of resources remains limited. Integrating new technologies, such as artificial intelligence (AI), is essential for evolving clinical workflows and meeting this demand. This study evaluates the impact of AI on automating the history of present illness (HPI) section of consultation notes in radiation oncology. By leveraging large language models (LLMs), the study explores whether AI can enhance efficiency while maintaining or improving documentation quality, thereby reducing clinicians' administrative burdens. We assessed the quality and efficacy of a commercial AI documentation system in radiation oncology. The system processes batches of intake documentation or scanned images (e.g., referral documents, pathology reports, laboratory reports, imaging reports) using customized optical character recognition (OCR) to extract relevant structured data for the HPI. A commercial, HIPAA-compliant LLM then generates the HPI from the extracted data. To validate this approach, we evaluated two aspects of the workflow for 13 prostate cancer patients: (1) the quality of data extracted via the OCR system, and (2) the quality of the AI-generated HPI using six common LLM metrics: faithfulness, hallucination, bias, tone alignment, keyword presence, and answer relevance. Our comparison revealed that the OCR system was 97.3% accurate in data extraction, and the AI-generated HPIs demonstrated strong factual consistency, minimal misinformation (<1% hallucination rate), neutrality (0% bias), 100% tone alignment, 84% keyword presence, and 97.3% answer relevance. These findings indicate that the AI-driven workflow excels in extracting structured data from diverse document types and consistently achieves high-quality HPI documentation. The use of AI in managing patient documents and generating HPIs for radiation oncology consultations shows significant potential to reduce documentation time without compromising quality. Future studies will expand to include more disease sites and test cases, incorporating human oversight to refine and verify extracted data for clinical use. Authors: Y. Elnady 1, C. SMITH 1, L. Mancuso1, M. Terry1, and C. Jahraus2 1 Fuse Oncology 2 Generations Radiotherapy & Oncology PC